-- Logs begin at Tue 2022-09-27 09:07:44 UTC, end at Tue 2022-09-27 11:34:28 UTC. -- Sep 27 09:37:40.966995 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Sep 27 09:37:40.978133 ncdv-master-1 devstack@keystone.service[104120]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 27 09:37:40 2022] *** Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: nodename: ncdv-master-1 Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: machine: x86_64 Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: clock source: unix Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: pcre jit disabled Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: detected number of CPU cores: 4 Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: current working directory: / Sep 27 09:37:40.980178 ncdv-master-1 devstack@keystone.service[104120]: detected binary path: /usr/bin/uwsgi-core Sep 27 09:37:40.980805 ncdv-master-1 devstack@keystone.service[104120]: your processes number limit is 47680 Sep 27 09:37:40.980805 ncdv-master-1 devstack@keystone.service[104120]: your memory page size is 4096 bytes Sep 27 09:37:40.980805 ncdv-master-1 devstack@keystone.service[104120]: detected max file descriptor number: 2048 Sep 27 09:37:40.980805 ncdv-master-1 devstack@keystone.service[104120]: lock engine: pthread robust mutexes Sep 27 09:37:40.980805 ncdv-master-1 devstack@keystone.service[104120]: thunder lock: enabled Sep 27 09:37:40.980805 ncdv-master-1 devstack@keystone.service[104120]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 27 09:37:40.980805 ncdv-master-1 devstack@keystone.service[104120]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 27 09:37:41.025527 ncdv-master-1 devstack@keystone.service[104120]: Python main interpreter initialized at 0x55ec2746b9c0 Sep 27 09:37:41.025527 ncdv-master-1 devstack@keystone.service[104120]: python threads support enabled Sep 27 09:37:41.025527 ncdv-master-1 devstack@keystone.service[104120]: your server socket listen backlog is limited to 100 connections Sep 27 09:37:41.025527 ncdv-master-1 devstack@keystone.service[104120]: your mercy for graceful operations on workers is 90 seconds Sep 27 09:37:41.025958 ncdv-master-1 devstack@keystone.service[104120]: mapped 403077 bytes (393 KB) for 2 cores Sep 27 09:37:41.025958 ncdv-master-1 devstack@keystone.service[104120]: *** Operational MODE: preforking *** Sep 27 09:37:41.026265 ncdv-master-1 devstack@keystone.service[104120]: *** uWSGI is running in multiple interpreter mode *** Sep 27 09:37:41.026265 ncdv-master-1 devstack@keystone.service[104120]: spawned uWSGI master process (pid: 104120) Sep 27 09:37:41.026496 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Sep 27 09:37:41.027587 ncdv-master-1 devstack@keystone.service[104120]: spawned uWSGI worker 1 (pid: 104131, cores: 1) Sep 27 09:37:41.027587 ncdv-master-1 devstack@keystone.service[104120]: spawned uWSGI worker 2 (pid: 104132, cores: 1) Sep 27 09:37:41.027587 ncdv-master-1 devstack@keystone.service[104120]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 27 09:37:45.641394 ncdv-master-1 devstack@keystone.service[104131]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:37:45.641394 ncdv-master-1 devstack@keystone.service[104131]: from cryptography.utils import int_from_bytes Sep 27 09:37:45.642235 ncdv-master-1 devstack@keystone.service[104131]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:37:45.642235 ncdv-master-1 devstack@keystone.service[104131]: from cryptography.utils import int_from_bytes Sep 27 09:37:45.651276 ncdv-master-1 devstack@keystone.service[104132]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:37:45.651276 ncdv-master-1 devstack@keystone.service[104132]: from cryptography.utils import int_from_bytes Sep 27 09:37:45.651808 ncdv-master-1 devstack@keystone.service[104132]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:37:45.651808 ncdv-master-1 devstack@keystone.service[104132]: from cryptography.utils import int_from_bytes Sep 27 09:37:45.697337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 09:37:45.697617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 09:37:45.697844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] command line args: [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 09:37:45.698107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 09:37:45.698236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ================================================================================ {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 09:37:45.698527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] admin_token = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.698772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] config_dir = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.698937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.699147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] config_source = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.699278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] debug = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.699517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.700954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.700954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] insecure_debug = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.700954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.700954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.700954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] list_limit = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.700954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] log_config_append = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.701313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.701313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] log_dir = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.701432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] log_file = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.701658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.701908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.702115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 09:37:45.702202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.702392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.702502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 09:37:45.702573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.702637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] command line args: [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 09:37:45.702696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.702879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 09:37:45.702987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.703051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ================================================================================ {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 09:37:45.703168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.703298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] admin_token = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.703426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.703559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] config_dir = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.703702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.703838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.703838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] config_source = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] debug = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] notification_format = cadf {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] insecure_debug = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.704965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.705172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.705306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.705306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] list_limit = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.705507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.705624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] log_config_append = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.705761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] publish_errors = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.705987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] log_dir = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] log_file = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.706970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] standard_threads = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] strict_password_check = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] use_eventlog = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.707823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] use_journal = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.708045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.708161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] use_json = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.708276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.708385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] use_stderr = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.708385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] use_syslog = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.708385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] watch_log_file = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.708758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.708888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.709006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.709116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.709224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.709340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.709449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.709558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.709666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.709774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] notification_format = cadf {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.709889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.710067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.710181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.710181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.710383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.710498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.710615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] publish_errors = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.710724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.710838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.710954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.711064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.711064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.711276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.711416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.711552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.711669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.711785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.711903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] standard_threads = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.712019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.712135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] strict_password_check = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.712236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.712328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.712431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.712524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] use_eventlog = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.712615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.712745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] use_journal = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.712745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] use_json = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.712967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.713091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] use_stderr = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.713202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.713310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] use_syslog = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.713418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.713532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] watch_log_file = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:37:45.713641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.713757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.713757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.714887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.715960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.716990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.717981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.718912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.719989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.720962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.721914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.722970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.723156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.723413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.723531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.723640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.723755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.723755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.724924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.725957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.enabled = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.726978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.727979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.728930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.729945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.730956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.enabled = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.731937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.732992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] auth.external = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] auth.mapped = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.733999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] auth.password = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] auth.token = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.734891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] catalog.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.735961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] credential.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] credential.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.736879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.737900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] auth.external = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.738684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] auth.mapped = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.739001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.739001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.739343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.739463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.739463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.739768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.739937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] auth.password = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] auth.token = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] catalog.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.740982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.741897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] credential.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] credential.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.742923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.743928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.744961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.745946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.746946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.747968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.748943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.749957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.750964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.password = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.751982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.752995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.753888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.754000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.754110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.754262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.754376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.754579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.754798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.754920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.755954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.756901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.password = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.757922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.758033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.758145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.758264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.758473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.758583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.758707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.758870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.759997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.760918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] policy.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.761928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] receipt.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.762942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.763956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.764967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] revoke.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] role.cache_time = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] role.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.765987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] role.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] role.list_limit = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.766976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] policy.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.767941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] receipt.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.768894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.769888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.770008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.770177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.770292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.770586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.770709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.770966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] revoke.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.771972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] role.cache_time = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] role.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] role.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] role.list_limit = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.772982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.cache_time = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.773974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.provider = fernet {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.774988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.775900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] trust.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.776972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.777908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.connection = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.778922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.779932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.cache_time = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.780948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.provider = fernet {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.781961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.782071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.782192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.782304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.782414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.782524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 09:37:45.782634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.782634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] trust.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.782858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.783339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.783582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.783826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.784107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.784334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.784575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.784864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.785126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.connection = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.785371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.785615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.786015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.786783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.787020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.787258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.787495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.787732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.787977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.788211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.788455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.788707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.788962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.789197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.789438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.789666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.789904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.790138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:37:45.790304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=104132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 09:37:45.812873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.813094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.813318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.815911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.816143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.816366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.821904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.825532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.851963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.852181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.854580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.854818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.905165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.905662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.954476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.954679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.956092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:45.956325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.022346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.022553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.022781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.022996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.023122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.023203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.023358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.023438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.023592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.023833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.024051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.024269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.047176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.047379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.047666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.047830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.047954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.048950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.049160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.060065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.060216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.073254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.073512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.073643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.073728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.073824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.073893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.073955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.074126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.074220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.074376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.074478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.074590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.074668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.074731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=104132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 09:37:46.091322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.091520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 09:37:46.091609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.091821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 09:37:46.091905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.092063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 09:37:46.092156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.092340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.092425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 09:37:46.092601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.092714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.092885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.092982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.093141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.093223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.093392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.102712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.102990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.103263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.103494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.103626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.103756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.104047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.104333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.110528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.111299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.113439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.113720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.114054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.114332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.114643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.114767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.114851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.114940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.115129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.115256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.115374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.115466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.115671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.115798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.115955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.116082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.116284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.116409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.116506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.116839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.117098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.117472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.140571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.142058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.142184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.142403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.143525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.143807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.147085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.147371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.147654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.149171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.149456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.149734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.154127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.154415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.156173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.156403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.158890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.159185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.159484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.159774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.160048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.160319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.160601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.160938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.161086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.161665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.161946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.162249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.162529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.162802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.163077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.163353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.176521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.176884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.179678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.179959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.180726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.183795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.183938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.184250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.187032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.187329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.188567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.191525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.191732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.195148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.195385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.195516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.195724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.196042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.196287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.196572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.198550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.198851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.199141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.201040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.201040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.201040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.212783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.213091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.213357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.213642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.216361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.216683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.216935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.217227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.220481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.222054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.224023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.224161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.224461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.225941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.226804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.227128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.227397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.227512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.231409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.231550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.231659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.231761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.231761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.231761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.231761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.231761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.239857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.240152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.240438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.240734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.241133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.241433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.241729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.243304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.243585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.243871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.244158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.244548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.244861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.245154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.257174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.260055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.260997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.263030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.263879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.264459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.265954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.266240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.266648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.266882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.268309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.269795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.270077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.270970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.273213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.274673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.277333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.277578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.278965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.279089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.279226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.279519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.281966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.284290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.284290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.284290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.287740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.288029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.288319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.292302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.292612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.292920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.295201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=104131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.295486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.295766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.296039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.296336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.296608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.296909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.297187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.297479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.297749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.298052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.298329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.298592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.298849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=104131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.299766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=104132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 09:37:46.300048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.300336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.300607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.300904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.301184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.301519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.302332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.302663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.303007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.303281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.303734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.303830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.304067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=104132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 09:37:46.332164 ncdv-master-1 devstack@keystone.service[104131]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ec2746b9c0 pid: 104131 (default app) Sep 27 09:37:46.335690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059961d-66f8-4d18-9ba8-326e9eadb374 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:46.335980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059961d-66f8-4d18-9ba8-326e9eadb374 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:46.336252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059961d-66f8-4d18-9ba8-326e9eadb374 None None] PATH_INFO: `/v3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:46.338324 ncdv-master-1 devstack@keystone.service[104132]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ec2746b9c0 pid: 104132 (default app) Sep 27 09:37:46.340506 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Tue Sep 27 09:37:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:37:53.830136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761f841-84c8-4ed5-9578-ca5d1d4d1ff7 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:53.830683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761f841-84c8-4ed5-9578-ca5d1d4d1ff7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:53.830683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761f841-84c8-4ed5-9578-ca5d1d4d1ff7 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:53.835313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:37:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:37:53.842538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:53.843437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:53.843437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:53.931832 ncdv-master-1 devstack@keystone.service[104131]: INFO dbcounter [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] Registered counter for database keystone Sep 27 09:37:53.951883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG oslo_db.sqlalchemy.engines [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104131) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:37:53.961745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writer thread running {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:37:54.168555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG passlib.handlers.bcrypt [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=104131) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 27 09:37:54.168998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG passlib.handlers.bcrypt [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=104131) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 27 09:37:54.230803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:37:54.232332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:37:54.388596 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:37:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 27 09:37:54.737988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761f841-84c8-4ed5-9578-ca5d1d4d1ff7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:37:54.814645 ncdv-master-1 devstack@keystone.service[104132]: INFO dbcounter [None req-0761f841-84c8-4ed5-9578-ca5d1d4d1ff7 None None] Registered counter for database keystone Sep 27 09:37:54.837381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG oslo_db.sqlalchemy.engines [None req-0761f841-84c8-4ed5-9578-ca5d1d4d1ff7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104132) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:37:54.845070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writer thread running {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:37:54.972553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761f841-84c8-4ed5-9578-ca5d1d4d1ff7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:37:54.977322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5829f-368a-4f68-87be-86fd8b50c34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:37:54.977989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5829f-368a-4f68-87be-86fd8b50c34e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:54.978260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5829f-368a-4f68-87be-86fd8b50c34e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:54.978537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5829f-368a-4f68-87be-86fd8b50c34e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:54.987382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5829f-368a-4f68-87be-86fd8b50c34e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:37:55.039567 ncdv-master-1 devstack@keystone.service[104132]: WARNING keystone.common.rbac_enforcer.enforcer [None req-03e5829f-368a-4f68-87be-86fd8b50c34e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 27 09:37:55.039923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5829f-368a-4f68-87be-86fd8b50c34e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:37:55.050242 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Sep 27 09:37:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 09:37:55.056566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:37:55.075342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db750d3f-818d-4ae4-a005-43d9b47bf42f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:37:55.080463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a3545-b5b8-46e6-8f0f-98705da7bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:37:55.081780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3545-b5b8-46e6-8f0f-98705da7bb86 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:55.082060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3545-b5b8-46e6-8f0f-98705da7bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:55.082330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3545-b5b8-46e6-8f0f-98705da7bb86 admin admin] PATH_INFO: `/v3/projects/7d515506957d41b7867232158c5ad868` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:55.084030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a3545-b5b8-46e6-8f0f-98705da7bb86 admin admin] RBAC: Authorizing `identity:get_project(project_id=7d515506957d41b7867232158c5ad868)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:37:55.134239 ncdv-master-1 devstack@keystone.service[104131]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ae0a3545-b5b8-46e6-8f0f-98705da7bb86 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 27 09:37:55.134577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a3545-b5b8-46e6-8f0f-98705da7bb86 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:37:55.139333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Tue Sep 27 09:37:55 2022] GET /identity/v3/projects/7d515506957d41b7867232158c5ad868 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:37:57.527323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212a075-2998-47e2-b68a-e6c20c4607fd None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:57.527902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212a075-2998-47e2-b68a-e6c20c4607fd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:57.527902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212a075-2998-47e2-b68a-e6c20c4607fd None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:57.531999 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:37:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:37:57.538063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e799f8-76b2-4bef-9b68-a766899be2f9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:57.538519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e799f8-76b2-4bef-9b68-a766899be2f9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:57.538604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e799f8-76b2-4bef-9b68-a766899be2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:57.560011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-40e799f8-76b2-4bef-9b68-a766899be2f9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:37:57.561632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-40e799f8-76b2-4bef-9b68-a766899be2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:37:57.571062 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:37:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 27 09:37:57.894614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212a075-2998-47e2-b68a-e6c20c4607fd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:37:57.908781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212a075-2998-47e2-b68a-e6c20c4607fd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:37:57.914380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bd137-b4bb-425c-be2e-58912c262d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:37:57.914985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bd137-b4bb-425c-be2e-58912c262d04 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:57.915257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bd137-b4bb-425c-be2e-58912c262d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:57.915524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bd137-b4bb-425c-be2e-58912c262d04 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:57.924357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bd137-b4bb-425c-be2e-58912c262d04 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:37:57.926182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bd137-b4bb-425c-be2e-58912c262d04 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:37:57.935796 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Sep 27 09:37:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 09:37:57.941776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e799f8-76b2-4bef-9b68-a766899be2f9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:37:57.952074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e799f8-76b2-4bef-9b68-a766899be2f9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:37:57.958187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e490dc4-a163-4ddb-a179-69c03d0b9875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:37:57.958831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e490dc4-a163-4ddb-a179-69c03d0b9875 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:37:57.959105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e490dc4-a163-4ddb-a179-69c03d0b9875 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:37:57.959379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e490dc4-a163-4ddb-a179-69c03d0b9875 admin admin] PATH_INFO: `/v3/users/7b007883f8f945d6b63ec8a2752bb899` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:37:57.960959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e490dc4-a163-4ddb-a179-69c03d0b9875 admin admin] RBAC: Authorizing `identity:get_user(user_id=7b007883f8f945d6b63ec8a2752bb899)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:37:57.962797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e490dc4-a163-4ddb-a179-69c03d0b9875 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:37:57.964354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Tue Sep 27 09:37:57 2022] GET /identity/v3/users/7b007883f8f945d6b63ec8a2752bb899 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:00.492346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58cd63-432f-42be-80df-030818592200 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:00.493100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58cd63-432f-42be-80df-030818592200 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:00.493100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58cd63-432f-42be-80df-030818592200 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:00.494241 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:00.497895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbc61-fa7c-4575-81e6-602e98514b40 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:00.498362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbc61-fa7c-4575-81e6-602e98514b40 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:00.498694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbc61-fa7c-4575-81e6-602e98514b40 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:00.499955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:00.500242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:00.500518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:00.500930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:00.506622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73a5f0-c094-429f-bcaa-64452e07107c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:00.506878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73a5f0-c094-429f-bcaa-64452e07107c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:00.507159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73a5f0-c094-429f-bcaa-64452e07107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:00.530167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2c73a5f0-c094-429f-bcaa-64452e07107c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:00.531476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2c73a5f0-c094-429f-bcaa-64452e07107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:00.547036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG passlib.handlers.bcrypt [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=104132) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 27 09:38:00.547431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG passlib.handlers.bcrypt [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=104132) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 27 09:38:00.598614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:00.600600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:00.755499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:00.761364 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:01.074951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:01.176038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73a5f0-c094-429f-bcaa-64452e07107c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:01.195942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:01.201019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f7590-a5da-4809-9eb2-bdef9438a69d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:01.206724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:01.257069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:01.258281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:01.258567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:01.258840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:01.271726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:01.280420 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:01.281829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:01.284153 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:38:01.288276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:01.320827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73a5f0-c094-429f-bcaa-64452e07107c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:01.374026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:01.374566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:01.374850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:01.375122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:01.423933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3301173-0cd3-4170-a878-051e2d9ab4c0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:01.460201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:01.460849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:01.461126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:01.461426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:01.462098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:01.462396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:01.463870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:01.465927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:01.468761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:01.477589 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:01.483751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:01.511751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:01.514892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:01.614588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3de2a-353a-4886-a3d2-138761d5eef9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:01.615271 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 09:38:01.621818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:01.768960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:01.769501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:01.769782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:01.770150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] PATH_INFO: `/v3/users/7b007883f8f945d6b63ec8a2752bb899` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:01.771966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] RBAC: Authorizing `identity:get_user(user_id=7b007883f8f945d6b63ec8a2752bb899)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:01.773512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:01.775150 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/users/7b007883f8f945d6b63ec8a2752bb899 => generated 247 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:01.781977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:01.787010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c407e9-db50-440d-a303-27f628ba0ada None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:01.826407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:01.839504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:01.840286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:01.840626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:01.840981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:01.855629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:01.857474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:38:01.879289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:01.881111 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:38:01.882366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:38:01.882366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:38:01.882366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:01.882366 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:38:01.890329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:01.911649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:01.923588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5548dcf-2062-4503-ac51-dc7f1c64a7e8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:01.972964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:01.973837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:01.974205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:01.974587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:01.975997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:01.977707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:01.979619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:01.987125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:02.026343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fdbd4-7334-4add-aef9-5cb91982aef3 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:02.069138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0d2a-c084-43f3-905a-29b47eec52d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:02.070147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0d2a-c084-43f3-905a-29b47eec52d7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:02.070669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0d2a-c084-43f3-905a-29b47eec52d7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:02.071114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0d2a-c084-43f3-905a-29b47eec52d7 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:02.072196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0d2a-c084-43f3-905a-29b47eec52d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:02.072727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0d2a-c084-43f3-905a-29b47eec52d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:02.075063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0d2a-c084-43f3-905a-29b47eec52d7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:02.096747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:02.122013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c4b47-7f3c-4459-a602-878333106abe None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:02.179885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32104456-181e-4c8d-b215-152323e10e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:02.180875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104456-181e-4c8d-b215-152323e10e05 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:02.181299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104456-181e-4c8d-b215-152323e10e05 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:02.181712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104456-181e-4c8d-b215-152323e10e05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:02.182905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32104456-181e-4c8d-b215-152323e10e05 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.domain.id=default, user.id=7b007883f8f945d6b63ec8a2752bb899) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:02.183381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32104456-181e-4c8d-b215-152323e10e05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:02.185888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32104456-181e-4c8d-b215-152323e10e05 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:02.208720 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 11/21] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Sep 27 09:38:01 2022] GET /identity/v3/role_assignments?user.id=7b007883f8f945d6b63ec8a2752bb899&scope.domain.id=default&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 233 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:05.438203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5cc4a-d4e3-4417-b545-2bf0681eba95 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:05.438804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5cc4a-d4e3-4417-b545-2bf0681eba95 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:05.439117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5cc4a-d4e3-4417-b545-2bf0681eba95 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:05.441567 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:05.462384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddbbb8-f1c1-4a47-8685-0647815fcdaa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:05.463372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddbbb8-f1c1-4a47-8685-0647815fcdaa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:05.463954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddbbb8-f1c1-4a47-8685-0647815fcdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:05.491622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c7ddbbb8-f1c1-4a47-8685-0647815fcdaa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:05.493378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7ddbbb8-f1c1-4a47-8685-0647815fcdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:05.521854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:05.576281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014c91-d1e3-4c56-9ed8-8a81bad1ab03 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:05.577431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014c91-d1e3-4c56-9ed8-8a81bad1ab03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:05.578173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014c91-d1e3-4c56-9ed8-8a81bad1ab03 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:05.586419 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 12/23] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:05.596418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ffdd-850c-4b03-a2fd-d844c78a3a4d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:05.597401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ffdd-850c-4b03-a2fd-d844c78a3a4d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:05.598198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ffdd-850c-4b03-a2fd-d844c78a3a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:05.600896 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 12/24] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:05.647065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-41f7ffdd-850c-4b03-a2fd-d844c78a3a4d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:05.649974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-41f7ffdd-850c-4b03-a2fd-d844c78a3a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:05.736650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:05.825942 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 13/25] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:06.258008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddbbb8-f1c1-4a47-8685-0647815fcdaa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:06.399708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddbbb8-f1c1-4a47-8685-0647815fcdaa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:06.445484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:06.446563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:06.447086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:06.447587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:06.474653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:06.477244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:06.483066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09c84-79e2-4021-b46d-261bacec6808 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:06.484056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09c84-79e2-4021-b46d-261bacec6808 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:06.484534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09c84-79e2-4021-b46d-261bacec6808 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:06.486250 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 14/26] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:06.494549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81a524-bec9-44c2-8697-1afe00bf8e02 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:06.495204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81a524-bec9-44c2-8697-1afe00bf8e02 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:06.495688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81a524-bec9-44c2-8697-1afe00bf8e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:38:06.535887 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:06.536852 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:38:06.537683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:38:06.537683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:38:06.537683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:06.539495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2d81a524-bec9-44c2-8697-1afe00bf8e02 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:06.542665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2d81a524-bec9-44c2-8697-1afe00bf8e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:06.545968 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 13/27] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:06.551084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:06.573103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:06.726295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 15/28] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:06.779331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3026aea-6be2-4e47-995c-c5c49d58e68f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:06.828624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:06.829177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:06.829461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:06.829736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:06.830406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:06.830708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:06.832275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:06.855932 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 14/29] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:06.865029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81a524-bec9-44c2-8697-1afe00bf8e02 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:06.878755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:07.082152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:07.108173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:07.117460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81a524-bec9-44c2-8697-1afe00bf8e02 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:07.189567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:07.190232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:07.190580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:07.190805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:07.191497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:07.193421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:07.229934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7493f-c995-43f8-bf16-70eeb153100a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:07.337517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d70ee-0f15-4fd7-9587-f01843bec94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:07.338043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d70ee-0f15-4fd7-9587-f01843bec94a None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:07.338320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d70ee-0f15-4fd7-9587-f01843bec94a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:07.338592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d70ee-0f15-4fd7-9587-f01843bec94a None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:07.339277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d70ee-0f15-4fd7-9587-f01843bec94a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:07.340700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d70ee-0f15-4fd7-9587-f01843bec94a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:07.502309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 15/30] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 27 09:38:06 2022] POST /identity/v3/roles => generated 219 bytes in 625 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:07.505139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3f346-d5f3-4f56-8a88-590b76c8b697 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:07.505590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3f346-d5f3-4f56-8a88-590b76c8b697 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:07.506022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3f346-d5f3-4f56-8a88-590b76c8b697 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:07.518679 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 16/31] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:07 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:07.526064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ecd8-c503-4d16-97b7-45e1bb000a60 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:07.526564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ecd8-c503-4d16-97b7-45e1bb000a60 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:07.526970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ecd8-c503-4d16-97b7-45e1bb000a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:07.551549 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 16/32] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:38:06 2022] POST /identity/v3/projects => generated 301 bytes in 687 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:38:07.562189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f457ecd8-c503-4d16-97b7-45e1bb000a60 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:07.564181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f457ecd8-c503-4d16-97b7-45e1bb000a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:07.675287 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 17/33] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:07.724418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:07.878510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aeb24-be3d-48f5-bed3-948d3703f062 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:07.923399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b977d6-6ade-41d3-bc2b-38663e2eb2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:07.924064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b977d6-6ade-41d3-bc2b-38663e2eb2fb None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:07.924391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b977d6-6ade-41d3-bc2b-38663e2eb2fb None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:07.924685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b977d6-6ade-41d3-bc2b-38663e2eb2fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:07.944863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b977d6-6ade-41d3-bc2b-38663e2eb2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-f6b977d6-6ade-41d3-bc2b-38663e2eb2fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:07.951509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:07.953068 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:07.954544 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 17/34] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:38:07.962256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457ecd8-c503-4d16-97b7-45e1bb000a60 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:08.063466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:08.138737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457ecd8-c503-4d16-97b7-45e1bb000a60 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:08.183459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaca76-3dd4-4988-88df-3447446a4519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:08.184109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaca76-3dd4-4988-88df-3447446a4519 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.184389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaca76-3dd4-4988-88df-3447446a4519 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.184692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaca76-3dd4-4988-88df-3447446a4519 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.185399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaca76-3dd4-4988-88df-3447446a4519 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:08.185764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaca76-3dd4-4988-88df-3447446a4519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:08.187282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaca76-3dd4-4988-88df-3447446a4519 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:08.194363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09141-a534-4421-a2a9-a2a7a7e57544 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.194743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09141-a534-4421-a2a9-a2a7a7e57544 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.194961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09141-a534-4421-a2a9-a2a7a7e57544 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.196264 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 18/35] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:08.197949 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 18/36] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:08.204078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09141-a534-4421-a2a9-a2a7a7e57544 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:08.206125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb4b7-8bda-4b43-b1ba-ec8262aa865d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.206457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb4b7-8bda-4b43-b1ba-ec8262aa865d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.206694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb4b7-8bda-4b43-b1ba-ec8262aa865d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.230005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ffadb4b7-8bda-4b43-b1ba-ec8262aa865d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:08.231318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:08.231998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ffadb4b7-8bda-4b43-b1ba-ec8262aa865d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:08.332615 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 19/37] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:08.385224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09141-a534-4421-a2a9-a2a7a7e57544 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:08.399660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadb4b7-8bda-4b43-b1ba-ec8262aa865d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:08.428965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:08.430064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.430609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.430890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] PATH_INFO: `/v3/users/7b007883f8f945d6b63ec8a2752bb899` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.432906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] RBAC: Authorizing `identity:get_user(user_id=7b007883f8f945d6b63ec8a2752bb899)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:08.434855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:08.437855 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 19/38] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:08 2022] GET /identity/v3/users/7b007883f8f945d6b63ec8a2752bb899 => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:08.445837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:08.456217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:08.528492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadb4b7-8bda-4b43-b1ba-ec8262aa865d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:08.545156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:08.572109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76736e0c-3d24-4743-a554-89b77ed0160a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:08.574472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:08.575316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.575777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.576202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.577063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:08.579080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:08.612602 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 20/39] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 27 09:38:08 2022] POST /identity/v3/roles => generated 225 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:38:08.620303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:08.626741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.626741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.626741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.626741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:08.626741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:08.626741 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 20/40] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:08.633001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:08.726588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:08.766447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bfbb2-9eda-4a25-b5ab-0198cbf40890 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:08.775579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:08.811192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb778b93-16c4-41bf-a724-d4eeddcd37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:08.812375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb778b93-16c4-41bf-a724-d4eeddcd37a8 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.812729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb778b93-16c4-41bf-a724-d4eeddcd37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.813209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb778b93-16c4-41bf-a724-d4eeddcd37a8 None admin] PATH_INFO: `/v3/domains/default/users/7b007883f8f945d6b63ec8a2752bb899/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.816027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb778b93-16c4-41bf-a724-d4eeddcd37a8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7b007883f8f945d6b63ec8a2752bb899, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:08.818044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb778b93-16c4-41bf-a724-d4eeddcd37a8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:08.840849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 21/41] 10.222.0.26 () {58 vars in 1361 bytes} [Tue Sep 27 09:38:08 2022] PUT /identity/v3/domains/default/users/7b007883f8f945d6b63ec8a2752bb899/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:38:08.883141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972221f7-534e-47bd-8102-c7f577f5cdcd None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:08.929710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378ec84-d1d2-407d-a7b8-374888455034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:08.930637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ec84-d1d2-407d-a7b8-374888455034 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.931127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ec84-d1d2-407d-a7b8-374888455034 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.931595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ec84-d1d2-407d-a7b8-374888455034 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.932497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378ec84-d1d2-407d-a7b8-374888455034 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:08.934218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378ec84-d1d2-407d-a7b8-374888455034 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:08.957192 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 21/42] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 27 09:38:08 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:08.992609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598c8a-cc74-479a-a6f1-d13045cab83d None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:08.992877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598c8a-cc74-479a-a6f1-d13045cab83d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:08.993083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598c8a-cc74-479a-a6f1-d13045cab83d None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:08.994871 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 22/43] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:09.000971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef9c5c-b1e1-4f85-b151-98ade2acb14b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:09.001306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef9c5c-b1e1-4f85-b151-98ade2acb14b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:09.001594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef9c5c-b1e1-4f85-b151-98ade2acb14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:09.017484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:09.025458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e5ef9c5c-b1e1-4f85-b151-98ade2acb14b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:09.028535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e5ef9c5c-b1e1-4f85-b151-98ade2acb14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:09.127617 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 22/44] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:09.445850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7598c8a-cc74-479a-a6f1-d13045cab83d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:09.568787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7598c8a-cc74-479a-a6f1-d13045cab83d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:09.607914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc1566-a976-4e56-80c8-a047f55d4e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:09.608572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc1566-a976-4e56-80c8-a047f55d4e4d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:09.608947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc1566-a976-4e56-80c8-a047f55d4e4d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:09.609224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc1566-a976-4e56-80c8-a047f55d4e4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:09.610491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc1566-a976-4e56-80c8-a047f55d4e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:09.611994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc1566-a976-4e56-80c8-a047f55d4e4d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:09.613730 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 23/45] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:09.621373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef9c5c-b1e1-4f85-b151-98ade2acb14b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:09.728377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef9c5c-b1e1-4f85-b151-98ade2acb14b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:09.763381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418573cb-bd82-43c3-b418-0014f18634bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:09.763979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418573cb-bd82-43c3-b418-0014f18634bd None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:09.764258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418573cb-bd82-43c3-b418-0014f18634bd None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:09.764564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418573cb-bd82-43c3-b418-0014f18634bd None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:09.765265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418573cb-bd82-43c3-b418-0014f18634bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:09.766767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418573cb-bd82-43c3-b418-0014f18634bd None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:09.773069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:09.797970 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 23/46] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:38:09 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:11.330770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f92eed-78af-44b0-9875-b496e02035ee None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:11.331349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f92eed-78af-44b0-9875-b496e02035ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:11.331349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f92eed-78af-44b0-9875-b496e02035ee None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:11.332745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 24/47] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:11.338245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290dc0-fab7-41f7-815b-55d47b5a3cac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:11.338673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290dc0-fab7-41f7-815b-55d47b5a3cac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:11.338813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290dc0-fab7-41f7-815b-55d47b5a3cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:11.358050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7f290dc0-fab7-41f7-815b-55d47b5a3cac None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:11.359263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7f290dc0-fab7-41f7-815b-55d47b5a3cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:11.429186 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 24/48] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:11.750996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f92eed-78af-44b0-9875-b496e02035ee None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:11.789677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:11.861941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f92eed-78af-44b0-9875-b496e02035ee None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:11.899072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:11.899710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:11.899989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:11.900260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:11.910926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:11.916259 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:11.917157 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:11.917892 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 25/49] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:11.923564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290dc0-fab7-41f7-815b-55d47b5a3cac None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:12.026737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290dc0-fab7-41f7-815b-55d47b5a3cac None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:12.044032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:12.062035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:12.062665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:12.062952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:12.063201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:12.063881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:12.064255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:12.065590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:12.078951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 25/50] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:12.084884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:12.185515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:12.193193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5f865-af36-45a7-b774-3c0ff649e2a6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:12.227407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb7d9-4399-4ad1-97cc-2ff6381c1631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:12.228059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb7d9-4399-4ad1-97cc-2ff6381c1631 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:12.228325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb7d9-4399-4ad1-97cc-2ff6381c1631 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:12.228599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb7d9-4399-4ad1-97cc-2ff6381c1631 None admin] PATH_INFO: `/v3/users/7b007883f8f945d6b63ec8a2752bb899` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:12.229819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb7d9-4399-4ad1-97cc-2ff6381c1631 None admin] RBAC: Authorizing `identity:get_user(user_id=7b007883f8f945d6b63ec8a2752bb899)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:12.231299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb7d9-4399-4ad1-97cc-2ff6381c1631 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:12.232966 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 26/51] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:12 2022] GET /identity/v3/users/7b007883f8f945d6b63ec8a2752bb899 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:12.238655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:12.293501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac99c54-34e9-47c5-9a1c-fb94c8cd98c1 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:12.293800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac99c54-34e9-47c5-9a1c-fb94c8cd98c1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:12.294069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac99c54-34e9-47c5-9a1c-fb94c8cd98c1 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:12.295408 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 27/52] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:12.300504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472d7e-9eaf-429a-87b6-24d1830cb370 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:12.300821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472d7e-9eaf-429a-87b6-24d1830cb370 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:12.301061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c472d7e-9eaf-429a-87b6-24d1830cb370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:12.319808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2c472d7e-9eaf-429a-87b6-24d1830cb370 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:12.321060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2c472d7e-9eaf-429a-87b6-24d1830cb370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:12.342270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa945ad-325a-4f0f-bbc0-7d7901230f5b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:12.377283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:12.377931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:12.378210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:12.378485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] PATH_INFO: `/v3/domains/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:12.379654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:12.381164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:12.382788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:12.388290 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 28/54] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:12.388838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:12.450313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:12.507744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f36b5-3bfd-4dce-a3d5-e057c58ee66c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:12.542552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:12.543168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:12.543447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:12.543721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:12.544444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.domain.id=default, user.id=7b007883f8f945d6b63ec8a2752bb899) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:12.544789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:12.546235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:12.554126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Sep 27 09:38:12 2022] GET /identity/v3/role_assignments?user.id=7b007883f8f945d6b63ec8a2752bb899&scope.domain.id=default&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 530 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:12.719613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c472d7e-9eaf-429a-87b6-24d1830cb370 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:12.821798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:12.831019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c472d7e-9eaf-429a-87b6-24d1830cb370 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:12.867117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58a811-2bac-4870-8284-5a65e2e35274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:12.867766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58a811-2bac-4870-8284-5a65e2e35274 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:12.868048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58a811-2bac-4870-8284-5a65e2e35274 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:12.868323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58a811-2bac-4870-8284-5a65e2e35274 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:12.869533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58a811-2bac-4870-8284-5a65e2e35274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:12.871035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58a811-2bac-4870-8284-5a65e2e35274 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:12.872689 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 29/56] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:12.878483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:12.979446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707556ca-c2dd-4d15-982d-5f07a1797b86 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:12.995008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:13.018411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02abc77-7089-4154-ac56-ba3afa752934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:13.019033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02abc77-7089-4154-ac56-ba3afa752934 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:13.019380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02abc77-7089-4154-ac56-ba3afa752934 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:13.019579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02abc77-7089-4154-ac56-ba3afa752934 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:13.020905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02abc77-7089-4154-ac56-ba3afa752934 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:13.022368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02abc77-7089-4154-ac56-ba3afa752934 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:13.052327 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 28/57] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Sep 27 09:38:12 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:15.491464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb2907-6af5-43f6-ab23-d6dc93c8615b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:15.492024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb2907-6af5-43f6-ab23-d6dc93c8615b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:15.492024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb2907-6af5-43f6-ab23-d6dc93c8615b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:15.493340 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 30/58] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:15.498919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbb63b-e675-4101-a79a-d7164576e64c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:15.499350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbb63b-e675-4101-a79a-d7164576e64c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:15.499548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbb63b-e675-4101-a79a-d7164576e64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:15.519672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2fdbb63b-e675-4101-a79a-d7164576e64c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:15.521690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2fdbb63b-e675-4101-a79a-d7164576e64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:15.621106 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 29/59] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:15.940093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb2907-6af5-43f6-ab23-d6dc93c8615b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:16.053633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb2907-6af5-43f6-ab23-d6dc93c8615b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:16.092570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafd5fd-3317-4527-8334-6e3d18492ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:16.093286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafd5fd-3317-4527-8334-6e3d18492ea5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:16.093545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafd5fd-3317-4527-8334-6e3d18492ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:16.093838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafd5fd-3317-4527-8334-6e3d18492ea5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:16.094984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafd5fd-3317-4527-8334-6e3d18492ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:16.096459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafd5fd-3317-4527-8334-6e3d18492ea5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:16.098060 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 31/60] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:16.104292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbb63b-e675-4101-a79a-d7164576e64c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:16.229097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:16.236699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbb63b-e675-4101-a79a-d7164576e64c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:16.273644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be3097-8f3d-44c2-ba85-3bda63f84439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:16.274264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be3097-8f3d-44c2-ba85-3bda63f84439 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:16.274932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be3097-8f3d-44c2-ba85-3bda63f84439 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:16.275586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be3097-8f3d-44c2-ba85-3bda63f84439 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:16.276249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be3097-8f3d-44c2-ba85-3bda63f84439 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:16.277736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be3097-8f3d-44c2-ba85-3bda63f84439 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:16.310234 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 30/61] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:38:16 2022] POST /identity/v3/users => generated 275 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:20.989616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed5e9-8425-4e8d-8350-489d37f926c4 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:20.989616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed5e9-8425-4e8d-8350-489d37f926c4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:20.989616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed5e9-8425-4e8d-8350-489d37f926c4 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:20.989616 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 32/62] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:20.996304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88170600-d20b-48ec-bd65-769bcb889f3d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:20.997144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88170600-d20b-48ec-bd65-769bcb889f3d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:20.997144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88170600-d20b-48ec-bd65-769bcb889f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.023455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-88170600-d20b-48ec-bd65-769bcb889f3d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:21.024715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-88170600-d20b-48ec-bd65-769bcb889f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:21.124293 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 31/63] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:21.300244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef297214-d826-4dba-8007-fac84f8a36c8 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.300244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef297214-d826-4dba-8007-fac84f8a36c8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.300244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef297214-d826-4dba-8007-fac84f8a36c8 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.300244 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 33/64] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:21.308835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05670b1-bfe3-4415-b6b4-ee4ba48464c6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.309100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05670b1-bfe3-4415-b6b4-ee4ba48464c6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.309396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05670b1-bfe3-4415-b6b4-ee4ba48464c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.331483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a05670b1-bfe3-4415-b6b4-ee4ba48464c6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:21.332851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a05670b1-bfe3-4415-b6b4-ee4ba48464c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:21.439484 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 32/65] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:21.500525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3cd38-e241-4d64-a24f-39f0d12abdd7 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.501586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3cd38-e241-4d64-a24f-39f0d12abdd7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.502345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3cd38-e241-4d64-a24f-39f0d12abdd7 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.505284 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 34/66] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:21.511844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2b80b-cc6c-48c4-8447-faf8db7916a6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.512255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2b80b-cc6c-48c4-8447-faf8db7916a6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.512427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2b80b-cc6c-48c4-8447-faf8db7916a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.539051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5cd2b80b-cc6c-48c4-8447-faf8db7916a6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:21.540341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5cd2b80b-cc6c-48c4-8447-faf8db7916a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:21.548514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:21.655426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 33/67] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:21.724135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3cd38-e241-4d64-a24f-39f0d12abdd7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:21.782342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b14bc-2e9f-4374-b513-6d51effbb2f4 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.782980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b14bc-2e9f-4374-b513-6d51effbb2f4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.783453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b14bc-2e9f-4374-b513-6d51effbb2f4 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.785799 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 34/68] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:21.788484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:21.796612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e146-d0a3-4d32-94cd-090f7d8ef39e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.796909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e146-d0a3-4d32-94cd-090f7d8ef39e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.797244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e146-d0a3-4d32-94cd-090f7d8ef39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.830551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7dd5e146-d0a3-4d32-94cd-090f7d8ef39e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:21.832619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7dd5e146-d0a3-4d32-94cd-090f7d8ef39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:21.917230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3cd38-e241-4d64-a24f-39f0d12abdd7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:21.965232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec666df3-e3ef-4d0c-8aaf-278ef246460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:21.966047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec666df3-e3ef-4d0c-8aaf-278ef246460a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.966326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec666df3-e3ef-4d0c-8aaf-278ef246460a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.966602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec666df3-e3ef-4d0c-8aaf-278ef246460a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.980108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec666df3-e3ef-4d0c-8aaf-278ef246460a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:21.981542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 35/69] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:21.983890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509e939-6204-4cf0-8828-4f5faf0b45ea None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:21.984370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509e939-6204-4cf0-8828-4f5faf0b45ea None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:21.984942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509e939-6204-4cf0-8828-4f5faf0b45ea None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ec666df3-e3ef-4d0c-8aaf-278ef246460a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:21.985183 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:21.986717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:21.989094 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:21.992275 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 36/71] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:21.992494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:38:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:21.996872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509e939-6204-4cf0-8828-4f5faf0b45ea None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.006935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0da4-23f2-4d71-a287-ad2dcc2f86e4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.007372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0da4-23f2-4d71-a287-ad2dcc2f86e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.007626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0da4-23f2-4d71-a287-ad2dcc2f86e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.037157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-acea0da4-23f2-4d71-a287-ad2dcc2f86e4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:22.038966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-acea0da4-23f2-4d71-a287-ad2dcc2f86e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:22.102658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:22.137242 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 36/72] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:22.138320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea0da4-23f2-4d71-a287-ad2dcc2f86e4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.166725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:22.172361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509e939-6204-4cf0-8828-4f5faf0b45ea None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.267640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.270080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.270791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.271815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.274348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:22.275094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:22.275614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea0da4-23f2-4d71-a287-ad2dcc2f86e4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.279913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:22.292077 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 37/73] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:38:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:22.297850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.333237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.334158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.334537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.334906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.347658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:22.353747 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:22.355746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:22.357487 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 37/74] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:22.360563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.449591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39479df-7c82-4897-81dd-123ca7c50de1 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.480970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:22.500542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:22.505464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9963e-3dea-4d5e-ae3d-baaa21f330e1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.508284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.509420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.510031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.510494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.526084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:22.531577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:22.532565 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:22.533425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 38/75] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:22.534424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.545896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.546539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.546843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.547108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.564510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:22.566448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:22.569331 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 38/76] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:22.570492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.660203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978b182-6554-490c-a520-a310b5677be8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.713345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7877f4-a5a5-4fd9-92b0-d7922a02f34b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.715999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.717023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.718297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.718723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.720611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:22.723285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:22.726223 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 39/77] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:22.727495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.772811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.774075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.774534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.775001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.776106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:22.776529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:22.778889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:22.782462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:22.792150 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 39/78] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:22.793362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.823708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:22.875362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73634bee-10ea-435f-b57f-4d848813cc09 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.922614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.923616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.924053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.924418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bfe08-95bf-4120-8b88-c8185fcc28c7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:22.924575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.940754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:22.949664 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:22.950723 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:22.951649 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:22.952280 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 40/79] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:22.953442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:22.979891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:22.980762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:22.981214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:22.981660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:22.982749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:22.983188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:22.985548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:23.000113 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 40/80] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:23.001249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:23.078519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:23.081665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54cf3b-3b50-4113-88b4-265b506bf869 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:23.104637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:23.123221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:23.124061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:23.124426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:23.124811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:23.129006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850786c-738e-4cd3-b6da-1da598963351 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:23.168746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:23.169481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:23.169665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:23.169900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:23.170568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:23.172027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:23.205053 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 41/81] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:38:22 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:23.206129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:23.224857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:23.227100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:23.362514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1247e8-fca5-4650-86bf-44a59064e975 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:23.386725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:23.411220 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 41/82] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:23.411220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:23.430199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:23.431178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:23.431620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:23.432056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:23.434267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:23.437740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:23.445910 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 42/83] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:23 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:23.445910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:23.451997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:23.597989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030cfce-7a68-4c62-b955-7f93a65500f1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:23.598775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc5306-5288-497d-9a67-1f66c1694cf0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:23.669104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:23.669851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:23.670276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:23.670672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] PATH_INFO: `/v3/users/7b007883f8f945d6b63ec8a2752bb899` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:23.672159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] RBAC: Authorizing `identity:get_user(user_id=7b007883f8f945d6b63ec8a2752bb899)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:23.672804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:23.673607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:23.674011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:23.674406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:23.674713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:23.675526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:23.676053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:23.677528 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 43/84] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:23 2022] GET /identity/v3/users/7b007883f8f945d6b63ec8a2752bb899 => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:23.684759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:23.684965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:23.693179 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 42/85] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:23.695036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:23.795326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:23.874669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:23.899225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fbfdb-6d2c-4281-83e1-c48837f73763 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:23.916396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcaa780-1caa-48b9-8fe9-ae932b631a0f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:23.965559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:23.966975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:23.967721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:23.968475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:23.970843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:23.973991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:23.977653 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 44/86] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:23 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:23.979283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:23.999671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:24.000885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:24.002181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:24.002848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:24.143017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:24.147163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:24.202127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23658b-de59-4155-bbf7-0d10f444e310 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:24.249310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:24.283205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:24.285112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:24.286079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:24.286854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:24.333215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:24.373077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:24.374600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:24.380523 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 43/87] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:24.380523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:24.499852 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 45/88] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:24.500628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:24.543693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:24.625495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c534614-e8cb-4ac2-a6b0-147c805dc413 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:24.708689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:24.709982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:24.710286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:24.710763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:24.713079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:24.716012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:24.719062 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 44/89] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:24 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:24.725462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:24.727634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad50d7-5da5-482a-921d-727929d62abc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:24.807320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:24.809891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:24.810684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:24.811419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:24.813713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] RBAC: Adding query filter params (role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=cc2b0cd654dd4bf8a616c60ea6b268d3) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:24.815090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:24.818868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:24.828872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:24.839882 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 46/90] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:24 2022] GET /identity/v3/role_assignments?user.id=cc2b0cd654dd4bf8a616c60ea6b268d3&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=6f0bec1f01514029a1162fe631d80648 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:24.839882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:25.020119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbf339-f6bf-4135-a61b-b062084d95b2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:25.096111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:25.097109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:25.097401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:25.097677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:25.103114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:25.104341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:25.108759 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 45/91] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:24 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:25.109723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:25.136197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:25.168751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057175-9641-40cb-9abc-e3f69b3f3fbc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:25.215680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:25.216886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:25.216886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:25.216886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:25.218965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:25.220583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:25.222570 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 47/92] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:24 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:25.223889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:25.350507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d19de5-e0c0-4605-97c8-aa2388b5cc88 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:25.397996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:25.397996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:25.397996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:25.397996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:25.398702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=cc2b0cd654dd4bf8a616c60ea6b268d3) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:25.398702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:25.398702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:25.403390 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 48/93] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:25 2022] GET /identity/v3/role_assignments?user.id=cc2b0cd654dd4bf8a616c60ea6b268d3&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:25.404417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:25.429502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:25.459891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b7880-d563-452e-9a3c-6c000a23706b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:25.525562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:25.579155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736f8b-a795-479e-a759-0d993c8091bd None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:25.593986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:25.595020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:25.595494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:25.595969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:25.662506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:25.663147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:25.663428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:25.663706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:25.664458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=7b007883f8f945d6b63ec8a2752bb899) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:25.664798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:25.666351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:25.678544 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 49/94] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:25 2022] GET /identity/v3/role_assignments?user.id=7b007883f8f945d6b63ec8a2752bb899&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:25.856623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:25.859473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:26.087036 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 46/95] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 977 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:26.093442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:26.197312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:26.217703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d387f-29db-4ae4-8e32-81ab924a7d19 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:26.257889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d9ef3-0176-4eda-97d8-30749a7da760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:26.258553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9ef3-0176-4eda-97d8-30749a7da760 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:26.258823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9ef3-0176-4eda-97d8-30749a7da760 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:26.259102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9ef3-0176-4eda-97d8-30749a7da760 None admin] PATH_INFO: `/v3/projects/29febf817c7542668ddf224f88054212` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:26.260293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d9ef3-0176-4eda-97d8-30749a7da760 None admin] RBAC: Authorizing `identity:get_project(project_id=29febf817c7542668ddf224f88054212)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:26.262051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d9ef3-0176-4eda-97d8-30749a7da760 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:26.264208 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 50/96] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:26 2022] GET /identity/v3/projects/29febf817c7542668ddf224f88054212 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:26.270773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:26.319511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:26.445469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655ec93-e0c9-4cb5-8574-dbde869d8150 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:26.492703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b33f39-89e2-4914-bb3e-07befaaa120d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:26.493774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33f39-89e2-4914-bb3e-07befaaa120d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:26.493774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33f39-89e2-4914-bb3e-07befaaa120d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:26.494021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33f39-89e2-4914-bb3e-07befaaa120d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:26.494613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b33f39-89e2-4914-bb3e-07befaaa120d None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=29febf817c7542668ddf224f88054212, user.id=cc2b0cd654dd4bf8a616c60ea6b268d3) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:26.494922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b33f39-89e2-4914-bb3e-07befaaa120d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:26.496411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b33f39-89e2-4914-bb3e-07befaaa120d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:26.503287 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 47/97] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:26 2022] GET /identity/v3/role_assignments?user.id=cc2b0cd654dd4bf8a616c60ea6b268d3&scope.project.id=29febf817c7542668ddf224f88054212&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:27.664103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8d14-d136-475b-828c-420c19349c44 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:27.664968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8d14-d136-475b-828c-420c19349c44 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:27.665327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8d14-d136-475b-828c-420c19349c44 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:27.668144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 51/98] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:27.677301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9089a-fd5d-4f8b-bdea-efbdfce3eda7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:27.678273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9089a-fd5d-4f8b-bdea-efbdfce3eda7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:27.678273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9089a-fd5d-4f8b-bdea-efbdfce3eda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:27.711947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dbd9089a-fd5d-4f8b-bdea-efbdfce3eda7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:27.713929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dbd9089a-fd5d-4f8b-bdea-efbdfce3eda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:27.861357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26222e-8467-4629-a2d9-476254774abe None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:27.861775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26222e-8467-4629-a2d9-476254774abe None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:27.862166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26222e-8467-4629-a2d9-476254774abe None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:27.865236 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 52/99] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:27.868536 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 48/100] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:27.875849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a849-8e38-4a9b-8a57-6d748645b8bb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:27.876295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a849-8e38-4a9b-8a57-6d748645b8bb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:27.876799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a849-8e38-4a9b-8a57-6d748645b8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:27.920512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0991a849-8e38-4a9b-8a57-6d748645b8bb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:27.922520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0991a849-8e38-4a9b-8a57-6d748645b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:28.076072 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 53/101] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:28.542716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9089a-fd5d-4f8b-bdea-efbdfce3eda7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:28.574153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:28.633978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991a849-8e38-4a9b-8a57-6d748645b8bb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:28.763441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9089a-fd5d-4f8b-bdea-efbdfce3eda7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:28.817464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:28.837187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991a849-8e38-4a9b-8a57-6d748645b8bb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:28.840555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:28.843011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:28.843469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:28.843914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:28.875669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:28.893159 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:28.894134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:28.895030 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:28.896456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 49/102] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:28.901784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:28.915867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:28.917340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:28.917832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:28.918308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:28.925455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:28.933815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:28.937019 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 54/103] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:28.951176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:29.174223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:29.216023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7738a-b580-4f73-a03b-3ce663926476 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:29.268331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e366d3-db86-4fa9-aca4-0864f351c136 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:29.301027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd797c-6246-4a86-a4ee-6b12c6847663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:29.302331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd797c-6246-4a86-a4ee-6b12c6847663 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:29.303070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd797c-6246-4a86-a4ee-6b12c6847663 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:29.303728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd797c-6246-4a86-a4ee-6b12c6847663 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:29.305221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd797c-6246-4a86-a4ee-6b12c6847663 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:29.308407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd797c-6246-4a86-a4ee-6b12c6847663 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:29.382867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:29.384335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:29.385057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:29.385686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:29.387627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:29.388427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:29.391984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:29.398883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:29.410304 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 55/104] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:38:28 2022] POST /identity/v3/projects => generated 302 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:38:29.416529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b593d34-334c-4a53-bb30-20ae42b051d9 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:29.420370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b593d34-334c-4a53-bb30-20ae42b051d9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:29.420979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b593d34-334c-4a53-bb30-20ae42b051d9 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:29.427787 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 56/105] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:29.433302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf17b8-f2e3-4428-bb5f-72ee2bddf385 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:29.433775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf17b8-f2e3-4428-bb5f-72ee2bddf385 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:29.434217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf17b8-f2e3-4428-bb5f-72ee2bddf385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:29.437087 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 50/106] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 536 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:29.443947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:29.470367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-accf17b8-f2e3-4428-bb5f-72ee2bddf385 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:29.473396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-accf17b8-f2e3-4428-bb5f-72ee2bddf385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:29.629967 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 57/107] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:29.723907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8d7a1-21d3-4c34-9397-48a2187f40d9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:29.778656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58567bf-fcd2-481c-8dc2-4e7050c35065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:29.780143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58567bf-fcd2-481c-8dc2-4e7050c35065 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:29.780674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58567bf-fcd2-481c-8dc2-4e7050c35065 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:29.781150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58567bf-fcd2-481c-8dc2-4e7050c35065 None admin] PATH_INFO: `/v3/users/6f14bb62d7544723b5d26e0834c8884c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:29.787358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:29.804336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58567bf-fcd2-481c-8dc2-4e7050c35065 None admin] RBAC: Authorizing `identity:get_user(user_id=6f14bb62d7544723b5d26e0834c8884c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:29.806295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58567bf-fcd2-481c-8dc2-4e7050c35065 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:29.808342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 51/108] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:29 2022] GET /identity/v3/users/6f14bb62d7544723b5d26e0834c8884c => generated 289 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:29.814947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf17b8-f2e3-4428-bb5f-72ee2bddf385 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:29.834619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d527d-4bdc-49e5-9a71-7c0f5d0f8d7b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:29.835284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d527d-4bdc-49e5-9a71-7c0f5d0f8d7b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:29.835821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d527d-4bdc-49e5-9a71-7c0f5d0f8d7b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:29.838501 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 52/109] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:29.844249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7102d-ab86-4128-a88a-ec52e5cdf62a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:29.845242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7102d-ab86-4128-a88a-ec52e5cdf62a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:29.845723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7102d-ab86-4128-a88a-ec52e5cdf62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:29.884937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-03f7102d-ab86-4128-a88a-ec52e5cdf62a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:29.886662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-03f7102d-ab86-4128-a88a-ec52e5cdf62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:30.116550 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 53/110] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:30.140480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf17b8-f2e3-4428-bb5f-72ee2bddf385 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:30.193145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:30.212070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:30.213138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:30.213629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:30.214100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:30.354166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:30.356885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:30.377121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7102d-ab86-4128-a88a-ec52e5cdf62a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:30.594867 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 58/111] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 781 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:30.595761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:30.681681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7102d-ab86-4128-a88a-ec52e5cdf62a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:30.750850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:30.757128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:30.779509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:30.780292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:30.780584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:30.780767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:30.808533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:30.813676 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:30.815147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 54/112] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:38:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:38:30.816474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:30.853539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b059d7-4360-40ed-8e01-1d447136342c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:30.954860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:30.955771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:30.956230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:30.956640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:30.987345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041d389-af08-4688-9a87-c2d8c45c8bc0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:31.016794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:31.026474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:31.027410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:31.028189 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:31.028808 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 59/113] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:31.042969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:31.088537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:31.089630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.090119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.090600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.094321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:31.097080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:31.100164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 55/114] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:30 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:31.106150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:31.196483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:31.251261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:31.262899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a956c2e-3638-4109-91e2-1f14ff9a942c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:31.280408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa082f80-f96d-43db-9b92-da9292ab70b4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:31.308003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:31.308773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.309124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.309460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.310217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:31.310586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:31.312256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:31.323165 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 56/115] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:31.329237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:31.355613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8035154-0425-465c-a09d-941c3fbdef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:31.356690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035154-0425-465c-a09d-941c3fbdef0c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.357318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035154-0425-465c-a09d-941c3fbdef0c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.357799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035154-0425-465c-a09d-941c3fbdef0c None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.359079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8035154-0425-465c-a09d-941c3fbdef0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:31.359606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8035154-0425-465c-a09d-941c3fbdef0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:31.365229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8035154-0425-465c-a09d-941c3fbdef0c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:31.383292 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 60/116] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:38:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:31.386003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de811d0-d375-4c9b-b0f9-2bda6d0ad80b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.386497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de811d0-d375-4c9b-b0f9-2bda6d0ad80b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.387394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de811d0-d375-4c9b-b0f9-2bda6d0ad80b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.389970 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 61/117] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:31.392120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de811d0-d375-4c9b-b0f9-2bda6d0ad80b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:31.491268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b8b55-1377-486a-a9b5-cb2b5b49dc98 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:31.514986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:31.525565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de811d0-d375-4c9b-b0f9-2bda6d0ad80b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:31.547841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb054dd1-001f-4fc1-81ef-d8bf0ac9cfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:31.548807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb054dd1-001f-4fc1-81ef-d8bf0ac9cfd9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.549254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb054dd1-001f-4fc1-81ef-d8bf0ac9cfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.549728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb054dd1-001f-4fc1-81ef-d8bf0ac9cfd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.550958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb054dd1-001f-4fc1-81ef-d8bf0ac9cfd9 None admin] RBAC: Adding query filter params (role.id=338edc307ed345d6b18db9185e15aa19, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=6f14bb62d7544723b5d26e0834c8884c) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:31.551477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb054dd1-001f-4fc1-81ef-d8bf0ac9cfd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:31.553905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb054dd1-001f-4fc1-81ef-d8bf0ac9cfd9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:31.563843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 57/118] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:31 2022] GET /identity/v3/role_assignments?user.id=6f14bb62d7544723b5d26e0834c8884c&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=338edc307ed345d6b18db9185e15aa19 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:31.576375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a812ec-35b2-48fa-93cf-de7cee8ccb85 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.576375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a812ec-35b2-48fa-93cf-de7cee8ccb85 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.576375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a812ec-35b2-48fa-93cf-de7cee8ccb85 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.576375 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 58/119] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:31.580811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a812ec-35b2-48fa-93cf-de7cee8ccb85 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:31.603597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad620e-340d-4477-a327-5c4a6bc7de1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:31.613368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad620e-340d-4477-a327-5c4a6bc7de1e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.613368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad620e-340d-4477-a327-5c4a6bc7de1e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.613368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad620e-340d-4477-a327-5c4a6bc7de1e None admin] PATH_INFO: `/v3/users/7b007883f8f945d6b63ec8a2752bb899` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.613368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad620e-340d-4477-a327-5c4a6bc7de1e None admin] RBAC: Authorizing `identity:get_user(user_id=7b007883f8f945d6b63ec8a2752bb899)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:31.617217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad620e-340d-4477-a327-5c4a6bc7de1e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:31.628689 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 62/120] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:31 2022] GET /identity/v3/users/7b007883f8f945d6b63ec8a2752bb899 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:31.628689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16f0e-f24a-4c07-a7ed-30a695f2496c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.628689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16f0e-f24a-4c07-a7ed-30a695f2496c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.628689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16f0e-f24a-4c07-a7ed-30a695f2496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.673449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7ae16f0e-f24a-4c07-a7ed-30a695f2496c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:31.676286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7ae16f0e-f24a-4c07-a7ed-30a695f2496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:31.711980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:31.836959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a812ec-35b2-48fa-93cf-de7cee8ccb85 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:31.861142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 63/121] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:31.864472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168bfa4-daa2-4e95-99e4-460a1975f1a6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.865269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168bfa4-daa2-4e95-99e4-460a1975f1a6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.866127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168bfa4-daa2-4e95-99e4-460a1975f1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.901328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b168bfa4-daa2-4e95-99e4-460a1975f1a6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:31.903596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b168bfa4-daa2-4e95-99e4-460a1975f1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:31.904374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:31.905610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:31.906184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:31.906757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:31.909442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:31.912504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:31.915881 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 59/122] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:31 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:31.923089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:32.015342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:32.076519 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 64/123] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:32.087135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168bfa4-daa2-4e95-99e4-460a1975f1a6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:32.235331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28aeb0-8038-46b6-9477-0a2a58057686 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:32.335756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:32.362225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:32.362815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:32.363133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:32.363408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:32.382364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:32.383881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:32.385954 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 60/124] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:31 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:32.392985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168bfa4-daa2-4e95-99e4-460a1975f1a6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:32.396166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:32.479576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:32.481552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:32.483889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:32.484381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:32.490551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:32.495950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:32.504105 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 65/125] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:32 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:32.533066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:32.699399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2781ac-8ff8-41fe-ac32-ec1f73d90760 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:32.719742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2d8d-e073-4b6c-b355-b4be5a27e0fc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:32.726660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:32.767436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:32.768396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:32.768948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:32.769395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/cc2b0cd654dd4bf8a616c60ea6b268d3/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:32.771060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:32.772499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:32.773314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:32.773472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] RBAC: Authorizing `identity:create_grant(project_id=6e71bfc5cdae43b9baf98881e1a87f1c, user_id=cc2b0cd654dd4bf8a616c60ea6b268d3, role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:32.774185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/7b007883f8f945d6b63ec8a2752bb899/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:32.775014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:32.778661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] RBAC: Authorizing `identity:create_grant(project_id=6e71bfc5cdae43b9baf98881e1a87f1c, user_id=7b007883f8f945d6b63ec8a2752bb899, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:32.782058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:32.797170 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 66/126] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:32 2022] PUT /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/cc2b0cd654dd4bf8a616c60ea6b268d3/roles/6f0bec1f01514029a1162fe631d80648 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:38:32.798419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:32.810494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:32.826107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 61/127] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:32 2022] PUT /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/7b007883f8f945d6b63ec8a2752bb899/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:38:32.828854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:32.949526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc790d-0871-419f-8582-4f1e593dd2cb None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:33.001477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83a3be-bec4-456c-9518-449ae1de26ba None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:33.037259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:33.038809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:33.039260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:33.039740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:33.052837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:33.053918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:33.054404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:33.054874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:33.069433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:33.087703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:33.088981 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:33.089824 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 67/128] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:33.098019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:33.101162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:33.104694 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:33.105617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:33.106401 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:33.108953 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 62/129] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:33.119425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:33.186499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:33.288988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:33.363938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00435-a125-498a-92ce-f4b9b457f718 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:33.372601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbefe82-da64-43a1-8598-984637969af0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:33.423862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:33.424918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:33.425401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:33.425866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:33.426837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:33.427369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:33.431399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:33.443717 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 63/130] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:33.449962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:33.456489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:33.457818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:33.458262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:33.458808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:33.460045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:33.460579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:33.465098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:33.494178 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 68/131] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:33.504179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:33.573890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:33.584387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f41fa1-15ae-4089-bb8f-8a65d97d879c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:33.670102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:33.671338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:33.671876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:33.672447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:33.678156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:33.683423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:33.686962 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 64/132] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:33 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:33.697215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:33.698129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c05415-2ae0-45b1-98bc-5fd709b72fd4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:33.735814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:33.754468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:33.756779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:33.757391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:33.758390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:33.761040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:33.767235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:33.770138 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 69/133] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:33 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:33.778489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:33.935607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f5ae7-5035-4c22-8674-0776d26ddc15 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:34.008495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:34.011243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:34.011686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:34.012107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] PATH_INFO: `/v3/projects/29febf817c7542668ddf224f88054212` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:34.013808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] RBAC: Authorizing `identity:get_project(project_id=29febf817c7542668ddf224f88054212)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:34.015465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:34.016303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec819389-000e-4d30-8fc7-eed16af12b9d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:34.017518 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 65/134] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:33 2022] GET /identity/v3/projects/29febf817c7542668ddf224f88054212 => generated 312 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:34.039186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:34.116002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:34.116679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:34.116963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:34.117234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:34.119146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:34.120628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:34.122374 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 70/135] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:33 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:34.128682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:34.131210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:34.317234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:34.343092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd9864-c949-46ff-9296-1022f46193d6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:34.404832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d865286-3da5-4b15-abe7-eb224f9d306f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:34.457746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb47d95-6390-40fb-a4ad-93fbb19052c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:34.458568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb47d95-6390-40fb-a4ad-93fbb19052c0 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:34.458943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb47d95-6390-40fb-a4ad-93fbb19052c0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:34.459218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb47d95-6390-40fb-a4ad-93fbb19052c0 None admin] PATH_INFO: `/v3/projects/29febf817c7542668ddf224f88054212/users/cc2b0cd654dd4bf8a616c60ea6b268d3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:34.459867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a947e2-3ecf-411d-bc1a-5fab85246df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:34.460570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a947e2-3ecf-411d-bc1a-5fab85246df0 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:34.460927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a947e2-3ecf-411d-bc1a-5fab85246df0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:34.461216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a947e2-3ecf-411d-bc1a-5fab85246df0 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/cc2b0cd654dd4bf8a616c60ea6b268d3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:34.461311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb47d95-6390-40fb-a4ad-93fbb19052c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=29febf817c7542668ddf224f88054212, user_id=cc2b0cd654dd4bf8a616c60ea6b268d3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:34.462810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb47d95-6390-40fb-a4ad-93fbb19052c0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:34.466260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a947e2-3ecf-411d-bc1a-5fab85246df0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e71bfc5cdae43b9baf98881e1a87f1c, user_id=cc2b0cd654dd4bf8a616c60ea6b268d3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:34.467759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a947e2-3ecf-411d-bc1a-5fab85246df0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:34.496993 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 71/136] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:34 2022] PUT /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/cc2b0cd654dd4bf8a616c60ea6b268d3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:38:34.501797 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 66/137] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:34 2022] PUT /identity/v3/projects/29febf817c7542668ddf224f88054212/users/cc2b0cd654dd4bf8a616c60ea6b268d3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:38:34.808137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156878-30f6-48fb-bb76-6f8f56460815 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:34.808420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156878-30f6-48fb-bb76-6f8f56460815 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:34.808972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156878-30f6-48fb-bb76-6f8f56460815 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:34.810611 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 72/138] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:34.816856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46529c69-b3a8-4060-bff1-20023736cc15 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:34.817142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46529c69-b3a8-4060-bff1-20023736cc15 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:34.817576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46529c69-b3a8-4060-bff1-20023736cc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:34.844880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-46529c69-b3a8-4060-bff1-20023736cc15 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:34.847504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46529c69-b3a8-4060-bff1-20023736cc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:35.020812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 67/139] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:35.473204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156878-30f6-48fb-bb76-6f8f56460815 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:35.633679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156878-30f6-48fb-bb76-6f8f56460815 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:35.667896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:35.690426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3e204-b752-4b2f-8d39-8e61018e6777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:35.691117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3e204-b752-4b2f-8d39-8e61018e6777 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:35.691395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3e204-b752-4b2f-8d39-8e61018e6777 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:35.691825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3e204-b752-4b2f-8d39-8e61018e6777 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:35.693412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3e204-b752-4b2f-8d39-8e61018e6777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:35.695008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3e204-b752-4b2f-8d39-8e61018e6777 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:35.697591 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 73/140] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:35.703806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46529c69-b3a8-4060-bff1-20023736cc15 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:35.741194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:35.870964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46529c69-b3a8-4060-bff1-20023736cc15 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:35.925813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ac24e-0162-4daa-914e-18bd7873233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:35.926604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ac24e-0162-4daa-914e-18bd7873233b None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:35.926988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ac24e-0162-4daa-914e-18bd7873233b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:35.927126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ac24e-0162-4daa-914e-18bd7873233b None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:35.927740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ac24e-0162-4daa-914e-18bd7873233b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:35.929290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ac24e-0162-4daa-914e-18bd7873233b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:35.965025 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 68/141] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:38:35 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:38:36.295059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426acda-e996-4ce5-9bf2-c7eb7527c4c4 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:36.295312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426acda-e996-4ce5-9bf2-c7eb7527c4c4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:36.295673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426acda-e996-4ce5-9bf2-c7eb7527c4c4 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:36.297120 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 74/142] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:36.307923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e41de-7ed3-4be3-947c-bd687df147cf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:36.308713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e41de-7ed3-4be3-947c-bd687df147cf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:36.308713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e41de-7ed3-4be3-947c-bd687df147cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:36.338105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-341e41de-7ed3-4be3-947c-bd687df147cf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:36.339410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-341e41de-7ed3-4be3-947c-bd687df147cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:36.442743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 69/143] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:37.315616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426acda-e996-4ce5-9bf2-c7eb7527c4c4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:37.457826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9165b7-a1b1-4837-bec4-3ab5d070c835 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:37.458618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9165b7-a1b1-4837-bec4-3ab5d070c835 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:37.458618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9165b7-a1b1-4837-bec4-3ab5d070c835 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:37.460172 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 70/144] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:37.461828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426acda-e996-4ce5-9bf2-c7eb7527c4c4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:37.466236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91974933-44fd-40fd-9adc-d2cbc1ab2119 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:37.466759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91974933-44fd-40fd-9adc-d2cbc1ab2119 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:37.467180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91974933-44fd-40fd-9adc-d2cbc1ab2119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:37.487454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:37.501648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-91974933-44fd-40fd-9adc-d2cbc1ab2119 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:37.503342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-91974933-44fd-40fd-9adc-d2cbc1ab2119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:37.510747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:37.511483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:37.511767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:37.512034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:37.541061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:37.550170 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:37.551152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:37.552367 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:37.552978 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 75/145] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:37.562410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:37.683456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 71/146] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:37.701333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:37.781549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2e1d2-e5ee-43dd-917e-e818f5deca89 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:37.782063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2e1d2-e5ee-43dd-917e-e818f5deca89 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:37.782502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2e1d2-e5ee-43dd-917e-e818f5deca89 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:37.784412 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 72/147] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:37.791938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ee475-9c1c-4959-96a8-ab1f4454cc2a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:37.792249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ee475-9c1c-4959-96a8-ab1f4454cc2a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:37.792505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ee475-9c1c-4959-96a8-ab1f4454cc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:37.815518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bf30d-40d2-4c7d-a158-b7897d418a42 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:37.835304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9b7ee475-9c1c-4959-96a8-ab1f4454cc2a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:37.836985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b7ee475-9c1c-4959-96a8-ab1f4454cc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:37.856687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:37.857483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:37.857938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:37.858345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:37.859181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:37.859794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:37.861675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:37.873134 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 76/148] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:37.885879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:38.031289 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 73/149] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:38.105346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dacca-7246-4ab4-ab10-89b2a0af9c58 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:38.125628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:38.155326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:38.155994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:38.156271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:38.156546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] PATH_INFO: `/v3/users/6f14bb62d7544723b5d26e0834c8884c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:38.157849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] RBAC: Authorizing `identity:get_user(user_id=6f14bb62d7544723b5d26e0834c8884c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:38.159835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:38.161732 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 77/150] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:37 2022] GET /identity/v3/users/6f14bb62d7544723b5d26e0834c8884c => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:38.171837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ee475-9c1c-4959-96a8-ab1f4454cc2a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:38.318364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:38.335593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:38.403053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ee475-9c1c-4959-96a8-ab1f4454cc2a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:38.516022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:38.518367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:38.519079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:38.519916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:38.526193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:38.529745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:38.533484 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 74/151] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:38 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:38.543968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:38.563989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e950a54-b3d8-4584-af6b-a934b60c7909 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:38.643924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:38.645005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:38.645462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:38.645905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:38.670383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:38.676089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:38.677133 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:38.677905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:38.680302 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 78/152] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:38.686341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:38.767897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:38.852707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30f2d8-32b6-4685-9652-3dad63c12ca1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:38.903998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:38.904633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:38.909427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:38.909700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:38.910559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:38.910971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:38.912936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:38.926297 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 79/153] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:38.931763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba5002-3217-439c-afaf-3fba01774678 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:38.935278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:39.032112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:39.033222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:39.033505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:39.034336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/6f14bb62d7544723b5d26e0834c8884c/roles/338edc307ed345d6b18db9185e15aa19` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:39.038013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e71bfc5cdae43b9baf98881e1a87f1c, user_id=6f14bb62d7544723b5d26e0834c8884c, role_id=338edc307ed345d6b18db9185e15aa19)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:39.039542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:39.048434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:39.085098 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 75/154] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:38 2022] PUT /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/users/6f14bb62d7544723b5d26e0834c8884c/roles/338edc307ed345d6b18db9185e15aa19 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:38:39.118388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac48bad-b234-44db-bb3f-b693709e5b97 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:39.120490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:39.125511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:39.185575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:39.186180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:39.187129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:39.187498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] PATH_INFO: `/v3/users/7b007883f8f945d6b63ec8a2752bb899` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:39.188931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] RBAC: Authorizing `identity:get_user(user_id=7b007883f8f945d6b63ec8a2752bb899)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:39.190727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:39.196517 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 80/155] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:38 2022] GET /identity/v3/users/7b007883f8f945d6b63ec8a2752bb899 => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:39.221927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:39.419040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d931f-6d68-4b57-a678-0a56912bb013 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:39.423407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55d35c-736e-4ca0-8d7e-7fe544dd5f04 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:39.491506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:39.492958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:39.493863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:39.494546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:39.518179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:39.531832 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:39.534284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:39.536035 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:39.540861 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 76/156] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:38:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:39.541812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:39.542939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:39.543323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:39.543765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:39.547505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:39.669228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:39.712325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:39.796141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa8c9e-ed94-480d-a518-de03b3cef9ec None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:39.806302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:39.808945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:39.907474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:39.909615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:39.910104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:39.910637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:39.911920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:39.912445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:39.915169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:39.944170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 77/157] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:38:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:39.950665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:40.080126 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 81/158] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 860 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:40.104951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:40.216180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce8e68-1e01-4151-afbf-eca3d15750b7 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:40.263881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:40.308774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:40.309531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:40.309817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:40.310318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:40.312549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:40.314718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:40.316942 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 78/159] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:39 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:40.321795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:40.323189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:40.388693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b1fe-f552-4fb6-9b08-e37aa701114e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:40.482064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:40.483297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:40.483754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:40.484102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:40.486354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:40.488255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:40.497637 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 82/160] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:40 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:40.504122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:40.590074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9912c-af1d-4f8d-94d3-7b7a9a2e5024 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:40.669223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:40.670102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:40.670419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:40.670820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:40.778361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e223aa9-4a0d-44ab-a78c-8a091442738d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:40.815452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:40.888511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:40.897786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:40.924034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2d410-a5b5-487e-ad72-50b4dc50e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:40.924921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2d410-a5b5-487e-ad72-50b4dc50e1cb None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:40.925332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2d410-a5b5-487e-ad72-50b4dc50e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:40.925732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2d410-a5b5-487e-ad72-50b4dc50e1cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:40.927060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2d410-a5b5-487e-ad72-50b4dc50e1cb None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=7b007883f8f945d6b63ec8a2752bb899) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:40.927765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2d410-a5b5-487e-ad72-50b4dc50e1cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:40.930221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2d410-a5b5-487e-ad72-50b4dc50e1cb None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:40.956502 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 83/161] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:40 2022] GET /identity/v3/role_assignments?user.id=7b007883f8f945d6b63ec8a2752bb899&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:40.958768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c18b-0fbc-4121-a3e2-268cf763537c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:40.959954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c18b-0fbc-4121-a3e2-268cf763537c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:40.960359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c18b-0fbc-4121-a3e2-268cf763537c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:40.967655 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 84/162] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:40.969923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ced3e5-496a-4c6c-bf43-767808249ab1 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:40.970360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ced3e5-496a-4c6c-bf43-767808249ab1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:40.970776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ced3e5-496a-4c6c-bf43-767808249ab1 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:40.985971 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 85/163] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:40 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:40.988077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a9c5-c2b4-4bfc-be3d-c4bf49590c81 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:40.988541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a9c5-c2b4-4bfc-be3d-c4bf49590c81 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:40.989109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a9c5-c2b4-4bfc-be3d-c4bf49590c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:41.009428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:41.042048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-23d3a9c5-c2b4-4bfc-be3d-c4bf49590c81 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:41.043348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23d3a9c5-c2b4-4bfc-be3d-c4bf49590c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:41.342284 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 86/164] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:41.343683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ccf18-5810-412a-b686-cff7b45c66df None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:41.343971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ccf18-5810-412a-b686-cff7b45c66df None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:41.344240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ccf18-5810-412a-b686-cff7b45c66df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:41.345258 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 79/165] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1022 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:41.352573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:41.396400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c78ccf18-5810-412a-b686-cff7b45c66df None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:41.397745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c78ccf18-5810-412a-b686-cff7b45c66df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:41.586957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:41.593450 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 87/166] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:41.617319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9af2b-4c14-4c58-83f1-856655ec7b7e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:41.682091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:41.682761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:41.683035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:41.683307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:41.684472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:41.685979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:41.688023 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 80/167] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:41 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:41.694110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ccf18-5810-412a-b686-cff7b45c66df None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:41.793036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:41.871501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ccf18-5810-412a-b686-cff7b45c66df None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:41.945917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc7cc4-8928-4ffc-be18-19bbd17d7ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:41.946967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc7cc4-8928-4ffc-be18-19bbd17d7ea0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:41.947445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc7cc4-8928-4ffc-be18-19bbd17d7ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:41.947900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc7cc4-8928-4ffc-be18-19bbd17d7ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:41.949303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc7cc4-8928-4ffc-be18-19bbd17d7ea0 None admin] RBAC: Adding query filter params (role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=cc2b0cd654dd4bf8a616c60ea6b268d3) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:41.949783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc7cc4-8928-4ffc-be18-19bbd17d7ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:41.952509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc7cc4-8928-4ffc-be18-19bbd17d7ea0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:41.963940 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 88/168] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:41 2022] GET /identity/v3/role_assignments?user.id=cc2b0cd654dd4bf8a616c60ea6b268d3&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=6f0bec1f01514029a1162fe631d80648 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:42.170109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:42.376377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d853-14f6-4ea2-8566-be3b5c15fb10 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:42.380224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d853-14f6-4ea2-8566-be3b5c15fb10 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:42.380224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d853-14f6-4ea2-8566-be3b5c15fb10 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:42.380224 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 89/169] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:42.395570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f44f42-dc1b-4599-ae99-e852b31657ac None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:42.396037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f44f42-dc1b-4599-ae99-e852b31657ac None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:42.400694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f44f42-dc1b-4599-ae99-e852b31657ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:42.468764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d9f44f42-dc1b-4599-ae99-e852b31657ac None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:42.468764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d9f44f42-dc1b-4599-ae99-e852b31657ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:42.633204 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 90/170] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:42.693514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d3a2d-fe5e-4611-8b15-1b6820eeefdf None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:42.827030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:42.828480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:42.829180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:42.829846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:42.892875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:42.908472 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:42.910929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:42.912800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:42.916527 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 81/171] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 748 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:42.929435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f44f42-dc1b-4599-ae99-e852b31657ac None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:43.102657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:43.112012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:43.155707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:43.182209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f44f42-dc1b-4599-ae99-e852b31657ac None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:43.275736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3cb94-7585-42d1-8f9b-68d8540273f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:43.276386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3cb94-7585-42d1-8f9b-68d8540273f9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:43.276682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3cb94-7585-42d1-8f9b-68d8540273f9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:43.276964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3cb94-7585-42d1-8f9b-68d8540273f9 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:43.277646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3cb94-7585-42d1-8f9b-68d8540273f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:43.277946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3cb94-7585-42d1-8f9b-68d8540273f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:43.279363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3cb94-7585-42d1-8f9b-68d8540273f9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:43.296362 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 91/172] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:43.307165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9625a-afcc-417f-9027-bcfea266144d None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:43.307677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9625a-afcc-417f-9027-bcfea266144d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:43.308089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9625a-afcc-417f-9027-bcfea266144d None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:43.309729 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 92/173] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:38:43.310662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9625a-afcc-417f-9027-bcfea266144d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:43.393331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63e8d6-0fa5-4ed2-92bd-3ae4df19c330 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:43.442779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9625a-afcc-417f-9027-bcfea266144d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:43.460906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:43.462631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:43.463413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:43.464228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:43.481082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a89190-c8b3-4e86-8912-5fc3f8d86ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:43.481531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a89190-c8b3-4e86-8912-5fc3f8d86ea5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:43.481684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a89190-c8b3-4e86-8912-5fc3f8d86ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:43.481932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a89190-c8b3-4e86-8912-5fc3f8d86ea5 None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:43.483130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a89190-c8b3-4e86-8912-5fc3f8d86ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:43.484607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a89190-c8b3-4e86-8912-5fc3f8d86ea5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:43.486281 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 93/174] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:43 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:43.487596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94cbd-9595-48b8-9c11-9cbb69a6e95e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:43.487919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94cbd-9595-48b8-9c11-9cbb69a6e95e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:43.488224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94cbd-9595-48b8-9c11-9cbb69a6e95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:43.490103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:43.502341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:43.505362 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:43.507360 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:43.509403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:43.509403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d8f94cbd-9595-48b8-9c11-9cbb69a6e95e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:43.509403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d8f94cbd-9595-48b8-9c11-9cbb69a6e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:43.512699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 82/175] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:43.514553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:43.604204 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 94/176] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:43.606380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f94cbd-9595-48b8-9c11-9cbb69a6e95e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:43.641448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:43.684120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2b62c-965c-4735-ae69-81c7b36089b8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:43.746581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:43.748139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:43.748627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:43.749478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:43.775061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:43.789629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:43.791371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 83/177] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:43.797486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:43.798576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f94cbd-9595-48b8-9c11-9cbb69a6e95e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:43.821822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:43.841551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:43.842472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:43.842891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:43.843300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:43.941270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:43.943064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:44.002882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b515f8-2d62-4bc2-8868-cee046b665cf None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:44.036398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:44.055171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3712ca-9841-41c6-8a54-8d32b80ff3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:44.055946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3712ca-9841-41c6-8a54-8d32b80ff3e3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.056289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3712ca-9841-41c6-8a54-8d32b80ff3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.056627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3712ca-9841-41c6-8a54-8d32b80ff3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.057428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3712ca-9841-41c6-8a54-8d32b80ff3e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:44.057800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3712ca-9841-41c6-8a54-8d32b80ff3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:44.059291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3712ca-9841-41c6-8a54-8d32b80ff3e3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:44.072021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 84/178] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:44.078434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc069-f890-4f86-9f7c-dae422e3a266 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.078693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc069-f890-4f86-9f7c-dae422e3a266 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.079274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc069-f890-4f86-9f7c-dae422e3a266 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.084815 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 85/179] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:44.084815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dc069-f890-4f86-9f7c-dae422e3a266 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:44.129663 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 95/180] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:44.130977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:44.164454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:44.279726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dc069-f890-4f86-9f7c-dae422e3a266 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:44.305849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bc437-5340-4e68-a78f-7201dc0001eb None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:44.344898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78cc58-a040-4cdc-81a9-7cc5467f3fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:44.345901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78cc58-a040-4cdc-81a9-7cc5467f3fbb None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.346370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78cc58-a040-4cdc-81a9-7cc5467f3fbb None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.346845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78cc58-a040-4cdc-81a9-7cc5467f3fbb None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.348137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78cc58-a040-4cdc-81a9-7cc5467f3fbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:44.348619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78cc58-a040-4cdc-81a9-7cc5467f3fbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:44.351207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78cc58-a040-4cdc-81a9-7cc5467f3fbb None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:44.354706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:44.355899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.356380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.356885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.358360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:44.360124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:44.362873 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 96/181] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:44.364747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:44.372804 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 86/182] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:44.375539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b85c48-74a3-436a-ad97-788ba6d985b1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.376007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b85c48-74a3-436a-ad97-788ba6d985b1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.376513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b85c48-74a3-436a-ad97-788ba6d985b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.407560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d5b85c48-74a3-436a-ad97-788ba6d985b1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:44.410082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5b85c48-74a3-436a-ad97-788ba6d985b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:44.499686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:44.507286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:44.522461 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 87/183] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:44.524069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b85c48-74a3-436a-ad97-788ba6d985b1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:44.542480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829903f-857d-4f06-8b67-5330f4278996 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:44.621236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:44.622810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.623579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.624774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] PATH_INFO: `/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.629374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] RBAC: Authorizing `identity:get_user(user_id=cc2b0cd654dd4bf8a616c60ea6b268d3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:44.632575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:44.636444 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 97/184] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:44 2022] GET /identity/v3/users/cc2b0cd654dd4bf8a616c60ea6b268d3 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:44.638254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:44.748652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b85c48-74a3-436a-ad97-788ba6d985b1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:44.812075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:44.813180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.813728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.814326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] PATH_INFO: `/v3/projects/29febf817c7542668ddf224f88054212` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.816704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] RBAC: Authorizing `identity:get_project(project_id=29febf817c7542668ddf224f88054212)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:44.819497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:44.821695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c93386-722a-4f77-abf4-5d7f47c8b22a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:44.827529 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 88/185] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:44 2022] GET /identity/v3/projects/29febf817c7542668ddf224f88054212 => generated 312 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:44.827529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:44.871723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:44.872844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:44.873314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:44.873753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:44.874866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:44.876468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:44.904690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 98/186] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:38:44 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:44.905975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:44.923739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:44.936578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:45.026824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fef48-f56d-479d-9a3b-6475cb34a2be None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:45.076105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:45.076739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.077640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.078011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] PATH_INFO: `/v3/users/85b8193783724940bbe6d21782d38286` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.096282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] RBAC: Authorizing `identity:get_user(user_id=85b8193783724940bbe6d21782d38286)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:45.097857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:45.099997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 89/187] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:44 2022] GET /identity/v3/users/85b8193783724940bbe6d21782d38286 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:45.100864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:45.115789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3f08e-c6a9-46ae-a735-4f9e189378f1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:45.157326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddfc2b-3b66-46f3-b7f8-9454a893acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:45.157976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddfc2b-3b66-46f3-b7f8-9454a893acd8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.158239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddfc2b-3b66-46f3-b7f8-9454a893acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.158510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddfc2b-3b66-46f3-b7f8-9454a893acd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.260373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddfc2b-3b66-46f3-b7f8-9454a893acd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:45.263075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddfc2b-3b66-46f3-b7f8-9454a893acd8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:45.283775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:45.291679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da0459-d13f-4add-8b77-cbc258ef96c4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:45.309263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:45.344629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:45.345771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.346234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.347189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.348610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=29febf817c7542668ddf224f88054212, user.id=cc2b0cd654dd4bf8a616c60ea6b268d3) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:45.349148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:45.351196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:45.359135 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 90/188] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:45 2022] GET /identity/v3/role_assignments?user.id=cc2b0cd654dd4bf8a616c60ea6b268d3&scope.project.id=29febf817c7542668ddf224f88054212&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:45.360416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:45.471574 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 99/189] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:45.474974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f5b15-69c3-4f45-9682-001b101601bf None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.475472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f5b15-69c3-4f45-9682-001b101601bf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.476107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f5b15-69c3-4f45-9682-001b101601bf None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.478897 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 100/190] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:45.480488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f5b15-69c3-4f45-9682-001b101601bf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:45.566012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d99fc-a142-47cc-922b-27817e856682 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:45.633740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:45.634793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.635261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.635735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.663153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:45.671224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f5b15-69c3-4f45-9682-001b101601bf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:45.677088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:45.678080 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:45.678885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:45.679497 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 91/191] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:38:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:45.680345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:45.704382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:45.734315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:45.768628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:45.769443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.769648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.770063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.846415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d01c5-ec33-4647-aab0-b17ec42eec64 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:45.878761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:45.882457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:45.905602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72e526-8ce2-4b11-8d47-261b2113139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:45.907238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72e526-8ce2-4b11-8d47-261b2113139b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.908014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72e526-8ce2-4b11-8d47-261b2113139b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.909223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72e526-8ce2-4b11-8d47-261b2113139b None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.911762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72e526-8ce2-4b11-8d47-261b2113139b None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:45.915386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72e526-8ce2-4b11-8d47-261b2113139b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:45.919117 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 92/192] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:45 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:45.922044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac60b7-cf9d-4edb-8913-d1904008a4b4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:45.922887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac60b7-cf9d-4edb-8913-d1904008a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:45.923697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac60b7-cf9d-4edb-8913-d1904008a4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:45.959744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ebac60b7-cf9d-4edb-8913-d1904008a4b4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:45.962724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ebac60b7-cf9d-4edb-8913-d1904008a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:46.028485 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 101/193] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:46.030706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:46.059288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:46.087618 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 93/194] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:46.088632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac60b7-cf9d-4edb-8913-d1904008a4b4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:46.162263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:46.271109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a709f-bf6b-46e5-900e-31443b8b3c1a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:46.320478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac60b7-cf9d-4edb-8913-d1904008a4b4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:46.365508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:46.366157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:46.366740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:46.367028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:46.367800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=cc2b0cd654dd4bf8a616c60ea6b268d3) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:46.368114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:46.369649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:46.377309 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 94/195] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:46 2022] GET /identity/v3/role_assignments?user.id=cc2b0cd654dd4bf8a616c60ea6b268d3&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:46.378670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:46.402231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:46.402884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:46.403159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:46.403421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:46.404119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:46.404414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:46.405952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:46.438869 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 102/196] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:38:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:46.445581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:46.592997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:46.606323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b68f2-4b3a-42da-a6af-8cbf61922c11 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:46.638589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:46.669586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a91d20-5248-4325-addd-8aaf31a026b9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:46.713094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:46.714036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:46.714512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:46.714954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] PATH_INFO: `/v3/users/85b8193783724940bbe6d21782d38286` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:46.717550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] RBAC: Authorizing `identity:get_user(user_id=85b8193783724940bbe6d21782d38286)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:46.720216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:46.723322 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 103/197] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:46 2022] GET /identity/v3/users/85b8193783724940bbe6d21782d38286 => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:46.724861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:46.729486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:46.730165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:46.730447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:46.730719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:46.734514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:46.736215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:46.738381 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 95/198] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:46 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:46.749808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:46.972726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84234b33-5fef-4991-96ee-f646440dd9f9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:47.015092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0ad54-b1f5-4ac1-8871-0b38f88dc9c5 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:47.057708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:47.058798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:47.059257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:47.059722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:47.078067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:47.085094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:47.096524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:47.097592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:47.098143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:47.098319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:47.099514 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:47.100317 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:47.100938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:47.100938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=85b8193783724940bbe6d21782d38286) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:47.100938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:47.101132 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 104/199] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:47.102693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:47.106268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:47.114664 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 96/200] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:46 2022] GET /identity/v3/role_assignments?user.id=85b8193783724940bbe6d21782d38286&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:47.117322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:47.188029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:47.362057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc65275-0e36-41c7-96f2-6a024434ba51 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:47.443574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1424fa3-8e87-4e54-8fb6-7e84b03c8521 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:47.505547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:47.506195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:47.506473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:47.506745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:47.507453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:47.507759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:47.509265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:47.523940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014c8ec-cd59-47c5-9d6a-1629173e25d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:47.524951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014c8ec-cd59-47c5-9d6a-1629173e25d2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:47.525397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014c8ec-cd59-47c5-9d6a-1629173e25d2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:47.525846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014c8ec-cd59-47c5-9d6a-1629173e25d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:47.535202 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 97/201] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:47.549881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:47.875008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:47.885683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014c8ec-cd59-47c5-9d6a-1629173e25d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:47.887222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014c8ec-cd59-47c5-9d6a-1629173e25d2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:47.999573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:48.061841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3218-4b31-4a79-bab2-ec9757b769dd None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:48.186192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:48.187178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:48.187651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:48.188136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] PATH_INFO: `/v3/users/6f14bb62d7544723b5d26e0834c8884c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:48.193502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] RBAC: Authorizing `identity:get_user(user_id=6f14bb62d7544723b5d26e0834c8884c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:48.196289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:48.208771 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 98/202] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:47 2022] GET /identity/v3/users/6f14bb62d7544723b5d26e0834c8884c => generated 289 bytes in 659 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:48.217783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:48.303494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 105/203] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1202 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:48.306740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876ba7-6b00-4810-841c-0e76b5a326ab None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:48.307463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876ba7-6b00-4810-841c-0e76b5a326ab None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:48.308152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876ba7-6b00-4810-841c-0e76b5a326ab None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:48.311041 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 106/204] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:48.318337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e876ba7-6b00-4810-841c-0e76b5a326ab None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:48.451884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a659-251f-4f2a-a5f2-e356428f14fa None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:48.516506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:48.517210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98df4e-93e5-489b-a1b4-94b30b4e4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:48.520185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98df4e-93e5-489b-a1b4-94b30b4e4963 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:48.521090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98df4e-93e5-489b-a1b4-94b30b4e4963 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:48.522181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98df4e-93e5-489b-a1b4-94b30b4e4963 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:48.523413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e876ba7-6b00-4810-841c-0e76b5a326ab None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:48.581821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:48.583221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91faec-93d1-4d15-b703-e116afba8d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:48.584556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91faec-93d1-4d15-b703-e116afba8d31 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:48.585156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91faec-93d1-4d15-b703-e116afba8d31 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:48.585899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91faec-93d1-4d15-b703-e116afba8d31 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:48.588049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91faec-93d1-4d15-b703-e116afba8d31 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:48.590704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91faec-93d1-4d15-b703-e116afba8d31 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:48.596753 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 107/205] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:48 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:48.596753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc0a40-9f8d-4757-8455-523b56b0e56c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:48.596753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc0a40-9f8d-4757-8455-523b56b0e56c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:48.596753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc0a40-9f8d-4757-8455-523b56b0e56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:48.627107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a7cc0a40-9f8d-4757-8455-523b56b0e56c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:48.628986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a7cc0a40-9f8d-4757-8455-523b56b0e56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:48.645078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98df4e-93e5-489b-a1b4-94b30b4e4963 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:48.646684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98df4e-93e5-489b-a1b4-94b30b4e4963 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:48.758550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 108/206] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:48.760504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e56b4c-dacd-4727-b5d2-a362d3ffdf93 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:48.761010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e56b4c-dacd-4727-b5d2-a362d3ffdf93 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:48.761470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e56b4c-dacd-4727-b5d2-a362d3ffdf93 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:48.770817 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 109/207] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:48.770817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e56b4c-dacd-4727-b5d2-a362d3ffdf93 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:48.813614 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 99/208] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:48.817528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02e33-fd28-41a8-8322-f8f2ae264446 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:48.818255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02e33-fd28-41a8-8322-f8f2ae264446 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:48.819309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02e33-fd28-41a8-8322-f8f2ae264446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:48.856849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-88a02e33-fd28-41a8-8322-f8f2ae264446 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:48.859203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-88a02e33-fd28-41a8-8322-f8f2ae264446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:48.957793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:48.966752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:48.968766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e56b4c-dacd-4727-b5d2-a362d3ffdf93 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:48.993214 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 100/209] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:48.995184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02e33-fd28-41a8-8322-f8f2ae264446 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:49.011175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:49.017416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:49.017416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:49.017416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:49.017416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] RBAC: Adding query filter params (role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=85b8193783724940bbe6d21782d38286) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:49.017416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:49.018075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:49.021749 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 110/210] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:48 2022] GET /identity/v3/role_assignments?user.id=85b8193783724940bbe6d21782d38286&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=6f0bec1f01514029a1162fe631d80648 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:49.237854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02e33-fd28-41a8-8322-f8f2ae264446 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:49.302537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:49.304043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:49.304818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:49.305576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:49.307809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:49.310795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:49.314161 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 101/211] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:48 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:49.320527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:49.357865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:49.477176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:49.506236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eee73-cc4d-450d-933c-446b6e967665 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:49.549369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:49.550387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:49.550860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:49.551441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:49.552397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] RBAC: Adding query filter params (role.id=338edc307ed345d6b18db9185e15aa19, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c, user.id=6f14bb62d7544723b5d26e0834c8884c) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:49.552999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:49.554748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:49.562612 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 111/212] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:49 2022] GET /identity/v3/role_assignments?user.id=6f14bb62d7544723b5d26e0834c8884c&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=338edc307ed345d6b18db9185e15aa19 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:49.605943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8ea7-60f1-4123-b568-28a46fae7c20 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:49.680961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:49.706371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:49.707052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:49.707941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:49.708347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:49.720122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:49.749841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:49.756038 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:49.757053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:49.757840 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:49.765697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 102/213] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:38:49.778310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:49.824324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b5523-1105-414e-95b4-6164610edd66 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:49.867988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:49.869129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:49.869609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:49.870144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:49.872457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:49.874423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:49.876943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 112/214] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:49.884583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:50.013474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:50.032443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82d4b-f7ab-4377-a9c6-e5d4093860d6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:50.103207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f455ce3-947e-428b-87df-3b48d6c34883 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:50.105796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:50.105796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:50.106157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:50.106526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:50.108191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:50.112036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:50.146073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:50.151605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 113/215] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:38:49 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:50.184130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7e16-e3a2-4134-a962-99068c5066c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:50.185654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7e16-e3a2-4134-a962-99068c5066c3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:50.186394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7e16-e3a2-4134-a962-99068c5066c3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:50.187174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7e16-e3a2-4134-a962-99068c5066c3 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:50.188695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7e16-e3a2-4134-a962-99068c5066c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:50.189452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7e16-e3a2-4134-a962-99068c5066c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:50.192024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7e16-e3a2-4134-a962-99068c5066c3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:50.211102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 103/216] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:50.217824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:50.400145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1d85-751b-4522-ac9c-0d19483d819b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:50.401864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1d85-751b-4522-ac9c-0d19483d819b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:50.402162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1d85-751b-4522-ac9c-0d19483d819b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:50.403964 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 104/217] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:50.410522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed95391-560f-4cc3-abe9-8ac7f13c9a44 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:50.411010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed95391-560f-4cc3-abe9-8ac7f13c9a44 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:50.411400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed95391-560f-4cc3-abe9-8ac7f13c9a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:50.433633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5ed95391-560f-4cc3-abe9-8ac7f13c9a44 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:50.434913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5ed95391-560f-4cc3-abe9-8ac7f13c9a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:50.472380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d771ce-7e35-44ab-976f-f95eea071e72 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:50.516969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:50.517955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:50.518236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:50.518508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] PATH_INFO: `/v3/users/e1e1c2b1360149bd8d8016577c9d044c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:50.538414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] RBAC: Authorizing `identity:get_user(user_id=e1e1c2b1360149bd8d8016577c9d044c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:50.539909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:50.543896 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 114/218] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:50 2022] GET /identity/v3/users/e1e1c2b1360149bd8d8016577c9d044c => generated 297 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:50.547774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:50.548847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 105/219] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:38:50.577697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:50.710186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eff828-a31c-41d3-b3bd-14c13fa19ecc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:50.765761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:50.766802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:50.767284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:50.767762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:50.901821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:50.904618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:50.910201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed95391-560f-4cc3-abe9-8ac7f13c9a44 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:50.925211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:51.093393 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 115/220] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:51.102043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:51.203957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed95391-560f-4cc3-abe9-8ac7f13c9a44 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:51.227983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:51.285053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:51.285833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:51.286042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:51.286322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:51.314269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:51.325581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:51.327520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:51.329340 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 106/221] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:51.336332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:51.346687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4346e0-56ff-4d0e-99eb-df427bd5d18e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:51.387412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:51.446080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:51.447674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:51.448420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:51.451213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:51.453690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:51.460675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:51.468754 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 116/222] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:51 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:51.485994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:51.699642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47099-afd2-4055-b8dd-a427ac7d4e4b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:51.834985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:51.836028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:51.836506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:51.837022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:51.838304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:51.838834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:51.841624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:51.849434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081640a-9f37-4603-8e26-1dbb195dab08 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:51.885602 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 107/223] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:51.892351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:51.948322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:51.949503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:51.949961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:51.950414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:51.955885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=e1e1c2b1360149bd8d8016577c9d044c) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:51.955885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:51.955885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:51.966719 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 117/224] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:51 2022] GET /identity/v3/role_assignments?user.id=e1e1c2b1360149bd8d8016577c9d044c&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 259 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:52.000004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:52.156064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff2055-cf27-4147-a17a-ca824986d3ae None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:52.244879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:52.245978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:52.246437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:52.246898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] PATH_INFO: `/v3/users/85b8193783724940bbe6d21782d38286` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:52.249520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] RBAC: Authorizing `identity:get_user(user_id=85b8193783724940bbe6d21782d38286)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:52.251025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:52.252765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 108/225] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:51 2022] GET /identity/v3/users/85b8193783724940bbe6d21782d38286 => generated 283 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:52.258853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:52.312643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:52.381050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c95654-53cf-4715-b3ed-1078d0e5f8a8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:52.419313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd459e5e-4170-4d99-bcb1-1aabcb2ed4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:52.419866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459e5e-4170-4d99-bcb1-1aabcb2ed4c8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:52.420138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459e5e-4170-4d99-bcb1-1aabcb2ed4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:52.420414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459e5e-4170-4d99-bcb1-1aabcb2ed4c8 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:52.421638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd459e5e-4170-4d99-bcb1-1aabcb2ed4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:52.423146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd459e5e-4170-4d99-bcb1-1aabcb2ed4c8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:52.425154 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 118/226] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:52 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:38:52.433832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:52.579264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:52.583867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c240f9-9be0-46e4-829a-f551282349b0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:52.655375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c1a7-d074-48e1-b9b6-bb08556ac750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:52.656085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c1a7-d074-48e1-b9b6-bb08556ac750 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:52.656367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c1a7-d074-48e1-b9b6-bb08556ac750 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:52.656644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c1a7-d074-48e1-b9b6-bb08556ac750 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9/users/85b8193783724940bbe6d21782d38286/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:52.660613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176c1a7-d074-48e1-b9b6-bb08556ac750 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5015560e7c748678c104319614c21f9, user_id=85b8193783724940bbe6d21782d38286, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:52.662132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176c1a7-d074-48e1-b9b6-bb08556ac750 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:52.693327 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 109/227] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:52 2022] PUT /identity/v3/projects/c5015560e7c748678c104319614c21f9/users/85b8193783724940bbe6d21782d38286/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:38:53.412955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b251-7449-474e-a270-4404c0b747b5 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:53.413760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b251-7449-474e-a270-4404c0b747b5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:53.413760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b251-7449-474e-a270-4404c0b747b5 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:53.415169 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 119/228] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:53.422272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba54dc5-63a8-4246-a7a5-3bc8df4453fc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:53.423225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba54dc5-63a8-4246-a7a5-3bc8df4453fc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:53.423225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba54dc5-63a8-4246-a7a5-3bc8df4453fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:53.457375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bba54dc5-63a8-4246-a7a5-3bc8df4453fc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:53.460263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bba54dc5-63a8-4246-a7a5-3bc8df4453fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:53.618802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 110/229] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:53.746904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd80c-92de-4221-8e61-fd60fade1d21 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:53.747139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd80c-92de-4221-8e61-fd60fade1d21 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:53.747476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd80c-92de-4221-8e61-fd60fade1d21 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:53.749161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 120/230] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:53.763663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666435-cb14-4282-af36-b9ee942406d4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:53.764547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666435-cb14-4282-af36-b9ee942406d4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:53.765338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666435-cb14-4282-af36-b9ee942406d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:53.806469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6f666435-cb14-4282-af36-b9ee942406d4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:53.808970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6f666435-cb14-4282-af36-b9ee942406d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:53.907403 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 111/231] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:54.117781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd80c-92de-4221-8e61-fd60fade1d21 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:54.288411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd80c-92de-4221-8e61-fd60fade1d21 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:54.301400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:54.358320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:54.359491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:54.359971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:54.360495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:54.382958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:54.392757 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:54.393724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:38:54.394518 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:54.397365 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 121/232] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:38:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:38:54.405778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f666435-cb14-4282-af36-b9ee942406d4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:54.452410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:54.540086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:54.599721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f666435-cb14-4282-af36-b9ee942406d4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:54.679137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81f04-aa78-4658-b3ae-7cbdd1d8acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:54.679682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81f04-aa78-4658-b3ae-7cbdd1d8acd0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:54.679958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81f04-aa78-4658-b3ae-7cbdd1d8acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:54.680225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81f04-aa78-4658-b3ae-7cbdd1d8acd0 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:54.680972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81f04-aa78-4658-b3ae-7cbdd1d8acd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:54.681275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81f04-aa78-4658-b3ae-7cbdd1d8acd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:54.682703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81f04-aa78-4658-b3ae-7cbdd1d8acd0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:54.693781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 112/233] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:38:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:54.695520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34dcc1-75f1-4f67-aa09-ebb7f760a701 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:54.695800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34dcc1-75f1-4f67-aa09-ebb7f760a701 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:54.696092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34dcc1-75f1-4f67-aa09-ebb7f760a701 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:54.698431 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 113/234] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:54.705757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34dcc1-75f1-4f67-aa09-ebb7f760a701 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:54.753241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a2698-3a95-460e-97c8-b63f784fb3c7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:54.819045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d612df-b299-4346-9235-b6d5537380ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:54.820046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d612df-b299-4346-9235-b6d5537380ea None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:54.820479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d612df-b299-4346-9235-b6d5537380ea None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:54.820939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d612df-b299-4346-9235-b6d5537380ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:54.822819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d612df-b299-4346-9235-b6d5537380ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:54.826896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d612df-b299-4346-9235-b6d5537380ea None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:54.829121 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 122/235] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:38:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:54.831065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642c5ba-5b5c-430d-8cff-9e2187ce4aa1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:54.832351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642c5ba-5b5c-430d-8cff-9e2187ce4aa1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:54.832838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642c5ba-5b5c-430d-8cff-9e2187ce4aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:54.870560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d642c5ba-5b5c-430d-8cff-9e2187ce4aa1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:54.873220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d642c5ba-5b5c-430d-8cff-9e2187ce4aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:54.923674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:54.931167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:54.964866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34dcc1-75f1-4f67-aa09-ebb7f760a701 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:55.007029 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 123/236] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:55.008276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642c5ba-5b5c-430d-8cff-9e2187ce4aa1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:55.032648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:55.033760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:55.034565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:55.035026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] PATH_INFO: `/v3/users/85b8193783724940bbe6d21782d38286` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:55.037214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] RBAC: Authorizing `identity:get_user(user_id=85b8193783724940bbe6d21782d38286)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:55.039861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:55.042869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 114/237] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:54 2022] GET /identity/v3/users/85b8193783724940bbe6d21782d38286 => generated 283 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:55.050269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:55.332017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642c5ba-5b5c-430d-8cff-9e2187ce4aa1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:55.385500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41b05-3554-4afa-b8c1-e4c559dd5809 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:55.451939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:55.453082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:55.453642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:55.453960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:55.454449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:55.455551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:55.455963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:55.456068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:55.456384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:55.457729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:55.458928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:55.460177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:55.462242 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 115/238] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:55 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:55.463626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:55.500800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 124/239] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:38:55 2022] POST /identity/v3/users => generated 293 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:38:55.502660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:55.516819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:55.564262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:55.718188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d044e-3be4-4ed5-92c0-651fb5b264fa None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:55.723701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10687-c274-4763-9b4e-764468d1f6de None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:55.786321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589592cc-3663-4ae4-b51c-b40dddea2a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:55.787596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589592cc-3663-4ae4-b51c-b40dddea2a54 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:55.788161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589592cc-3663-4ae4-b51c-b40dddea2a54 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:55.788618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589592cc-3663-4ae4-b51c-b40dddea2a54 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9/users/85b8193783724940bbe6d21782d38286/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:55.792327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589592cc-3663-4ae4-b51c-b40dddea2a54 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5015560e7c748678c104319614c21f9, user_id=85b8193783724940bbe6d21782d38286, role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:55.792516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:55.793375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:55.793841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:55.794276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:55.794911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589592cc-3663-4ae4-b51c-b40dddea2a54 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:55.817816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:55.821876 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 125/240] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:55 2022] PUT /identity/v3/projects/c5015560e7c748678c104319614c21f9/users/85b8193783724940bbe6d21782d38286/roles/6f0bec1f01514029a1162fe631d80648 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:38:55.824260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a1270-b7a2-4023-851b-85cefe2850ad None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:55.826050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a1270-b7a2-4023-851b-85cefe2850ad None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:55.826528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a1270-b7a2-4023-851b-85cefe2850ad None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:55.827982 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:55.829646 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:38:55.831480 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:55.833092 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 126/241] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:55.833246 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 116/242] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:55.839339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942846d-0051-41ad-bfc9-a5aa0bd18fe3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:55.839754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942846d-0051-41ad-bfc9-a5aa0bd18fe3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:55.840167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942846d-0051-41ad-bfc9-a5aa0bd18fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:55.842205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:55.875062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f942846d-0051-41ad-bfc9-a5aa0bd18fe3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:55.876929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f942846d-0051-41ad-bfc9-a5aa0bd18fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:55.968161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:56.011171 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 127/243] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:56.022978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77a50-b698-4611-aeac-831334548e31 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:56.097675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:56.098877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:56.099418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:56.099844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:56.101041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:56.101505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:56.104353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:56.122205 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 117/244] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:56.128364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942846d-0051-41ad-bfc9-a5aa0bd18fe3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:56.155270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:56.328291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942846d-0051-41ad-bfc9-a5aa0bd18fe3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:56.373978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:56.375154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:56.375594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:56.376058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] PATH_INFO: `/v3/users/cbdb35c2ef0e4cc8854585b044ea7368` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:56.403410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] RBAC: Authorizing `identity:get_user(user_id=cbdb35c2ef0e4cc8854585b044ea7368)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:56.405884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:56.409609 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 128/245] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:56 2022] GET /identity/v3/users/cbdb35c2ef0e4cc8854585b044ea7368 => generated 297 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:56.418652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:56.552862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:56.683296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:56.706949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb7b35-b7a3-4671-8a25-f4b3b79abf25 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:56.735191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edd14e-62ff-43ce-8ede-3c52b0d0b11b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:56.832783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc96d40-ce3f-4fe8-9219-7da4d2a3d3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:56.833790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc96d40-ce3f-4fe8-9219-7da4d2a3d3f0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:56.834204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc96d40-ce3f-4fe8-9219-7da4d2a3d3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:56.834627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc96d40-ce3f-4fe8-9219-7da4d2a3d3f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:56.850878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fbbf9-7c49-4c05-a0cb-686b0f56227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:56.852522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fbbf9-7c49-4c05-a0cb-686b0f56227c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:56.853391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fbbf9-7c49-4c05-a0cb-686b0f56227c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:56.854262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fbbf9-7c49-4c05-a0cb-686b0f56227c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:56.855377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc96d40-ce3f-4fe8-9219-7da4d2a3d3f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-9fc96d40-ce3f-4fe8-9219-7da4d2a3d3f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:56.866060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:56.867766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:38:56.869391 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 129/246] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:56.870831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e83e6f-76bf-449d-84cf-0112c697d23a None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:56.870831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e83e6f-76bf-449d-84cf-0112c697d23a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:56.872314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e83e6f-76bf-449d-84cf-0112c697d23a None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:56.873717 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 130/247] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:38:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:38:56.881927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e83e6f-76bf-449d-84cf-0112c697d23a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:56.887343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:57.030603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fbbf9-7c49-4c05-a0cb-686b0f56227c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:57.032131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fbbf9-7c49-4c05-a0cb-686b0f56227c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:57.180908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e83e6f-76bf-449d-84cf-0112c697d23a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:57.304707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:57.319406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:57.320436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:57.321354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:57.321836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:57.322944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:57.323470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:57.326478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:57.339113 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 118/248] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 922 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:38:57.351069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d641-75ff-452f-b44a-0a294812c05c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:57.351501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d641-75ff-452f-b44a-0a294812c05c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:57.351978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d641-75ff-452f-b44a-0a294812c05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:57.360491 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 131/249] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:38:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:57.362688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:57.389991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2fc7d641-75ff-452f-b44a-0a294812c05c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:38:57.391290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2fc7d641-75ff-452f-b44a-0a294812c05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:38:57.509187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:57.579403 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 119/250] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:38:57.580196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7d641-75ff-452f-b44a-0a294812c05c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:57.625979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a9598-bfdf-44d0-9e00-3b6d29e3d9c2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:57.726485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:57.727513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:57.727513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:57.727777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:57.731295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:57.732834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:57.738321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 132/251] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:57 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:57.744814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:57.786648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7d641-75ff-452f-b44a-0a294812c05c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:57.861456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:57.863340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:57.863729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:57.864001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] PATH_INFO: `/v3/users/e1e1c2b1360149bd8d8016577c9d044c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:57.866086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] RBAC: Authorizing `identity:get_user(user_id=e1e1c2b1360149bd8d8016577c9d044c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:57.867560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:57.869498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:57.869822 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 120/252] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:57 2022] GET /identity/v3/users/e1e1c2b1360149bd8d8016577c9d044c => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:57.880167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:57.960949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a82ec-65f7-47ff-b5fd-f2508da5b334 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:57.966055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:58.059501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:58.061250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:58.062164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:58.062966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:58.064502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] RBAC: Adding query filter params (role.id=338edc307ed345d6b18db9185e15aa19, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=cbdb35c2ef0e4cc8854585b044ea7368) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:58.065194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:58.071856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:58.085233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56cc7-c579-4fcc-ba84-8caf8e845455 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:58.096428 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 133/253] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:38:57 2022] GET /identity/v3/role_assignments?user.id=cbdb35c2ef0e4cc8854585b044ea7368&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=338edc307ed345d6b18db9185e15aa19 => generated 259 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:58.200412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:58.201515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:58.202008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:58.202492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:58.210202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:58.213122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:58.216601 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 121/254] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:38:57 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:58.224045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:58.497709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8026-5ffa-4437-9cfa-e49ff9110511 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:58.541876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:58.561775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:58.562908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:58.563361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:58.563825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9/users/e1e1c2b1360149bd8d8016577c9d044c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:58.567265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5015560e7c748678c104319614c21f9, user_id=e1e1c2b1360149bd8d8016577c9d044c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:58.570264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:58.608451 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 134/255] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:38:58 2022] PUT /identity/v3/projects/c5015560e7c748678c104319614c21f9/users/e1e1c2b1360149bd8d8016577c9d044c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:38:58.631174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:58.913760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f10be-2d02-4f8b-a60d-bf8ddc845ee2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:58.931841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:58.998041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:58.999131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:58.999607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:59.000074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:59.032381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:38:59.046542 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:38:59.047513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:38:59.048298 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:38:59.049419 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 122/256] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:38:59.062472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:59.238519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ec450-87e9-45d6-806b-706ca53c9711 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:59.293484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:59.294527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:59.294995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:59.295473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:59.296383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:38:59.296929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:59.298541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:59.313708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 135/257] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:38:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:59.323414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:59.563800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217936d4-55a3-4cab-99b3-855459389db9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:59.657637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86776c0f-a61b-40db-b5b0-4a50821fd9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:59.658768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776c0f-a61b-40db-b5b0-4a50821fd9a7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:59.659577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776c0f-a61b-40db-b5b0-4a50821fd9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:59.660044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776c0f-a61b-40db-b5b0-4a50821fd9a7 None admin] PATH_INFO: `/v3/users/85b8193783724940bbe6d21782d38286` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:38:59.662179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86776c0f-a61b-40db-b5b0-4a50821fd9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=85b8193783724940bbe6d21782d38286)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:38:59.665097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86776c0f-a61b-40db-b5b0-4a50821fd9a7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:38:59.668557 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 123/258] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:38:59 2022] GET /identity/v3/users/85b8193783724940bbe6d21782d38286 => generated 283 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:38:59.674470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:38:59.762389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:38:59.845975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be012c83-e074-417d-b977-eb054a18a892 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:38:59.903721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:38:59.904834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:38:59.905342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:38:59.905826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:00.015873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:00.017887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:00.116692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b003e-bb42-4ecd-a6d7-d546e35fead3 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:00.117207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:00.118696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b003e-bb42-4ecd-a6d7-d546e35fead3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:00.119340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b003e-bb42-4ecd-a6d7-d546e35fead3 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:00.121422 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 124/259] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:39:00.128797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112c3a1-27b7-4d37-9543-20978e853695 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:00.130188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112c3a1-27b7-4d37-9543-20978e853695 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:00.130680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112c3a1-27b7-4d37-9543-20978e853695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:00.155875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6112c3a1-27b7-4d37-9543-20978e853695 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:00.157489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6112c3a1-27b7-4d37-9543-20978e853695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:00.245273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:00.291854 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 136/260] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:38:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:00.300996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:00.323969 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 125/261] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:00.536310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc35303-4e66-434f-b74a-0568257fb527 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:00.619874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25ec06-3383-40fc-8562-560df0431669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:00.620997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ec06-3383-40fc-8562-560df0431669 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:00.621484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ec06-3383-40fc-8562-560df0431669 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:00.621960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ec06-3383-40fc-8562-560df0431669 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:00.623948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25ec06-3383-40fc-8562-560df0431669 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:00.626605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25ec06-3383-40fc-8562-560df0431669 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:00.629849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 137/262] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:00 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:00.639415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112c3a1-27b7-4d37-9543-20978e853695 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:00.856922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112c3a1-27b7-4d37-9543-20978e853695 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:00.927364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd77d80-5acc-49b0-bdcc-8dcdf51c5759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:00.928471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd77d80-5acc-49b0-bdcc-8dcdf51c5759 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:00.929033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd77d80-5acc-49b0-bdcc-8dcdf51c5759 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:00.929487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd77d80-5acc-49b0-bdcc-8dcdf51c5759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:00.930876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd77d80-5acc-49b0-bdcc-8dcdf51c5759 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=85b8193783724940bbe6d21782d38286) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:00.931412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd77d80-5acc-49b0-bdcc-8dcdf51c5759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:00.934420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd77d80-5acc-49b0-bdcc-8dcdf51c5759 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:00.946134 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 126/263] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:00 2022] GET /identity/v3/role_assignments?user.id=85b8193783724940bbe6d21782d38286&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:01.209022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499180f-38b1-4abe-a72f-733bac5331a4 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:01.210620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499180f-38b1-4abe-a72f-733bac5331a4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:01.211310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499180f-38b1-4abe-a72f-733bac5331a4 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:01.214334 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 138/264] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:01.224180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e40653-4a7a-4b5c-89b0-92de57e4f60c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:01.225673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e40653-4a7a-4b5c-89b0-92de57e4f60c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:01.226360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e40653-4a7a-4b5c-89b0-92de57e4f60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:01.266616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d3e40653-4a7a-4b5c-89b0-92de57e4f60c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:01.269493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3e40653-4a7a-4b5c-89b0-92de57e4f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:01.277883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499180f-38b1-4abe-a72f-733bac5331a4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:01.326463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:01.365239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:01.454108 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 127/265] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:01.456179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbef1d-a7bd-4746-8536-a7544c41cde0 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:01.456613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbef1d-a7bd-4746-8536-a7544c41cde0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:01.461694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbef1d-a7bd-4746-8536-a7544c41cde0 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:01.464734 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 128/266] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:01.472570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5271-8482-4585-800f-bd3039414ec5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:01.473122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5271-8482-4585-800f-bd3039414ec5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:01.473554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5271-8482-4585-800f-bd3039414ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:01.512351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cd2f5271-8482-4585-800f-bd3039414ec5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:01.514253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cd2f5271-8482-4585-800f-bd3039414ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:01.558987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499180f-38b1-4abe-a72f-733bac5331a4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:01.653953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:01.655086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:01.655537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:01.655971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:01.679009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:01.690646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:01.694136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:01.696749 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 139/267] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:01.708617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:01.727688 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 129/268] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:01.893654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:01.954893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe7c49-03aa-4dbb-9f71-388d6bc51625 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:02.019260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:02.020246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:02.020952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:02.021419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:02.022596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:02.023173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:02.025673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:02.054517 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 140/269] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:02.060938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f5271-8482-4585-800f-bd3039414ec5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:02.247064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:02.248528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:02.294051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f5271-8482-4585-800f-bd3039414ec5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:02.359694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:02.360816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:02.361279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:02.361746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] PATH_INFO: `/v3/users/85b8193783724940bbe6d21782d38286` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:02.364012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=85b8193783724940bbe6d21782d38286)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:02.366978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:02.372741 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 130/270] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:02 2022] GET /identity/v3/users/85b8193783724940bbe6d21782d38286 => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:02.372741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:02.457790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf46573-5416-4439-9ba3-9694b084150e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:02.508241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:02.509396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:02.509817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:02.510699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] PATH_INFO: `/v3/roles/member` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:02.514531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:02.527426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:02.534669 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:02.536154 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:02.537325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:02.539510 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 141/271] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:02.540602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:02.585780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3a96a-8d3b-48c9-81ec-685767ecd8c0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:02.644973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:02.646419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:02.646928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:02.647527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:02.649247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:02.651126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:02.653337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 131/272] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:02.654610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:02.687328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:02.690414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029eedb-4b7a-489c-a307-b85bc6c9507c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:02.730355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:02.730984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:02.731257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:02.731524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:02.780094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9f8aa-b14e-4fc9-9520-2ab3af272cf8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:02.818611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:02.820863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:02.821793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:02.822484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:02.822792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:02.823696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:02.824198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:02.826158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:02.826760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:02.828047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:02.837957 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 132/273] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:02.839162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:02.961784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:02.983031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72d76-b80d-4e41-9f08-6a25bf923643 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:03.006536 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 142/274] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:03.008806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:03.032558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418c4dd-c835-4ab2-933f-a3770a685c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:03.033841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c4dd-c835-4ab2-933f-a3770a685c26 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.034542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c4dd-c835-4ab2-933f-a3770a685c26 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.034944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c4dd-c835-4ab2-933f-a3770a685c26 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.036765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418c4dd-c835-4ab2-933f-a3770a685c26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:03.039256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418c4dd-c835-4ab2-933f-a3770a685c26 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:03.076140 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 133/275] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:39:02 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:39:03.078808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3fab8-1de7-4d36-8cd1-58804d2d1de8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.079633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3fab8-1de7-4d36-8cd1-58804d2d1de8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.080382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3fab8-1de7-4d36-8cd1-58804d2d1de8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.084277 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 134/276] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:03.086391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3fab8-1de7-4d36-8cd1-58804d2d1de8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:03.195271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:03.205835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd456ed-f3df-457d-8e2e-6ad6b4ad58d8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:03.275211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175084f9-aa51-4440-a221-00eb903d9be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:03.276375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175084f9-aa51-4440-a221-00eb903d9be7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.276846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175084f9-aa51-4440-a221-00eb903d9be7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.277379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175084f9-aa51-4440-a221-00eb903d9be7 None admin] PATH_INFO: `/v3/users/c5f1976d97f147368653536a08fef2e9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.305139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175084f9-aa51-4440-a221-00eb903d9be7 None admin] RBAC: Authorizing `identity:get_user(user_id=c5f1976d97f147368653536a08fef2e9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:03.309127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175084f9-aa51-4440-a221-00eb903d9be7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:03.312697 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 143/277] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:03 2022] GET /identity/v3/users/c5f1976d97f147368653536a08fef2e9 => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:03.315106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cc34d-c895-4835-a213-b58fbac28435 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.315652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cc34d-c895-4835-a213-b58fbac28435 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.316193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cc34d-c895-4835-a213-b58fbac28435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.319545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3fab8-1de7-4d36-8cd1-58804d2d1de8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:03.352221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-262cc34d-c895-4835-a213-b58fbac28435 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:03.354767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-262cc34d-c895-4835-a213-b58fbac28435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:03.400156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:03.401478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.402885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.403471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.406046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:03.409077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:03.413820 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 135/278] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:03 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:03.415503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:03.441796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:03.496821 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 144/279] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:03.498181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cc34d-c895-4835-a213-b58fbac28435 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:03.584918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7357e3-de45-49cf-99c3-e7e1a3522cd6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:03.653561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb4fb6-7631-4a41-a58d-10c1efc60598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:03.655372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb4fb6-7631-4a41-a58d-10c1efc60598 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.656283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb4fb6-7631-4a41-a58d-10c1efc60598 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.657142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb4fb6-7631-4a41-a58d-10c1efc60598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.658798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb4fb6-7631-4a41-a58d-10c1efc60598 None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, user.id=c5f1976d97f147368653536a08fef2e9, scope.system=all) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:03.660262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:03.662356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb4fb6-7631-4a41-a58d-10c1efc60598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:03.665988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb4fb6-7631-4a41-a58d-10c1efc60598 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:03.679235 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 136/280] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 27 09:39:03 2022] GET /identity/v3/role_assignments?user.id=c5f1976d97f147368653536a08fef2e9&scope.system=all&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:03.684732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cc34d-c895-4835-a213-b58fbac28435 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:03.686289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128a8f-e440-4346-951c-ebf4622be7b4 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.686723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128a8f-e440-4346-951c-ebf4622be7b4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.687145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128a8f-e440-4346-951c-ebf4622be7b4 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.690781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 137/281] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:03.699991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8263fbb-2da6-4f5b-9865-c84aca19bb6d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.700437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8263fbb-2da6-4f5b-9865-c84aca19bb6d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.700919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8263fbb-2da6-4f5b-9865-c84aca19bb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.725594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:03.725860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:03.726188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:03.726416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:03.727355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] RBAC: Adding query filter params (role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=85b8193783724940bbe6d21782d38286) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:03.727509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:03.729785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:03.734311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b8263fbb-2da6-4f5b-9865-c84aca19bb6d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:03.737689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b8263fbb-2da6-4f5b-9865-c84aca19bb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:03.739935 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 145/282] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:03 2022] GET /identity/v3/role_assignments?user.id=85b8193783724940bbe6d21782d38286&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=6f0bec1f01514029a1162fe631d80648 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:03.881615 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 138/283] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:04.195042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:04.394350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca5a3-8aad-4e03-a6ed-d6abd9866a21 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:04.468301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:04.469616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:04.470109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:04.470587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:04.493528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:04.504166 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:04.505864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:04.507394 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 146/284] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:04.517204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8263fbb-2da6-4f5b-9865-c84aca19bb6d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:04.524346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:04.702842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:04.740856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:04.805619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8263fbb-2da6-4f5b-9865-c84aca19bb6d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:04.930066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:04.931126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:04.931600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:04.932034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:04.933336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:04.933865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:04.936532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:04.958430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 139/285] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:04.966244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:04.986991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1f6c3-c8b5-4997-9429-05298d40e089 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:05.108694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:05.109795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:05.116599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:05.118988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:05.166594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:05.182020 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:05.183002 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:05.183788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:05.184416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 147/286] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:05.184966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:05.193640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:05.245904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce18d0-6691-4ea7-8248-d100b74bf0fa None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:05.303901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:05.314682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:05.315350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:05.315630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:05.315898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] PATH_INFO: `/v3/users/cbdb35c2ef0e4cc8854585b044ea7368` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:05.317752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] RBAC: Authorizing `identity:get_user(user_id=cbdb35c2ef0e4cc8854585b044ea7368)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:05.319261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:05.322507 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 140/287] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:04 2022] GET /identity/v3/users/cbdb35c2ef0e4cc8854585b044ea7368 => generated 297 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:05.331606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:05.383160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddad83-5f88-4c34-88d1-7be53be53d4e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:05.455545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:05.457214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:05.457982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:05.461613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:05.462672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:05.463170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:05.464839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:05.488198 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 148/288] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:05.493217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:05.550469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dc6dd-73c4-40e9-8e96-8e814d56ae5d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:05.627642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:05.628313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:05.628588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:05.628947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:05.635369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:05.636847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:05.639683 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 141/289] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:05 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:05.649760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:05.659178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:05.685796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4b456-dad4-434d-af7b-c041fedc8fc6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:05.729082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:05.734928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:05.735980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:05.736523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:05.737008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] PATH_INFO: `/v3/users/e1e1c2b1360149bd8d8016577c9d044c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:05.738829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] RBAC: Authorizing `identity:get_user(user_id=e1e1c2b1360149bd8d8016577c9d044c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:05.741098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:05.744019 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 149/290] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:05 2022] GET /identity/v3/users/e1e1c2b1360149bd8d8016577c9d044c => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:05.753949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:05.930909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16a53d-5558-4795-b8ce-4785a82b7fb9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:05.958748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7830b6-6750-4b32-8e9a-bee073b717f6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:06.012950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:06.013620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:06.013900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:06.014179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9/users/cbdb35c2ef0e4cc8854585b044ea7368/roles/338edc307ed345d6b18db9185e15aa19` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:06.023227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:06.023844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:06.024121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:06.024392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:06.028375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5015560e7c748678c104319614c21f9, user_id=cbdb35c2ef0e4cc8854585b044ea7368, role_id=338edc307ed345d6b18db9185e15aa19)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:06.029890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:06.058423 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 142/291] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:39:05 2022] PUT /identity/v3/projects/c5015560e7c748678c104319614c21f9/users/cbdb35c2ef0e4cc8854585b044ea7368/roles/338edc307ed345d6b18db9185e15aa19 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:39:06.187111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:06.193013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:06.195104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:06.338142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 150/292] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:06.345965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:06.432440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:06.483813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89881939-0cd7-41f1-be38-9a712facec23 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:06.539616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdff0e-fd79-4068-9d40-2a8600e1927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:06.540276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdff0e-fd79-4068-9d40-2a8600e1927d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:06.540551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdff0e-fd79-4068-9d40-2a8600e1927d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:06.540848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdff0e-fd79-4068-9d40-2a8600e1927d None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:06.542029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdff0e-fd79-4068-9d40-2a8600e1927d None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:06.543496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdff0e-fd79-4068-9d40-2a8600e1927d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:06.545240 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 143/293] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:06 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:06.551428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:06.698467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:06.709646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6235ae-b314-4669-b58f-99d9f69adab9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:06.798843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38c7a2-e6db-469e-ba2b-858a8ac5c82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:06.800309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c7a2-e6db-469e-ba2b-858a8ac5c82f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:06.801050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c7a2-e6db-469e-ba2b-858a8ac5c82f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:06.801849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c7a2-e6db-469e-ba2b-858a8ac5c82f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:06.803560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38c7a2-e6db-469e-ba2b-858a8ac5c82f None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=e1e1c2b1360149bd8d8016577c9d044c) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:06.804432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38c7a2-e6db-469e-ba2b-858a8ac5c82f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:06.807582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38c7a2-e6db-469e-ba2b-858a8ac5c82f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:06.820159 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 151/294] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:06 2022] GET /identity/v3/role_assignments?user.id=e1e1c2b1360149bd8d8016577c9d044c&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:07.121604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5e67-8eac-43f8-b92f-100b5f26b454 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.122253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5e67-8eac-43f8-b92f-100b5f26b454 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.122253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea5e67-8eac-43f8-b92f-100b5f26b454 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.123402 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 144/295] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:07.142738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4fa05-532c-4f46-90d1-8733c4e87c3e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.143051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4fa05-532c-4f46-90d1-8733c4e87c3e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.143273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4fa05-532c-4f46-90d1-8733c4e87c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.167644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6ae4fa05-532c-4f46-90d1-8733c4e87c3e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:07.168961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6ae4fa05-532c-4f46-90d1-8733c4e87c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:07.267512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 152/296] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:07.275204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abde9d-1579-49e1-931b-6494b1e4681a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.275554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abde9d-1579-49e1-931b-6494b1e4681a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.275745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abde9d-1579-49e1-931b-6494b1e4681a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.277200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 145/297] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:07.283357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15566bff-476b-46a6-95ea-cd5c9d77185d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.283679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15566bff-476b-46a6-95ea-cd5c9d77185d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.283898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15566bff-476b-46a6-95ea-cd5c9d77185d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.314471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-15566bff-476b-46a6-95ea-cd5c9d77185d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:07.316400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15566bff-476b-46a6-95ea-cd5c9d77185d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:07.391607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25b4cd-6a1f-4331-8bda-5287d1886b03 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.391803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25b4cd-6a1f-4331-8bda-5287d1886b03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.392072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25b4cd-6a1f-4331-8bda-5287d1886b03 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.393748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 146/298] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:07.406455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04556132-71b4-4178-b3b4-11f9f2219073 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.406734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04556132-71b4-4178-b3b4-11f9f2219073 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.407241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04556132-71b4-4178-b3b4-11f9f2219073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.444251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-04556132-71b4-4178-b3b4-11f9f2219073 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:07.447606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04556132-71b4-4178-b3b4-11f9f2219073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:07.447836 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 153/299] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:07.606519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 147/300] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:07.629328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15566bff-476b-46a6-95ea-cd5c9d77185d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:07.663962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:07.762335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15566bff-476b-46a6-95ea-cd5c9d77185d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:07.806784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:07.806998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04556132-71b4-4178-b3b4-11f9f2219073 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:07.807825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.808250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.808685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.810101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:07.811639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:07.813755 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 154/301] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:07.819797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:07.885809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:07.939150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:07.952615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04556132-71b4-4178-b3b4-11f9f2219073 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:07.953982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fc712-19da-487d-88e1-1ad28fe3ef8f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:07.996522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:07.996763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:07.997193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.997316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:07.997468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.997574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:07.997723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.997811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:07.998366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:07.999829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:08.010390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:08.015821 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:08.016986 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:08.017838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 148/302] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:08.018491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:08.022076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 155/303] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Sep 27 09:39:07 2022] POST /identity/v3/groups => generated 229 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:39:08.023080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:08.172341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43653-2f31-4afc-aa25-c9738b0f7952 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:08.202969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:08.207378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58d87-ff8e-4e11-b30a-e14773f9d420 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:08.220478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:08.221202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:08.221485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:08.221756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:08.235134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:08.240559 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:08.242218 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:08.243420 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 149/304] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:08.253820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:08.290070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:08.291435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:08.292049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:08.292705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:08.294226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:08.295053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:08.297868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:08.314146 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 156/305] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:08.321687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:08.347386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:08.439589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf354-4455-465e-bd1d-b85693e1e0c8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:08.492880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d47864-3353-44e5-8248-e7ebadd3cbb0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:08.498661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:08.499764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:08.500223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:08.500690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:08.501846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:08.502376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:08.504810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:08.518935 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 150/306] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:08.525834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:08.542214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:08.543139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:08.543774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:08.543968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] PATH_INFO: `/v3/users/3f90bdd1d5624c8b80b4e4c72d086a33` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:08.561009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] RBAC: Authorizing `identity:get_user(user_id=3f90bdd1d5624c8b80b4e4c72d086a33)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:08.562720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:08.570156 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 157/307] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:08 2022] GET /identity/v3/users/3f90bdd1d5624c8b80b4e4c72d086a33 => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:08.571773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:08.591724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:08.661007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918667d-ebde-4a3e-b21f-0e382e614637 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:08.694943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:08.711006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:08.711776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:08.712055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:08.712326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] PATH_INFO: `/v3/users/c5f1976d97f147368653536a08fef2e9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:08.713719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] RBAC: Authorizing `identity:get_user(user_id=c5f1976d97f147368653536a08fef2e9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:08.715212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:08.721874 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 151/308] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:08 2022] GET /identity/v3/users/c5f1976d97f147368653536a08fef2e9 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:08.723282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:08.731120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1f661-7ad2-43dc-b596-42656cf0ccf1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:08.781281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb436f2-8be5-4a30-ae26-800fa92d4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:08.782777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb436f2-8be5-4a30-ae26-800fa92d4820 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:08.783261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb436f2-8be5-4a30-ae26-800fa92d4820 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:08.783720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb436f2-8be5-4a30-ae26-800fa92d4820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:08.785011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb436f2-8be5-4a30-ae26-800fa92d4820 None admin] RBAC: Adding query filter params (role.id=338edc307ed345d6b18db9185e15aa19, user.id=3f90bdd1d5624c8b80b4e4c72d086a33, scope.system=all) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:08.786213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb436f2-8be5-4a30-ae26-800fa92d4820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:08.788682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb436f2-8be5-4a30-ae26-800fa92d4820 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:08.803119 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 158/309] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 27 09:39:08 2022] GET /identity/v3/role_assignments?user.id=3f90bdd1d5624c8b80b4e4c72d086a33&scope.system=all&role.id=338edc307ed345d6b18db9185e15aa19 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:08.849071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94350745-fe23-4975-b06a-df0bca91a173 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:08.863093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:08.894813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a39a7-111a-42d8-8cbb-4eea2448b8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:08.895723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a39a7-111a-42d8-8cbb-4eea2448b8f8 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:08.896003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a39a7-111a-42d8-8cbb-4eea2448b8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:08.896276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a39a7-111a-42d8-8cbb-4eea2448b8f8 None admin] PATH_INFO: `/v3/system/users/c5f1976d97f147368653536a08fef2e9/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:08.898012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a39a7-111a-42d8-8cbb-4eea2448b8f8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c5f1976d97f147368653536a08fef2e9, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:08.899458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a39a7-111a-42d8-8cbb-4eea2448b8f8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:08.914229 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 152/310] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Sep 27 09:39:08 2022] PUT /identity/v3/system/users/c5f1976d97f147368653536a08fef2e9/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:39:09.330669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b01b5-4d51-4a7e-8aaa-527503e89394 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:09.331018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b01b5-4d51-4a7e-8aaa-527503e89394 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:09.331219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b01b5-4d51-4a7e-8aaa-527503e89394 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:09.333658 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 159/311] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:09.339487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ddb0-791c-4753-a700-3c745dc6ab44 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:09.340002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ddb0-791c-4753-a700-3c745dc6ab44 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:09.340105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ddb0-791c-4753-a700-3c745dc6ab44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:09.359824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e3f0ddb0-791c-4753-a700-3c745dc6ab44 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:09.361112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e3f0ddb0-791c-4753-a700-3c745dc6ab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:09.439292 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 153/312] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:09.788997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b01b5-4d51-4a7e-8aaa-527503e89394 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:09.902815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b01b5-4d51-4a7e-8aaa-527503e89394 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:09.940556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:09.941676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:09.942384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:09.942972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:09.945236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:09.956804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:09.962857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:09.963771 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:09.964578 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:09.966956 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 160/313] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:09.973033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0ddb0-791c-4753-a700-3c745dc6ab44 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:10.091674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0ddb0-791c-4753-a700-3c745dc6ab44 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:10.094706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:10.144436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:10.145638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:10.146200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:10.146668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:10.147876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] RBAC: Adding query filter params (name=reader) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:10.148395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:10.150845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:10.169057 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 154/314] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:10.174363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:10.286746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5be06f-8384-4f34-bb2e-ff7c82de963f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:10.324338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:10.325412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:10.325889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:10.326353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] PATH_INFO: `/v3/users/cbdb35c2ef0e4cc8854585b044ea7368` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:10.327719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] RBAC: Authorizing `identity:get_user(user_id=cbdb35c2ef0e4cc8854585b044ea7368)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:10.329472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:10.331799 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 161/315] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:10 2022] GET /identity/v3/users/cbdb35c2ef0e4cc8854585b044ea7368 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:10.337782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:10.500948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39013f06-1ee7-4be8-9926-9f4f90ff432f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:10.560440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:10.561566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:10.562054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:10.562596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:10.625179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:10.696572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:10.698300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:10.885322 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 155/316] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:10.893838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:10.980791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:11.070643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea76bc-acd7-4d89-a2f4-3b77939dc769 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:11.112485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cbc30-598b-4272-8efd-48bb2f27b17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:11.113196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cbc30-598b-4272-8efd-48bb2f27b17e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.113853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cbc30-598b-4272-8efd-48bb2f27b17e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.113853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cbc30-598b-4272-8efd-48bb2f27b17e None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.115441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cbc30-598b-4272-8efd-48bb2f27b17e None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:11.116986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cbc30-598b-4272-8efd-48bb2f27b17e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:11.119158 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 162/317] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:10 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:11.128833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:11.256779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:11.311705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0b584-2a80-4363-9272-4fab5288a7a0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:11.367687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01afcd-7010-46bd-a885-5fb90cffd3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:11.368695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01afcd-7010-46bd-a885-5fb90cffd3bf None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.375601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01afcd-7010-46bd-a885-5fb90cffd3bf None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.376054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01afcd-7010-46bd-a885-5fb90cffd3bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.377482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01afcd-7010-46bd-a885-5fb90cffd3bf None admin] RBAC: Adding query filter params (role.id=338edc307ed345d6b18db9185e15aa19, scope.project.id=c5015560e7c748678c104319614c21f9, user.id=cbdb35c2ef0e4cc8854585b044ea7368) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:11.378012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01afcd-7010-46bd-a885-5fb90cffd3bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:11.381009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01afcd-7010-46bd-a885-5fb90cffd3bf None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:11.392698 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 156/318] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:11 2022] GET /identity/v3/role_assignments?user.id=cbdb35c2ef0e4cc8854585b044ea7368&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=338edc307ed345d6b18db9185e15aa19 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:11.502516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5463a6e-a326-489b-83aa-febc28899314 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.502863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5463a6e-a326-489b-83aa-febc28899314 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.503053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5463a6e-a326-489b-83aa-febc28899314 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.504731 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 163/319] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:11.511846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af617e-08c2-455b-aac1-05796f3c9565 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.512356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af617e-08c2-455b-aac1-05796f3c9565 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.512730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af617e-08c2-455b-aac1-05796f3c9565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.546754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d3af617e-08c2-455b-aac1-05796f3c9565 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:11.548945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3af617e-08c2-455b-aac1-05796f3c9565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:11.583546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2eccd-ff6a-453e-bc6a-0c50f0e6670b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.584139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2eccd-ff6a-453e-bc6a-0c50f0e6670b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.584615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2eccd-ff6a-453e-bc6a-0c50f0e6670b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.587846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 164/320] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:11.597289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e582-ca8b-44ed-84cc-4bb135a40e3a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.597890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e582-ca8b-44ed-84cc-4bb135a40e3a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.598367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e582-ca8b-44ed-84cc-4bb135a40e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.633688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3c65e582-ca8b-44ed-84cc-4bb135a40e3a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:11.636376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3c65e582-ca8b-44ed-84cc-4bb135a40e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:11.655290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 157/321] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:11.773283 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 165/322] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:11.824869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc358ffb-534f-499a-9d6a-18020de61c94 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.825127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc358ffb-534f-499a-9d6a-18020de61c94 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.825480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc358ffb-534f-499a-9d6a-18020de61c94 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.826774 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 158/323] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:39:11.833795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988b1f-7c07-49f0-9de3-cfd7b091d489 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:11.834040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988b1f-7c07-49f0-9de3-cfd7b091d489 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:11.834370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988b1f-7c07-49f0-9de3-cfd7b091d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:11.862254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ed988b1f-7c07-49f0-9de3-cfd7b091d489 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:11.863520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed988b1f-7c07-49f0-9de3-cfd7b091d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:11.930377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:11.945217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 166/324] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:12.122338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc358ffb-534f-499a-9d6a-18020de61c94 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.254893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:12.271259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed988b1f-7c07-49f0-9de3-cfd7b091d489 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.321013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc358ffb-534f-499a-9d6a-18020de61c94 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:12.359720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:12.360476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:12.360889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:12.361303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:12.373923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:12.378776 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:12.380536 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 159/325] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:12.382104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.430287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed988b1f-7c07-49f0-9de3-cfd7b091d489 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:12.490770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:12.491804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:12.492260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:12.492778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:12.494648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:12.497175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:12.500544 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 167/326] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:12.500929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e46f28-fc0a-456f-9afc-03841dd9936b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:12.501668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.527392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:12.548471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:12.549554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:12.549992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:12.550455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:12.568483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:12.577374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:12.577859 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:12.579228 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:12.580398 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 160/327] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:12.583201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.645152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91fc5d-0977-4113-aad1-b0d88b49e5a1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:12.695130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:12.696088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:12.696522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:12.697069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:12.698189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:12.698679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:12.701168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:12.715632 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 168/328] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:12.717496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.746585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf7312-7484-4847-9459-b38ac98a5db2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:12.790569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:12.791222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:12.791511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:12.791781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] PATH_INFO: `/v3/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:12.792439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:12.793986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:12.814806 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 161/329] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Sep 27 09:39:12 2022] POST /identity/v3/groups => generated 226 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:39:12.816086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.863653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec822199-98f7-4739-a143-770d2808a486 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:12.877504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:12.899761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:12.926146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:12.926968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:12.927356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:12.927768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:12.928845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:12.929447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:12.931616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:12.944491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb8235-676d-4d55-b01b-c8283b413ba1 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:12.946112 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 169/330] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:12.952248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:12.996019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:12.996735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:12.997017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:12.997290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] PATH_INFO: `/v3/users/c5f1976d97f147368653536a08fef2e9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:12.998595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=c5f1976d97f147368653536a08fef2e9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:13.000238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:13.002247 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 162/331] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:12 2022] GET /identity/v3/users/c5f1976d97f147368653536a08fef2e9 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:13.009252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:13.086133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b48dc9-88b5-4be2-b3ed-852ab7dba3ba None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:13.126152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:13.126805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:13.127105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:13.127341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] PATH_INFO: `/v3/users/3f90bdd1d5624c8b80b4e4c72d086a33` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:13.128643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] RBAC: Authorizing `identity:get_user(user_id=3f90bdd1d5624c8b80b4e4c72d086a33)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:13.130730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:13.132839 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 170/332] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:12 2022] GET /identity/v3/users/3f90bdd1d5624c8b80b4e4c72d086a33 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:13.138682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:13.175306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6495b7-6e72-436e-a85f-f536fd538bc7 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:13.190452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:13.229438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085008b-99fe-46db-81c1-9369a15d2a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:13.230135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085008b-99fe-46db-81c1-9369a15d2a33 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:13.230345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085008b-99fe-46db-81c1-9369a15d2a33 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:13.230697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085008b-99fe-46db-81c1-9369a15d2a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:13.231357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085008b-99fe-46db-81c1-9369a15d2a33 None admin] RBAC: Adding query filter params (role.id=a0597e11955a4725ae4938bb442c75f6, user.id=c5f1976d97f147368653536a08fef2e9, scope.system=all) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:13.232317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085008b-99fe-46db-81c1-9369a15d2a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:13.233367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085008b-99fe-46db-81c1-9369a15d2a33 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:13.233567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:13.249756 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 163/333] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 27 09:39:13 2022] GET /identity/v3/role_assignments?user.id=c5f1976d97f147368653536a08fef2e9&scope.system=all&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 510 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:13.309916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be673-eab7-4511-b0a6-d1645a654a44 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:13.358024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047117d-fddd-4672-aa0f-a171f4344b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:13.358729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047117d-fddd-4672-aa0f-a171f4344b54 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:13.359013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047117d-fddd-4672-aa0f-a171f4344b54 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:13.359281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047117d-fddd-4672-aa0f-a171f4344b54 None admin] PATH_INFO: `/v3/system/users/3f90bdd1d5624c8b80b4e4c72d086a33/roles/338edc307ed345d6b18db9185e15aa19` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:13.361112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047117d-fddd-4672-aa0f-a171f4344b54 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3f90bdd1d5624c8b80b4e4c72d086a33, role_id=338edc307ed345d6b18db9185e15aa19)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:13.362780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047117d-fddd-4672-aa0f-a171f4344b54 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:13.376190 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 171/334] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Sep 27 09:39:13 2022] PUT /identity/v3/system/users/3f90bdd1d5624c8b80b4e4c72d086a33/roles/338edc307ed345d6b18db9185e15aa19 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:39:16.527133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a2fb9-87ed-47f4-92db-e1b25660005a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:16.527673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a2fb9-87ed-47f4-92db-e1b25660005a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:16.527673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a2fb9-87ed-47f4-92db-e1b25660005a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:16.529327 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 164/335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:16.535772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d383d1-4720-4925-8641-19b321b48333 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:16.536815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d383d1-4720-4925-8641-19b321b48333 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:16.537331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d383d1-4720-4925-8641-19b321b48333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:16.565880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-53d383d1-4720-4925-8641-19b321b48333 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:16.567799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-53d383d1-4720-4925-8641-19b321b48333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:16.676186 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 172/336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:16.879150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15960f72-2c6c-4a58-984d-3ecd32e9633a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:16.879976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15960f72-2c6c-4a58-984d-3ecd32e9633a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:16.880493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15960f72-2c6c-4a58-984d-3ecd32e9633a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:16.882524 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 165/337] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:16.888367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2217af-a222-4ba1-acbc-c8ca9ebdab1e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:16.889047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2217af-a222-4ba1-acbc-c8ca9ebdab1e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:16.889541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2217af-a222-4ba1-acbc-c8ca9ebdab1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:16.913882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf2217af-a222-4ba1-acbc-c8ca9ebdab1e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:16.915471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf2217af-a222-4ba1-acbc-c8ca9ebdab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:16.975160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:17.015131 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 173/338] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:17.156327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef532-42d4-41bd-84b3-273f0760aa43 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:17.156975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef532-42d4-41bd-84b3-273f0760aa43 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:17.157484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef532-42d4-41bd-84b3-273f0760aa43 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:17.159335 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 166/339] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:17.165538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abed0-678e-4058-9c96-ffcf23933ff0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:17.166137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abed0-678e-4058-9c96-ffcf23933ff0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:17.166602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abed0-678e-4058-9c96-ffcf23933ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:17.193842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9d3abed0-678e-4058-9c96-ffcf23933ff0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:17.195433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9d3abed0-678e-4058-9c96-ffcf23933ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:17.277404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef532-42d4-41bd-84b3-273f0760aa43 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:17.285031 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 174/340] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:17.444928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef532-42d4-41bd-84b3-273f0760aa43 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:17.494613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:17.495655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:17.496129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:17.496589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:17.510808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:17.516261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:17.517261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 09:39:17.518098 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:17.520832 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 167/341] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:17.527223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3abed0-678e-4058-9c96-ffcf23933ff0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:17.538486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:17.606825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:17.767312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3abed0-678e-4058-9c96-ffcf23933ff0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:17.778577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:17.789054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2310f-07cd-4cbe-a2e0-a0ea7bf793ba None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:17.843560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33998fa-17ec-418d-a63b-4932e4fe4458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:17.844271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:17.844589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33998fa-17ec-418d-a63b-4932e4fe4458 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:17.844986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:17.845108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33998fa-17ec-418d-a63b-4932e4fe4458 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:17.845282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:17.845526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:17.845623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33998fa-17ec-418d-a63b-4932e4fe4458 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:17.847099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33998fa-17ec-418d-a63b-4932e4fe4458 None admin] RBAC: Adding query filter params (name=reader) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:17.848217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33998fa-17ec-418d-a63b-4932e4fe4458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:17.851481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33998fa-17ec-418d-a63b-4932e4fe4458 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:17.859714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:17.866430 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:17.867396 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:17.868189 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:17.872684 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 168/342] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:17.872684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:17.876224 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 175/343] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:17.878576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7ede5-193c-4705-abdc-1537cde9ee48 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:17.879150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7ede5-193c-4705-abdc-1537cde9ee48 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:17.879637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7ede5-193c-4705-abdc-1537cde9ee48 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:17.884226 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 176/344] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:17.889343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7ede5-193c-4705-abdc-1537cde9ee48 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:18.026408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:18.028128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7ede5-193c-4705-abdc-1537cde9ee48 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:18.049194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84cac-886a-449f-a2a1-4a0feac812b0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:18.070497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:18.071257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:18.071535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:18.071804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:18.072496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:18.072851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:18.074269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:18.085011 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 177/345] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:18.090339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:18.102511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60220f4-786e-4491-b8ec-c3f7ce059344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:18.104015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60220f4-786e-4491-b8ec-c3f7ce059344 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:18.104889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60220f4-786e-4491-b8ec-c3f7ce059344 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:18.105654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60220f4-786e-4491-b8ec-c3f7ce059344 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:18.125232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60220f4-786e-4491-b8ec-c3f7ce059344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:18.127358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-e60220f4-786e-4491-b8ec-c3f7ce059344 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:18.133577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:18.135277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:18.136144 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:18.139125 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 169/346] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:18.140831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94566d-db8a-4e06-8cd1-c278099bd188 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:18.141340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94566d-db8a-4e06-8cd1-c278099bd188 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:18.141889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94566d-db8a-4e06-8cd1-c278099bd188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:18.169013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5c94566d-db8a-4e06-8cd1-c278099bd188 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:18.170581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5c94566d-db8a-4e06-8cd1-c278099bd188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:18.248002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedffb4-026b-4443-b79c-c4f31a293a38 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:18.259274 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 170/347] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:18.260503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94566d-db8a-4e06-8cd1-c278099bd188 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:18.356163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:18.364994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:18.366122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:18.366605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:18.367088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] PATH_INFO: `/v3/users/3f90bdd1d5624c8b80b4e4c72d086a33` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:18.370603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] RBAC: Authorizing `identity:get_user(user_id=3f90bdd1d5624c8b80b4e4c72d086a33)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:18.373392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:18.376496 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 178/348] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:39:18 2022] GET /identity/v3/users/3f90bdd1d5624c8b80b4e4c72d086a33 => generated 293 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:18.385935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:18.438094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94566d-db8a-4e06-8cd1-c278099bd188 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:18.538495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:18.539611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:18.539982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:18.540443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:18.703474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:18.726852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d700-f05b-4568-b388-c97956dc58cc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:18.816287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:18.819583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:18.860971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:18.862698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:18.863292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:18.864512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:18.865291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:18.865608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:18.867036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:18.891260 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 179/349] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:18.896121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:18.990692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:19.020273 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 171/350] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:19.021234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:19.085983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467da4eb-86d1-4b2c-a9d5-4fa5d06df25d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:19.143202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9e75-f9ca-4ecc-91be-5304669088d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:19.143750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9e75-f9ca-4ecc-91be-5304669088d1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.144025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9e75-f9ca-4ecc-91be-5304669088d1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.144296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9e75-f9ca-4ecc-91be-5304669088d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.145319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a9e75-f9ca-4ecc-91be-5304669088d1 None admin] RBAC: Adding query filter params (role.id=338edc307ed345d6b18db9185e15aa19, user.id=3f90bdd1d5624c8b80b4e4c72d086a33, scope.system=all) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:19.145632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a9e75-f9ca-4ecc-91be-5304669088d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:19.147231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a9e75-f9ca-4ecc-91be-5304669088d1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:19.158327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2315fe-2846-4286-b194-5e7eb22a3105 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:19.169141 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 180/351] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 27 09:39:18 2022] GET /identity/v3/role_assignments?user.id=3f90bdd1d5624c8b80b4e4c72d086a33&scope.system=all&role.id=338edc307ed345d6b18db9185e15aa19 => generated 510 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:19.171311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc45aa-b180-4ed4-93d1-b5aa66cb2acd None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.171742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc45aa-b180-4ed4-93d1-b5aa66cb2acd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.172251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc45aa-b180-4ed4-93d1-b5aa66cb2acd None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.174794 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 181/352] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:19.176304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc45aa-b180-4ed4-93d1-b5aa66cb2acd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:19.213050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2588d5-5cba-4209-9b69-16ec587e1701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:19.213727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2588d5-5cba-4209-9b69-16ec587e1701 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.214156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2588d5-5cba-4209-9b69-16ec587e1701 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.214426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2588d5-5cba-4209-9b69-16ec587e1701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.230320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:19.292690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:19.342704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2588d5-5cba-4209-9b69-16ec587e1701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:19.348237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2588d5-5cba-4209-9b69-16ec587e1701 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:19.366133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc45aa-b180-4ed4-93d1-b5aa66cb2acd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:19.417720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:19.418804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.419297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.419779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.421953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:19.423577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:19.425529 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 182/353] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:19 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:19.426913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:19.549836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 172/354] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:19.551294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17590bd-a6c8-42c2-8612-3400d9a9403e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.551574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17590bd-a6c8-42c2-8612-3400d9a9403e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.551852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17590bd-a6c8-42c2-8612-3400d9a9403e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.562729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:19.576837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a17590bd-a6c8-42c2-8612-3400d9a9403e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:19.579104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a17590bd-a6c8-42c2-8612-3400d9a9403e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:19.582296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f73f91-a750-42ee-b202-8042ea5ed109 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:19.633506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffa666-7e41-49f8-8e89-99a4c3737a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:19.634165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa666-7e41-49f8-8e89-99a4c3737a78 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.634469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa666-7e41-49f8-8e89-99a4c3737a78 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.634743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa666-7e41-49f8-8e89-99a4c3737a78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.647227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffa666-7e41-49f8-8e89-99a4c3737a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e3ffa666-7e41-49f8-8e89-99a4c3737a78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:19.652711 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:19.653633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:19.654427 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 183/355] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:19.655247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fd52-c866-436a-9487-033707685d80 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.655671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fd52-c866-436a-9487-033707685d80 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.656107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fd52-c866-436a-9487-033707685d80 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.657733 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 184/356] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:19.663724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8fd52-c866-436a-9487-033707685d80 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:19.698626 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 173/357] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:19.700757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17590bd-a6c8-42c2-8612-3400d9a9403e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:19.716919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:19.818816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8fd52-c866-436a-9487-033707685d80 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:19.853644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17590bd-a6c8-42c2-8612-3400d9a9403e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:19.859504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:19.860191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.860447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.865383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.873381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:19.875198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:19.877147 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 185/358] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:19 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:19.878333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:19.892445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:19.924098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d962ff-c76c-45f4-af61-068e682e26bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:19.925240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d962ff-c76c-45f4-af61-068e682e26bd None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.925664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d962ff-c76c-45f4-af61-068e682e26bd None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.926240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d962ff-c76c-45f4-af61-068e682e26bd None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.928331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d962ff-c76c-45f4-af61-068e682e26bd None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:19.930896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d962ff-c76c-45f4-af61-068e682e26bd None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:19.934094 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 174/359] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:19 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:19.935988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157f73-db90-4b8b-8142-e374298b63d3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:19.936416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157f73-db90-4b8b-8142-e374298b63d3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:19.936868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157f73-db90-4b8b-8142-e374298b63d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:19.961398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-10157f73-db90-4b8b-8142-e374298b63d3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:19.963482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-10157f73-db90-4b8b-8142-e374298b63d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:20.055222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e7ef-84be-44ea-9b75-67a6589b9d83 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:20.066222 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 175/360] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:20.066926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157f73-db90-4b8b-8142-e374298b63d3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:20.075226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:20.106320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:20.107009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:20.107296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:20.107631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:20.108379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:20.108715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:20.110251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:20.127047 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 186/361] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:20.128452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:20.230816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:20.282373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157f73-db90-4b8b-8142-e374298b63d3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:20.288596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ea6c3-edf2-4283-9640-09901e99cbd6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:20.342077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:20.343064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:20.343531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:20.343987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:20.344312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:20.345226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:20.345552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:20.345716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:20.346243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:20.346706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:20.348101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:20.348639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:20.350299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:20.352142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 187/362] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:20 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:20.352910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:20.358164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 176/363] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:20 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=6f0bec1f01514029a1162fe631d80648 => generated 260 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:20.359451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:20.512455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:20.517812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96a80-f03f-419f-bbe5-2a9f2876236d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:20.583417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:20.584165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:20.584441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:20.585430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b796f-9f85-454a-be4b-441d656cbca3 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:20.587153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:20.598127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:20.611067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:20.617570 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:20.618527 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:20.619321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 188/364] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:20.619984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:20.668557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:20.669595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:20.669879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:20.670163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:20.787625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95176793-3e35-498c-b7e3-30f6c67b9abd None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:20.805044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:20.808033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:20.861599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:20.862685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:20.863109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:20.863585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:20.864479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=c5015560e7c748678c104319614c21f9) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:20.865035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:20.866879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:20.876074 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 189/365] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:20 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:20.877225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:20.946475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:20.996232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:21.025563 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 177/366] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:21.025979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:21.111203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b9bf8-ec8c-47c8-a27b-2cc2f80d841d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:21.187251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:21.188310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:21.188846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:21.189324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:21.212288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:21.222054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:21.223041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:39:21.223837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:21.224470 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 190/367] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:21.225930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:21.246638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fca56-cc15-419f-a8d9-703e831cdc50 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:21.320079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:21.321557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:21.322332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:21.323148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:21.324604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:21.325426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:21.328308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:21.345804 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 178/368] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:21.348029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:21.424564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:21.437714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4592939-c50f-4404-ac0f-25bc017f5a3b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:21.479577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:21.504248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:21.506255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:21.506255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:21.506850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:21.509104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:21.512212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:21.516181 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 191/369] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:21 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:21.517695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:21.586862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816318b-4d3a-40e6-a496-ffe2efddf9e6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:21.657203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:21.658255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:21.658738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:21.659212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:21.660933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:21.661547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:21.664364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:21.680447 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 179/370] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:21.681869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:21.707948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f61a2-ff04-42a4-bd1a-73dc83e5a3ba None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:21.751579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:21.752221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:21.752740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:21.753081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:21.832745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:21.846708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:21.849233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:21.920999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:21.938405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694675a-e07c-48d3-a1a3-8cc82500a644 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:21.996514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 192/371] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:21.997371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:22.017889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:22.018787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:22.019174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:22.019617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:22.021168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:22.022848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:22.028713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 180/372] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:21 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:22.028713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:22.147353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:22.161989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54554b17-8310-4a9a-874f-6fd308b38fa8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:22.168865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af32d5-b023-457d-8bac-cf086c4affb8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:22.211708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:22.212381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:22.212792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:22.213060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:22.214224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:22.214416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:22.215149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:22.215570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:22.215956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:22.216105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:22.219254 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 181/373] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:22 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:22.221510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:22.259213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:22.352363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:22.355221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:22.383144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b1580-f600-46ef-aa0b-5869ed1d2c23 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:22.454926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:22.455920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:22.456500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:22.458840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:22.460323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:22.461007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:22.464210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:22.475519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 182/374] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:22 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 260 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:22.477513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:22.555034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:22.588277 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 193/375] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:22.589479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:22.771697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:22.793598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50247d2-c4b5-4710-8236-fb6dae1699e6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:22.856678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e2fbe-c8d3-4bde-8e87-40fc5babeefc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:22.949313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:22.950401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:22.950867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:22.951319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:22.954483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:22.955547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:22.956016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:22.956501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] PATH_INFO: `/v3/projects/7d515506957d41b7867232158c5ad868` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:22.958836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:22.962527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] RBAC: Authorizing `identity:get_project(project_id=7d515506957d41b7867232158c5ad868)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:22.965233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:22.969013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:22.972153 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 194/376] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:22 2022] GET /identity/v3/projects/7d515506957d41b7867232158c5ad868 => generated 344 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:22.973483 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 183/377] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:22 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 497 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:22.978021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:22.984550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:23.133825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:23.216315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920d6a6-c12b-4cf3-91e1-31b0a283a04c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:23.232797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f8aca-3573-4900-ba02-b015dc81a6aa None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:23.322733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:23.326271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:23.328851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:23.330371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:23.333488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=c5015560e7c748678c104319614c21f9) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:23.337430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:23.342351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:23.352713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c986f61-9780-4275-a0d6-c87f77e0f015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:23.356432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c986f61-9780-4275-a0d6-c87f77e0f015 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:23.357255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c986f61-9780-4275-a0d6-c87f77e0f015 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:23.359362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c986f61-9780-4275-a0d6-c87f77e0f015 None admin] PATH_INFO: `/v3/groups/55e7cc106a2944a79c91b352d3f21e61` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:23.365102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 184/378] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:22 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=6f0bec1f01514029a1162fe631d80648 => generated 260 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:23.374882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c986f61-9780-4275-a0d6-c87f77e0f015 None admin] RBAC: Authorizing `identity:get_group(group_id=55e7cc106a2944a79c91b352d3f21e61)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:23.378316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c986f61-9780-4275-a0d6-c87f77e0f015 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:23.383380 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 195/379] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:22 2022] GET /identity/v3/groups/55e7cc106a2944a79c91b352d3f21e61 => generated 229 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:23.393635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:23.451884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:23.601909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaf820-fa16-4560-a4c2-b8f65f1210d7 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:23.666901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b43771-0458-4ddf-81ff-53127f581393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:23.667460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43771-0458-4ddf-81ff-53127f581393 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:23.667743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43771-0458-4ddf-81ff-53127f581393 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:23.668016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43771-0458-4ddf-81ff-53127f581393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:23.668806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b43771-0458-4ddf-81ff-53127f581393 None admin] RBAC: Adding query filter params (group.id=55e7cc106a2944a79c91b352d3f21e61, role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=7d515506957d41b7867232158c5ad868) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:23.669118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b43771-0458-4ddf-81ff-53127f581393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:23.670612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b43771-0458-4ddf-81ff-53127f581393 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:23.678190 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 185/380] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:23 2022] GET /identity/v3/role_assignments?group.id=55e7cc106a2944a79c91b352d3f21e61&scope.project.id=7d515506957d41b7867232158c5ad868&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 260 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:24.474439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5a92e-64a1-46d5-be42-64f848033971 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:24.475008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5a92e-64a1-46d5-be42-64f848033971 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:24.475346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5a92e-64a1-46d5-be42-64f848033971 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:24.478019 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 196/381] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:24.486947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5b1f7-0aac-417d-aad0-127289f0d012 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:24.487431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5b1f7-0aac-417d-aad0-127289f0d012 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:24.487778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5b1f7-0aac-417d-aad0-127289f0d012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:24.526651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-08a5b1f7-0aac-417d-aad0-127289f0d012 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:24.528793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-08a5b1f7-0aac-417d-aad0-127289f0d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:24.669307 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 186/382] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:24.801003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90f941-2415-4374-b2b9-2541dfab2905 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:24.801482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90f941-2415-4374-b2b9-2541dfab2905 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:24.802226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90f941-2415-4374-b2b9-2541dfab2905 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:24.805161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 197/383] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:24.811893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1777-3971-4bf2-aef8-38a7623e7e66 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:24.812475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1777-3971-4bf2-aef8-38a7623e7e66 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:24.812941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1777-3971-4bf2-aef8-38a7623e7e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:24.848598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9cca1777-3971-4bf2-aef8-38a7623e7e66 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:24.850507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9cca1777-3971-4bf2-aef8-38a7623e7e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:24.882025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:24.993542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 187/384] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:25.342868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90f941-2415-4374-b2b9-2541dfab2905 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:25.553479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90f941-2415-4374-b2b9-2541dfab2905 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:25.588210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:25.626675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1777-3971-4bf2-aef8-38a7623e7e66 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:25.633637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:25.639909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:25.640428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:25.641122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:25.670859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:25.679821 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:25.680995 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:25.681922 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:25.688014 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 198/385] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:25.693738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:25.848364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1777-3971-4bf2-aef8-38a7623e7e66 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:25.906737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1035c77-7b12-46dc-9137-8029c5a1661d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:25.925205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:25.926264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:25.927314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:25.927739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:25.951972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:25.960546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:25.961570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:25.962387 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:25.965263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 188/386] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:25.971638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:25.990076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:25.992418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:25.993048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:25.994436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:25.996762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:25.997419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:25.997994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:26.000096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:26.023739 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 199/387] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:26.030071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:26.095536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:26.162859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75ff8-1a21-4068-b209-6291c171660b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:26.223312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fbcbc-341e-4b5e-9119-a5f2ffc9ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:26.224414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fbcbc-341e-4b5e-9119-a5f2ffc9ab3f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:26.224914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fbcbc-341e-4b5e-9119-a5f2ffc9ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:26.225381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fbcbc-341e-4b5e-9119-a5f2ffc9ab3f None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:26.227638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fbcbc-341e-4b5e-9119-a5f2ffc9ab3f None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:26.230326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fbcbc-341e-4b5e-9119-a5f2ffc9ab3f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:26.234882 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 200/388] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:26 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:26.237679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f186a1-b6ba-4cd4-888b-584a64df59a1 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:26.238152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f186a1-b6ba-4cd4-888b-584a64df59a1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:26.238657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f186a1-b6ba-4cd4-888b-584a64df59a1 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:26.241656 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 201/389] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:26.243006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f186a1-b6ba-4cd4-888b-584a64df59a1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:26.259552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44c2a8-3161-4d5f-95de-155925dd1970 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:26.336985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59673-866e-4f25-b03d-c6d3fe5df2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:26.338385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59673-866e-4f25-b03d-c6d3fe5df2e1 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:26.338852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59673-866e-4f25-b03d-c6d3fe5df2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:26.339297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59673-866e-4f25-b03d-c6d3fe5df2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:26.340504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59673-866e-4f25-b03d-c6d3fe5df2e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:26.341305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59673-866e-4f25-b03d-c6d3fe5df2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:26.345078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59673-866e-4f25-b03d-c6d3fe5df2e1 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:26.364466 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 189/390] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:26.367325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd175f3-cdb2-4343-9101-77d58ba94393 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:26.367791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd175f3-cdb2-4343-9101-77d58ba94393 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:26.368536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd175f3-cdb2-4343-9101-77d58ba94393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:26.402354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5bd175f3-cdb2-4343-9101-77d58ba94393 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:26.404353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5bd175f3-cdb2-4343-9101-77d58ba94393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:26.470316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f186a1-b6ba-4cd4-888b-584a64df59a1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:26.497841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:26.530632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:26.539171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:26.540621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:26.541331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:26.542040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:26.544473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:26.547633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:26.549139 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 190/391] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:26.550919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd175f3-cdb2-4343-9101-77d58ba94393 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:26.552207 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 202/392] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:26 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:26.559975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:26.801806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbe045-9ac5-4a10-859a-a6605dad5c36 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:26.851492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd175f3-cdb2-4343-9101-77d58ba94393 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:26.892332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73098-56e1-4dde-b7a1-d75c54de50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:26.893068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73098-56e1-4dde-b7a1-d75c54de50cd None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:26.893347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73098-56e1-4dde-b7a1-d75c54de50cd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:26.893606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73098-56e1-4dde-b7a1-d75c54de50cd None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/groups/b47f65768dab4753ae7bb091c59fa850/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:26.895664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:26.896381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:26.896681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:26.896941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:26.898222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:26.899686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:26.901834 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 191/393] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:26 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:26.902130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73098-56e1-4dde-b7a1-d75c54de50cd None admin] RBAC: Authorizing `identity:create_grant(project_id=6e71bfc5cdae43b9baf98881e1a87f1c, group_id=b47f65768dab4753ae7bb091c59fa850, role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:26.903595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73098-56e1-4dde-b7a1-d75c54de50cd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:26.907345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:26.927293 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 203/394] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 27 09:39:26 2022] PUT /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/groups/b47f65768dab4753ae7bb091c59fa850/roles/6f0bec1f01514029a1162fe631d80648 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:39:27.017108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d82e-74d6-4811-8dec-ab7761f1148d None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.018109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d82e-74d6-4811-8dec-ab7761f1148d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.018896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d82e-74d6-4811-8dec-ab7761f1148d None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.022604 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 204/395] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:39:27.027747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50a15-b2b7-4b90-88c8-02a6093fa843 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.028191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50a15-b2b7-4b90-88c8-02a6093fa843 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.028351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f50a15-b2b7-4b90-88c8-02a6093fa843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.040896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:27.067975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c2f50a15-b2b7-4b90-88c8-02a6093fa843 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:27.069141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c2f50a15-b2b7-4b90-88c8-02a6093fa843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:27.104634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbcc2c-311a-4eb8-9c61-aca53c8aa938 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:27.149058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:27.167014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:27.167991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.168279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.168554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.170607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:27.172166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:27.175223 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 192/396] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:26 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:27.176270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:27.204515 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 205/397] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:27.206669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f50a15-b2b7-4b90-88c8-02a6093fa843 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:27.345625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de3e9-84bd-4c9a-a6de-7053f75a057b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:27.390007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f50a15-b2b7-4b90-88c8-02a6093fa843 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:27.422772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:27.423476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.423763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.424039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.427391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:27.449776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:27.455070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:27.457567 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:27.459176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 193/398] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:27.466504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:27.470614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:27.472169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.473106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.473892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9/groups/b47f65768dab4753ae7bb091c59fa850/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.481401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] RBAC: Authorizing `identity:create_grant(project_id=c5015560e7c748678c104319614c21f9, group_id=b47f65768dab4753ae7bb091c59fa850, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:27.484646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:27.524003 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 206/399] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 27 09:39:27 2022] PUT /identity/v3/projects/c5015560e7c748678c104319614c21f9/groups/b47f65768dab4753ae7bb091c59fa850/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:39:27.590418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:27.685199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:27.720474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af133c40-32cc-4e74-b4e8-f84ba6c056e2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:27.805482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:27.806129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.806410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.806694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.807396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] RBAC: Adding query filter params (name=member) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:27.807691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:27.809156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:27.824039 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 194/400] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:27.829983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:27.852954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b99b9e-a297-4826-bb1f-e1632a73456d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:27.926674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93e23-cc80-438d-b930-3bad396ba86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:27.927258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93e23-cc80-438d-b930-3bad396ba86f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.927537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93e23-cc80-438d-b930-3bad396ba86f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.927810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93e23-cc80-438d-b930-3bad396ba86f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.940492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c93e23-cc80-438d-b930-3bad396ba86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-91c93e23-cc80-438d-b930-3bad396ba86f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:27.946822 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:27.947908 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:27.948759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 207/401] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:27.949496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6965a6-cfff-468b-9ded-81b664c2e181 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.950075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6965a6-cfff-468b-9ded-81b664c2e181 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.950075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6965a6-cfff-468b-9ded-81b664c2e181 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.951521 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 208/402] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:27.953415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:27.957626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b8179-ed00-4d8a-b87f-79804fb352ba None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:27.957626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b8179-ed00-4d8a-b87f-79804fb352ba None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:27.958025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b8179-ed00-4d8a-b87f-79804fb352ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:27.978359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5e1b8179-ed00-4d8a-b87f-79804fb352ba None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:27.979692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5e1b8179-ed00-4d8a-b87f-79804fb352ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:28.043453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f316-72ce-4a4b-93e4-72629c104c22 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:28.072628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 209/403] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:28.074716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b8179-ed00-4d8a-b87f-79804fb352ba None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:28.147064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:28.148090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:28.148539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:28.149046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:28.156721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:28.159432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:28.161690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:28.167194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 195/404] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:27 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:28.176948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:28.342628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b8179-ed00-4d8a-b87f-79804fb352ba None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:28.398330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:28.398960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:28.399237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:28.399507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:28.400792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:28.400792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:28.401928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:28.411759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 210/405] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:28.422114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:28.438315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a841c-3f20-4405-abbd-b2c3282e88cf None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:28.493713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:28.517057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:28.518501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:28.519194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:28.519905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:28.522302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:28.525628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:28.529555 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 196/406] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:28 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:28.538377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:28.633322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceab944-e717-49e8-be09-e46cbe8bd4d3 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:28.637214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:28.680995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:28.681844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:28.682249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:28.682639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:28.684113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:28.685851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:28.688231 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 211/407] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:28 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:28.689350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:28.728431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21707588-e253-4e97-a1d8-028c44663b52 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:28.772820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:28.773563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:28.773839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:28.774116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/groups/b47f65768dab4753ae7bb091c59fa850/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:28.776183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e71bfc5cdae43b9baf98881e1a87f1c, group_id=b47f65768dab4753ae7bb091c59fa850, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:28.777912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:28.801462 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 197/408] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 27 09:39:28 2022] PUT /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c/groups/b47f65768dab4753ae7bb091c59fa850/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:39:28.802683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:28.841229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f5903-5acf-40b2-a9ff-9a3824da1195 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:28.915996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:28.947608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:28.948203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:28.948482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:28.948821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:28.979617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:28.993146 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:28.996937 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:29.000056 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 212/409] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:39:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:29.006889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097f45d-f2e3-43ae-ba76-a526841a5136 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:29.007816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:29.029782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:29.102491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:29.103188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:29.103472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:29.103753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:29.105104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:29.106708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:29.108881 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 198/410] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:28 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:29.115816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:29.168328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b09477-aecf-40f6-bbc2-992083c248ed None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:29.212217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:29.212792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:29.213578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:29.213872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:29.214583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:29.214892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:29.217313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:29.229218 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 213/411] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:29.238641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:29.245766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96730c-1ec6-4286-9f40-637d0a01fb4e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:29.295459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:29.333165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:29.335506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:29.336818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:29.337362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:29.337847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9/groups/b47f65768dab4753ae7bb091c59fa850/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:29.346442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] RBAC: Authorizing `identity:create_grant(project_id=c5015560e7c748678c104319614c21f9, group_id=b47f65768dab4753ae7bb091c59fa850, role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:29.349770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:29.372386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d5525-d45b-48c7-a353-b74860b32a93 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:29.414807 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 199/412] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 27 09:39:29 2022] PUT /identity/v3/projects/c5015560e7c748678c104319614c21f9/groups/b47f65768dab4753ae7bb091c59fa850/roles/6f0bec1f01514029a1162fe631d80648 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:39:29.425739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:29.426427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:29.426710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:29.427049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] PATH_INFO: `/v3/groups/55e7cc106a2944a79c91b352d3f21e61` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:29.429784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] RBAC: Authorizing `identity:get_group(group_id=55e7cc106a2944a79c91b352d3f21e61)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:29.431368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:29.433734 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 214/413] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:29 2022] GET /identity/v3/groups/55e7cc106a2944a79c91b352d3f21e61 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:29.443627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:29.700351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff16c-5a88-4c27-a93e-b1371138f99d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:29.763108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f7d8d-6d7b-4ee1-9aa8-c0c15e4ef9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:29.764739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7d8d-6d7b-4ee1-9aa8-c0c15e4ef9d6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:29.765372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7d8d-6d7b-4ee1-9aa8-c0c15e4ef9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:29.766162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7d8d-6d7b-4ee1-9aa8-c0c15e4ef9d6 None admin] PATH_INFO: `/v3/projects/7d515506957d41b7867232158c5ad868` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:29.768400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f7d8d-6d7b-4ee1-9aa8-c0c15e4ef9d6 None admin] RBAC: Authorizing `identity:get_project(project_id=7d515506957d41b7867232158c5ad868)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:29.771567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f7d8d-6d7b-4ee1-9aa8-c0c15e4ef9d6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:29.774767 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 200/414] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:29 2022] GET /identity/v3/projects/7d515506957d41b7867232158c5ad868 => generated 344 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:29.780651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:29.932859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25251c-af09-4b30-9569-094b62ca8faa None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:29.970114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718a28a-19ad-4820-a9f2-7e341cecfe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:29.970801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a28a-19ad-4820-a9f2-7e341cecfe4e None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:29.971081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a28a-19ad-4820-a9f2-7e341cecfe4e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:29.971358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a28a-19ad-4820-a9f2-7e341cecfe4e None admin] PATH_INFO: `/v3/projects/7d515506957d41b7867232158c5ad868/groups/55e7cc106a2944a79c91b352d3f21e61/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:29.973862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718a28a-19ad-4820-a9f2-7e341cecfe4e None admin] RBAC: Authorizing `identity:create_grant(project_id=7d515506957d41b7867232158c5ad868, group_id=55e7cc106a2944a79c91b352d3f21e61, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:29.975645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718a28a-19ad-4820-a9f2-7e341cecfe4e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:29.998062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:30.003617 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 215/415] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Sep 27 09:39:29 2022] PUT /identity/v3/projects/7d515506957d41b7867232158c5ad868/groups/55e7cc106a2944a79c91b352d3f21e61/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:39:31.299377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6282b0a-890f-44cc-a0bc-58175e6d3a74 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:31.299377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6282b0a-890f-44cc-a0bc-58175e6d3a74 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:31.299377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6282b0a-890f-44cc-a0bc-58175e6d3a74 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:31.300602 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 201/416] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:31.305318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7223-8eb0-4b72-ad4a-32e04b1317f1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:31.306278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7223-8eb0-4b72-ad4a-32e04b1317f1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:31.306766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7223-8eb0-4b72-ad4a-32e04b1317f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:31.333949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-198d7223-8eb0-4b72-ad4a-32e04b1317f1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:31.335431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49163272-b563-4131-9562-b82f78f4cde8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:31.335713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49163272-b563-4131-9562-b82f78f4cde8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:31.335986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49163272-b563-4131-9562-b82f78f4cde8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:31.336178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-198d7223-8eb0-4b72-ad4a-32e04b1317f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:31.337879 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 202/417] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:31.346175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5d1ff-4682-45fc-9761-a0f7f1158d44 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:31.346493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5d1ff-4682-45fc-9761-a0f7f1158d44 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:31.346734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5d1ff-4682-45fc-9761-a0f7f1158d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:31.369262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-79e5d1ff-4682-45fc-9761-a0f7f1158d44 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:31.370511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79e5d1ff-4682-45fc-9761-a0f7f1158d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:31.492206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:31.512458 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 216/418] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:31.535328 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 203/419] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:31.879169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7223-8eb0-4b72-ad4a-32e04b1317f1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:31.984994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5d1ff-4682-45fc-9761-a0f7f1158d44 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:32.037212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7223-8eb0-4b72-ad4a-32e04b1317f1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:32.100532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:32.109799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:32.110435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.110712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.111013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] PATH_INFO: `/v3/roles/member` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.136988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:32.141891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:32.142787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:32.143613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:32.149265 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 217/420] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:32.152947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:32.194325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5d1ff-4682-45fc-9761-a0f7f1158d44 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:32.277898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:32.278547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.278820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.279084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.305846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:32.322119 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:32.323088 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:32.323900 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 204/421] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:32.330950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:32.344333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeae63-9a8b-494f-a487-ae44fc04b762 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:32.407418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:32.408716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd47f-1674-445a-b2ec-478600f9a08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:32.409358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd47f-1674-445a-b2ec-478600f9a08d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.409631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd47f-1674-445a-b2ec-478600f9a08d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.410464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd47f-1674-445a-b2ec-478600f9a08d None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.411137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd47f-1674-445a-b2ec-478600f9a08d None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:32.411490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd47f-1674-445a-b2ec-478600f9a08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:32.412901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd47f-1674-445a-b2ec-478600f9a08d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:32.423384 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 218/422] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:32.427336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedb7a1-66a5-4c21-80de-d45e32b71903 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.429869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedb7a1-66a5-4c21-80de-d45e32b71903 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.430156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedb7a1-66a5-4c21-80de-d45e32b71903 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.432005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 219/423] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:32.439405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdedb7a1-66a5-4c21-80de-d45e32b71903 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:32.495994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c67959-fcce-431e-b6b1-95aea462ac26 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:32.510864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:32.544883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc2557-b227-46b7-a946-1cae58e4fd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:32.546764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc2557-b227-46b7-a946-1cae58e4fd75 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.547193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc2557-b227-46b7-a946-1cae58e4fd75 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.547607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc2557-b227-46b7-a946-1cae58e4fd75 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.548448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc2557-b227-46b7-a946-1cae58e4fd75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:32.548923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc2557-b227-46b7-a946-1cae58e4fd75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:32.550545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc2557-b227-46b7-a946-1cae58e4fd75 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:32.561990 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 205/424] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:32.569598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123af8-57fa-4894-8678-c2e5cf838716 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.569887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123af8-57fa-4894-8678-c2e5cf838716 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.570176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123af8-57fa-4894-8678-c2e5cf838716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.596511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ee123af8-57fa-4894-8678-c2e5cf838716 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:32.597887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee123af8-57fa-4894-8678-c2e5cf838716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:32.617070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdedb7a1-66a5-4c21-80de-d45e32b71903 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:32.661799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235c67b-3f5b-44da-8094-adb1a326aaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:32.662337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235c67b-3f5b-44da-8094-adb1a326aaac None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.662614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235c67b-3f5b-44da-8094-adb1a326aaac None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.662888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235c67b-3f5b-44da-8094-adb1a326aaac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.712479 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 206/425] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:32.712479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123af8-57fa-4894-8678-c2e5cf838716 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:32.773779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:32.797622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235c67b-3f5b-44da-8094-adb1a326aaac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:32.799282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235c67b-3f5b-44da-8094-adb1a326aaac None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:32.879736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:32.886041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123af8-57fa-4894-8678-c2e5cf838716 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:32.929973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:32.930588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.930894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.931440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.946857 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 220/426] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:32.948603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0882c-4860-4ec8-8c28-be40cb4303a0 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:32.949153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0882c-4860-4ec8-8c28-be40cb4303a0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:32.949611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0882c-4860-4ec8-8c28-be40cb4303a0 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:32.951509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 221/427] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:32.958273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0882c-4860-4ec8-8c28-be40cb4303a0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.038785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:33.040542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:33.081931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:33.084083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0882c-4860-4ec8-8c28-be40cb4303a0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.128013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113eeab-26fe-4238-bb94-4e7e2d30113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:33.128652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113eeab-26fe-4238-bb94-4e7e2d30113a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.129144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113eeab-26fe-4238-bb94-4e7e2d30113a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.129629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113eeab-26fe-4238-bb94-4e7e2d30113a None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.131041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113eeab-26fe-4238-bb94-4e7e2d30113a None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:33.132478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113eeab-26fe-4238-bb94-4e7e2d30113a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:33.134156 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 222/428] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:32 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:33.135304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076bbbc-224e-4390-85ed-f57e692f908f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.135582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076bbbc-224e-4390-85ed-f57e692f908f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.135889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076bbbc-224e-4390-85ed-f57e692f908f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.156473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d076bbbc-224e-4390-85ed-f57e692f908f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:33.157695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d076bbbc-224e-4390-85ed-f57e692f908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:33.174559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:33.176264 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 207/429] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:33.176613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.241017 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 223/430] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:33.241811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076bbbc-224e-4390-85ed-f57e692f908f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.299725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771f4a-db38-43e7-913d-4ebc4b29522d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.332919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:33.339295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:33.339953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.340229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.340497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] PATH_INFO: `/v3/roles/member` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.354463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:33.359447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:33.360581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 208/431] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:33.367382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.385465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076bbbc-224e-4390-85ed-f57e692f908f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.427598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a1a98-5890-4923-8717-603b5542e848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:33.428242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1a98-5890-4923-8717-603b5542e848 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.428516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1a98-5890-4923-8717-603b5542e848 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.428819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1a98-5890-4923-8717-603b5542e848 None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.430718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a1a98-5890-4923-8717-603b5542e848 None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:33.432189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a1a98-5890-4923-8717-603b5542e848 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:33.434154 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 224/432] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:33 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:33.435320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7668d3-e5f6-4158-b57f-47725e760b05 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.435594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7668d3-e5f6-4158-b57f-47725e760b05 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.435860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7668d3-e5f6-4158-b57f-47725e760b05 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.441504 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 225/433] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:33.441504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7668d3-e5f6-4158-b57f-47725e760b05 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.472802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:33.501200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a928a47-f025-46ce-842c-d5b399d2490d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.542564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:33.543371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.543718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.543989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.545231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:33.546690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:33.553438 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 209/434] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:33 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:33.553438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.589133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7668d3-e5f6-4158-b57f-47725e760b05 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.620448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:33.634375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3f68-0b2a-4e4f-8c93-4ce28724f02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:33.634980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3f68-0b2a-4e4f-8c93-4ce28724f02d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.635248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3f68-0b2a-4e4f-8c93-4ce28724f02d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.635510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3f68-0b2a-4e4f-8c93-4ce28724f02d None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.636175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3f68-0b2a-4e4f-8c93-4ce28724f02d None admin] RBAC: Adding query filter params (name=member) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:33.636466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3f68-0b2a-4e4f-8c93-4ce28724f02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:33.637918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3f68-0b2a-4e4f-8c93-4ce28724f02d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:33.646945 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 226/435] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:39:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:33.648227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff2bf4-0fe9-4531-a7b6-ffd5f54fb8be None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.648509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff2bf4-0fe9-4531-a7b6-ffd5f54fb8be None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.648813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff2bf4-0fe9-4531-a7b6-ffd5f54fb8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.670740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-85ff2bf4-0fe9-4531-a7b6-ffd5f54fb8be None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:33.671936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85ff2bf4-0fe9-4531-a7b6-ffd5f54fb8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:33.672200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d75308-253f-4f11-995e-079d53217e10 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.720996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:33.721704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:33.722016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:33.722274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:33.723213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=c5015560e7c748678c104319614c21f9) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:33.723213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:33.725195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:33.735015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 210/436] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:33 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:33.736231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.780873 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 227/437] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:33.780873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff2bf4-0fe9-4531-a7b6-ffd5f54fb8be None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:33.798175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:33.961101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07d37d-94a2-4d2e-8466-f5fb5a632b69 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.992698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff2bf4-0fe9-4531-a7b6-ffd5f54fb8be None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:33.994349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:34.008422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.009119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.010042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.010320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.011558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:34.013577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:34.015599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 211/438] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:33 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:34.016066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:34.069729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.070713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.071167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.071621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.093840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:34.099338 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:34.101028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 228/439] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:34.102539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:34.174720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb08458-c80e-4aa1-a84c-1dde08076441 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:34.203038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:34.232462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.232462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.232949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.233391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.256072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37ae99-6ec7-4c01-823d-3ebc14da4f14 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:34.314298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.314938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.315215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.315491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.316295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:34.316578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:34.318097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:34.323357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:34.325144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 229/440] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:34 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=6f0bec1f01514029a1162fe631d80648 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:34.332951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:34.353305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:34.354992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:34.517847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 212/441] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:34.517847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:34.538775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b6e95-573c-48c1-b107-cb57c8b10bf3 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:34.599772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.600641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.601086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.605571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.605571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:34.605571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:34.605571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:34.618929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:34.624872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 230/442] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:34.626846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:34.680176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:34.696354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61fa6a-1270-4c4b-804c-b560c84e24e4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:34.753332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.754568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.755006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.755445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.771665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:34.777163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544c619-08bc-46ee-8166-5f58ab8b41b8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:34.777496 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:34.779299 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:34.781039 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 213/443] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:34.783500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:34.820813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.822277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.822841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.823205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] PATH_INFO: `/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.825075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] RBAC: Authorizing `identity:get_project(project_id=6e71bfc5cdae43b9baf98881e1a87f1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:34.827526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:34.830627 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 231/444] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:34 2022] GET /identity/v3/projects/6e71bfc5cdae43b9baf98881e1a87f1c => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:34.831621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:34.917342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:34.920528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23143b-dce3-4050-82a0-af798ede2c7f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:34.962402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:34.963139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:34.963600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:34.963922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:34.971971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b5820-3fad-4769-8209-540f5d688914 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:34.991900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:35.015668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:35.016717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:35.017044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:35.017352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:35.018202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:35.018549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.020423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.030559 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 232/445] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:35.031787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.048645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.050421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.154687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d2709-64f8-4a17-b6ff-f5ff2e5d53bc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:35.175438 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 214/446] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:35.178107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.187745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:35.200688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:35.201683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:35.202050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:35.202444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:35.204187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.206629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.209766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 233/447] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:35.210576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.248992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:35.312192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac73f37-553d-48b3-9faf-6966c204e060 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:35.340603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2d573-8cf6-410c-b338-681b0452454f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:35.358187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:35.358927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:35.359302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:35.359736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:35.400921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:35.401747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:35.402092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:35.402546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] PATH_INFO: `/v3/projects/c5015560e7c748678c104319614c21f9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:35.403970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] RBAC: Authorizing `identity:get_project(project_id=c5015560e7c748678c104319614c21f9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.405779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.407566 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 234/448] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/projects/c5015560e7c748678c104319614c21f9 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:35.408582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.453444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.454993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.492273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:35.517580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:35.542475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1cc11-1d30-4d0d-b019-8a4a59daea4b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:35.586810 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 215/449] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:35.587910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.598054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:35.598774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:35.599232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:35.599645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:35.600511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=a0597e11955a4725ae4938bb442c75f6, scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:35.601044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.602831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.611402 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 235/450] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=6e71bfc5cdae43b9baf98881e1a87f1c&role.id=a0597e11955a4725ae4938bb442c75f6 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:35.612559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.773090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec9464-1b3f-4587-9b7d-46bd42629260 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:35.775355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c9b5-61fe-4bba-8577-9c5cd3fe7bb0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:35.813687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:35.831154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:35.832301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:35.832835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:35.833227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] PATH_INFO: `/v3/projects/7d515506957d41b7867232158c5ad868` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:35.834974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] RBAC: Authorizing `identity:get_project(project_id=7d515506957d41b7867232158c5ad868)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.837588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.840832 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 236/451] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/projects/7d515506957d41b7867232158c5ad868 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:35.841873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:35.842937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:35.843396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:35.843861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] PATH_INFO: `/v3/groups/b47f65768dab4753ae7bb091c59fa850` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:35.845964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] RBAC: Authorizing `identity:get_group(group_id=b47f65768dab4753ae7bb091c59fa850)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:35.848912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:35.850011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.851914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 216/452] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/groups/b47f65768dab4753ae7bb091c59fa850 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:35.860289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:35.900429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:36.022823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbaa84-e8c4-45b7-a20b-68ec9460811d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:36.029799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea16f6-3b4a-4288-9bdf-eaec70d91520 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:36.076484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:36.077187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:36.077470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:36.077745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] PATH_INFO: `/v3/groups/55e7cc106a2944a79c91b352d3f21e61` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:36.078968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] RBAC: Authorizing `identity:get_group(group_id=55e7cc106a2944a79c91b352d3f21e61)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:36.080933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:36.082494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 237/453] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/groups/55e7cc106a2944a79c91b352d3f21e61 => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:36.090110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:36.103442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b2c21-34b9-47d0-9e05-67b254f2f32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:36.104083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b2c21-34b9-47d0-9e05-67b254f2f32b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:36.104358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b2c21-34b9-47d0-9e05-67b254f2f32b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:36.104646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b2c21-34b9-47d0-9e05-67b254f2f32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:36.105587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b2c21-34b9-47d0-9e05-67b254f2f32b None admin] RBAC: Adding query filter params (group.id=b47f65768dab4753ae7bb091c59fa850, role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=c5015560e7c748678c104319614c21f9) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:36.105899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b2c21-34b9-47d0-9e05-67b254f2f32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:36.107491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b2c21-34b9-47d0-9e05-67b254f2f32b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:36.114675 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 217/454] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:35 2022] GET /identity/v3/role_assignments?group.id=b47f65768dab4753ae7bb091c59fa850&scope.project.id=c5015560e7c748678c104319614c21f9&role.id=6f0bec1f01514029a1162fe631d80648 => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:36.188855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:36.255491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4cb06-c413-4444-975c-a251deda2ad3 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:36.314059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b2225-542a-4a0a-a265-d0662e431364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:36.315040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2225-542a-4a0a-a265-d0662e431364 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:36.315564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2225-542a-4a0a-a265-d0662e431364 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:36.316096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2225-542a-4a0a-a265-d0662e431364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:36.317912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2225-542a-4a0a-a265-d0662e431364 None admin] RBAC: Adding query filter params (group.id=55e7cc106a2944a79c91b352d3f21e61, role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=7d515506957d41b7867232158c5ad868) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:36.318461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2225-542a-4a0a-a265-d0662e431364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:36.320306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2225-542a-4a0a-a265-d0662e431364 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:36.331080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 238/455] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Sep 27 09:39:36 2022] GET /identity/v3/role_assignments?group.id=55e7cc106a2944a79c91b352d3f21e61&scope.project.id=7d515506957d41b7867232158c5ad868&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:40.254956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec002b7-ab7c-48ad-8a68-5660b66cdfb4 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.255493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec002b7-ab7c-48ad-8a68-5660b66cdfb4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.255493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec002b7-ab7c-48ad-8a68-5660b66cdfb4 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.257437 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 218/456] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:40.258478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49aa36-d915-402b-8f0c-2df371ace354 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.259301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49aa36-d915-402b-8f0c-2df371ace354 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.259774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49aa36-d915-402b-8f0c-2df371ace354 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.261888 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 239/457] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:39:40.263139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c67a-7a58-4dcd-b604-ff67605dd76e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.263369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c67a-7a58-4dcd-b604-ff67605dd76e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.263638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c67a-7a58-4dcd-b604-ff67605dd76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.271322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704efe7-3563-4f27-8836-5223892c7177 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.271322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704efe7-3563-4f27-8836-5223892c7177 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.271322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704efe7-3563-4f27-8836-5223892c7177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.283289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7831c67a-7a58-4dcd-b604-ff67605dd76e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:40.285185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7831c67a-7a58-4dcd-b604-ff67605dd76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:40.296557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e704efe7-3563-4f27-8836-5223892c7177 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:40.297802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e704efe7-3563-4f27-8836-5223892c7177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:40.395357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 219/458] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:40.396982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb4ff-0458-4dcf-9759-a3d2a782759a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.397433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb4ff-0458-4dcf-9759-a3d2a782759a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.397945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb4ff-0458-4dcf-9759-a3d2a782759a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.399467 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 220/459] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:40.400837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ac0e4-f4f9-413c-9104-6e825424bc0a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.401285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ac0e4-f4f9-413c-9104-6e825424bc0a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.401724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ac0e4-f4f9-413c-9104-6e825424bc0a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.403354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 221/460] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:40.407576 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 240/461] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:40.410703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df5ee0-9976-4ffa-b179-ebd1f4939708 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.411328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df5ee0-9976-4ffa-b179-ebd1f4939708 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.411867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df5ee0-9976-4ffa-b179-ebd1f4939708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.418999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1127b8-0c0c-4f08-beb4-61e111c5685c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.419251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1127b8-0c0c-4f08-beb4-61e111c5685c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.419585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1127b8-0c0c-4f08-beb4-61e111c5685c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.434616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d3df5ee0-9976-4ffa-b179-ebd1f4939708 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:40.436176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3df5ee0-9976-4ffa-b179-ebd1f4939708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:40.447801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:40.462619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-eb1127b8-0c0c-4f08-beb4-61e111c5685c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:40.463827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eb1127b8-0c0c-4f08-beb4-61e111c5685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:40.526086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:40.587368 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 222/462] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:40.614523 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 241/463] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:39:40.784254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df5ee0-9976-4ffa-b179-ebd1f4939708 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:40.789754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1127b8-0c0c-4f08-beb4-61e111c5685c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:40.908783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df5ee0-9976-4ffa-b179-ebd1f4939708 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:40.921476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1127b8-0c0c-4f08-beb4-61e111c5685c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:40.971644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:40.972394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.972799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.973028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:40.973189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.973677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:40.973948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:40.974309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:40.986233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:40.986429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:40.987817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:40.987923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:40.992305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:41.008181 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:41.009421 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:41.010740 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:41.010740 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.010740 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:41.010740 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 242/464] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:41.010740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:41.014370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:41.015499 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:41.016546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:41.016546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.016546 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:41.018295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 223/465] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:41.020393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.066817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:41.137296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940620-ffac-4c84-938b-a75d06202985 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.142852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf12c9-1bf8-476e-b4af-e8c2368e9196 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.175722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.176623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.176937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.177207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.177878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:41.178185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.179578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.182680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.183281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.183554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.183819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.201652 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 224/466] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:41.201863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.201956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.202068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:41.219184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:41.220198 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:41.221616 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:41.221616 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.221616 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:41.226649 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 243/467] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:41.227822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.278120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:41.329631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654882a7-18b2-4bf9-a720-9bc87a194b07 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.342334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:41.353710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b339a-fa53-4748-a5a9-9e0ee066f70d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.368690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.369330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.369614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.369919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.370557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:41.370869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.372283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.384932 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 225/468] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:41.386017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.394844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.395488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.395759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.396026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.407305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.408807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:41.425269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:41.426512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:41.427406 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:41.427406 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:41.427406 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:41.427406 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 244/469] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:41.427690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.500953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285b263-e196-4756-ad5c-3c7ed708e0d4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.519067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:41.541304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc1e8a-b1d6-47b9-91c0-cb4d2de97979 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.549395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.550216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.550489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.550950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.551697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.553529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.582599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.582804 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 226/470] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:39:41 2022] POST /identity/v3/users => generated 249 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:39:41.583847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.584308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.584790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.585656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:41.586177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.587811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.589960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.617805 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 245/471] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:41.621044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.629495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:41.763601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff7947-1b11-480f-be50-9e159dcf0ef8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.785556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d7049-ad56-4b8f-952a-17bb17c9dcbe None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:41.808003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.808772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.809058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.809311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.809974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.811655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.834706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e2e96-3015-47d0-b41d-2c02010d5f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:41.835753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2e96-3015-47d0-b41d-2c02010d5f88 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.835753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2e96-3015-47d0-b41d-2c02010d5f88 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.835993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e2e96-3015-47d0-b41d-2c02010d5f88 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.836472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2e96-3015-47d0-b41d-2c02010d5f88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:41.836899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2e96-3015-47d0-b41d-2c02010d5f88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:41.838329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e2e96-3015-47d0-b41d-2c02010d5f88 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:41.838975 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 227/472] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:39:41 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:39:41.839597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:41.853061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 246/473] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:41.854539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80031f46-2c92-48b8-a966-4fc3e03aeae1 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.854868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80031f46-2c92-48b8-a966-4fc3e03aeae1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.855054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80031f46-2c92-48b8-a966-4fc3e03aeae1 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.856708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 247/474] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:41.867828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21e08a-89f6-4f04-816e-92b3a43cfb81 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:41.868356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21e08a-89f6-4f04-816e-92b3a43cfb81 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:41.868868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21e08a-89f6-4f04-816e-92b3a43cfb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:41.874356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:41.908103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7e21e08a-89f6-4f04-816e-92b3a43cfb81 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:41.912612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e21e08a-89f6-4f04-816e-92b3a43cfb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:42.015077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616670b-d159-446b-94aa-3ca7a15d391f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:42.032451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:42.042709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 248/475] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 27 09:39:42.048739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21e08a-89f6-4f04-816e-92b3a43cfb81 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:42.117365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:42.117888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:42.118173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:42.118448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:42.119127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:42.120605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:42.168805 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 228/476] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:39:41 2022] POST /identity/v3/users => generated 248 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:39:42.310906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21e08a-89f6-4f04-816e-92b3a43cfb81 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:42.354734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:42.355771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:42.355771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:42.356030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:42.356568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:42.358368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:42.388509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 249/477] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:39:42 2022] POST /identity/v3/users => generated 246 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:39:42.579101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:42.704747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:42.726719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605057d0-fe22-4509-984b-d5b990545096 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:42.772325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:42.773007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:42.773285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:42.773614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:42.774313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:42.775764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:42.788584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.sql.core [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 27 09:39:42.788584 ncdv-master-1 devstack@keystone.service[104131]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 27 09:39:42.788584 ncdv-master-1 devstack@keystone.service[104131]: [parameters: {'id': '546bdb8867a94634bede5986e7d27d92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 27 09:39:42.788584 ncdv-master-1 devstack@keystone.service[104131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application conn.query(q) Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result.read() Sep 27 09:39:42.806242 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return ref.to_dict() Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 09:39:42.807191 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application yield resource Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise self.value Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application session.commit() Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 09:39:42.808215 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.flush() Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application compat.raise_( Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 09:39:42.809100 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:39:42.809964 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application conn.query(q) Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result.read() Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:39:42.810785 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [parameters: {'id': '546bdb8867a94634bede5986e7d27d92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:42.811574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 27 09:39:42.812366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:42.813328 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 229/478] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 27 09:39:42 2022] POST /identity/v3/roles => generated 144 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:42.816917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:42.887794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:42.965769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438b944-c1b3-4e6f-8a54-dfe094d15c28 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:43.033564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73577bef-58f7-4a5d-8c6e-a277abd52c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:43.034245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73577bef-58f7-4a5d-8c6e-a277abd52c26 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:43.035876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73577bef-58f7-4a5d-8c6e-a277abd52c26 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:43.035876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73577bef-58f7-4a5d-8c6e-a277abd52c26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:43.054710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73577bef-58f7-4a5d-8c6e-a277abd52c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-73577bef-58f7-4a5d-8c6e-a277abd52c26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:43.065008 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:43.065959 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 09:39:43.066977 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:43.071538 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 250/479] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:39:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:43.083961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:43.389047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d952a-e5b8-43b3-85ed-fae294c8c5f6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:43.449315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6a5e3-df96-4d97-afd5-39698dcc02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:43.449957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a5e3-df96-4d97-afd5-39698dcc02d7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:43.450238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a5e3-df96-4d97-afd5-39698dcc02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:43.450511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a5e3-df96-4d97-afd5-39698dcc02d7 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:43.451203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a5e3-df96-4d97-afd5-39698dcc02d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:43.451510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a5e3-df96-4d97-afd5-39698dcc02d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:43.452931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a5e3-df96-4d97-afd5-39698dcc02d7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:43.470348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 230/480] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:44.981788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7be9-3136-4b7d-9e58-9e71d97824ca None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:44.982811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7be9-3136-4b7d-9e58-9e71d97824ca None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:44.983287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7be9-3136-4b7d-9e58-9e71d97824ca None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:44.985309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 251/481] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:44.991087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16e475-a467-476a-95be-dd4dd17bfb93 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:44.991588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16e475-a467-476a-95be-dd4dd17bfb93 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:44.991692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16e475-a467-476a-95be-dd4dd17bfb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:45.013927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4b16e475-a467-476a-95be-dd4dd17bfb93 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:45.015167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b16e475-a467-476a-95be-dd4dd17bfb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:45.022755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:45.104889 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 231/482] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:45.742104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7be9-3136-4b7d-9e58-9e71d97824ca None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:45.773025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b86ae-7c77-4564-bd93-cd13f0d7503e None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:45.773943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b86ae-7c77-4564-bd93-cd13f0d7503e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:45.774664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b86ae-7c77-4564-bd93-cd13f0d7503e None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:45.777641 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 232/483] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:45.787486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23815a-dcba-49b3-be49-3fa1fb5b6af5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:45.788387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23815a-dcba-49b3-be49-3fa1fb5b6af5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:45.789113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23815a-dcba-49b3-be49-3fa1fb5b6af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:45.819941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7f23815a-dcba-49b3-be49-3fa1fb5b6af5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:45.822329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7f23815a-dcba-49b3-be49-3fa1fb5b6af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:45.953916 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 233/484] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:45.956501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335e0af-f525-4125-a806-43709de6d245 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:45.957268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335e0af-f525-4125-a806-43709de6d245 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:45.958019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335e0af-f525-4125-a806-43709de6d245 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:45.962991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7be9-3136-4b7d-9e58-9e71d97824ca None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:45.967318 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 234/485] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:45.976887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb32a-b412-4985-92a7-188719c0c8d3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:45.977684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb32a-b412-4985-92a7-188719c0c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:45.978419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb32a-b412-4985-92a7-188719c0c8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:45.991520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:46.039486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-76ffb32a-b412-4985-92a7-188719c0c8d3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:46.042675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-76ffb32a-b412-4985-92a7-188719c0c8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:46.063746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:46.065316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.065980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.066755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.088625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:46.103142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:46.107187 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:46.109049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:46.113294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 252/486] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:46.118930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:46.190988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:46.200950 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 235/487] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:46.290817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9749b-7452-40cf-8ba4-6b58aedb1a5e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:46.324641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2e9c8-cdca-4ead-85dd-cd76fc76d958 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.324971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2e9c8-cdca-4ead-85dd-cd76fc76d958 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.325202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2e9c8-cdca-4ead-85dd-cd76fc76d958 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.329721 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 236/488] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:39:46.334412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:46.334724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6174385-1afe-4329-a709-3808fbec4a7a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.335012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6174385-1afe-4329-a709-3808fbec4a7a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.335123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.335289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6174385-1afe-4329-a709-3808fbec4a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.335380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.335549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.336225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:46.336524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:46.338042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:46.354442 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 253/489] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:46.363398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:46.377899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e6174385-1afe-4329-a709-3808fbec4a7a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:46.379922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e6174385-1afe-4329-a709-3808fbec4a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:46.400193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:46.474791 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 237/490] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:46.476060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6174385-1afe-4329-a709-3808fbec4a7a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:46.488688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8847608-cc24-4e92-92e0-8c34666c9b8c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:46.540986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:46.541956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.543202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.543409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.585106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:46.587561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:46.628374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:46.630213 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:46.631933 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:46.631933 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:46.631933 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:46.634952 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 254/491] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:46.636364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:46.692021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6174385-1afe-4329-a709-3808fbec4a7a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:46.728649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:46.737573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:46.740424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:46.741642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.742174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.742651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.764694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:46.771094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:46.772025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:46.772841 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:46.773543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10524287-6bc2-497f-ad19-e25e1350b4af None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:46.774888 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 238/492] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:46.776014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:46.838422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:46.840126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.840798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.841840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.842746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:46.843357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:46.846194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:46.874405 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 255/493] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:46.875920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:46.897116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbca64-8a7b-4381-bc00-01080256c797 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:46.942698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:46.943659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:46.944207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:46.944715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:46.956473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:46.962753 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:46.964700 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 239/494] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:46.966315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.022794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1319688-7d9b-440a-bbf7-9715a30255a5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.031349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:47.076295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.076975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.077290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.077625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.078441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:47.078922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:47.080720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:47.100707 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 256/495] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:47.100707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.101531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e11b72-7352-4947-8e65-775323f7bdb0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.107355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:47.147687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.148678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.148988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.149267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.166340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:47.175918 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:47.177708 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 240/496] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:47.180308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.262683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c418f6-309a-4f48-bfd0-547b70a74b93 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.306925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b1753-79b7-428c-bfb9-cdf24139b2b3 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.308625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.309328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.309600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.309867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.319868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:47.321553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:47.321900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:47.339427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:47.341481 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 257/497] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:47.343160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.355709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.356254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.356538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.356851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.357548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:47.357858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:47.359349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:47.369008 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 241/498] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:47.369923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.459772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:47.490837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c8ea9-4796-4365-b676-344bb71e944e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.533621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952054d-7848-4cb6-aa41-0771e6fcb249 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.534287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990ee3f-ae7c-4a7f-bc87-e40c7bcc71ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.534946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990ee3f-ae7c-4a7f-bc87-e40c7bcc71ff None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.535754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990ee3f-ae7c-4a7f-bc87-e40c7bcc71ff None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.536226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990ee3f-ae7c-4a7f-bc87-e40c7bcc71ff None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.537712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990ee3f-ae7c-4a7f-bc87-e40c7bcc71ff None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:47.538242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990ee3f-ae7c-4a7f-bc87-e40c7bcc71ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:47.541059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990ee3f-ae7c-4a7f-bc87-e40c7bcc71ff None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:47.557714 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 242/499] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:47.559895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1b44a-a816-4231-b5e8-6faf9df5490b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.560389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1b44a-a816-4231-b5e8-6faf9df5490b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.561159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1b44a-a816-4231-b5e8-6faf9df5490b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.563838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 243/500] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:47.565450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1b44a-a816-4231-b5e8-6faf9df5490b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.586749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.587428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.587709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.587981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.607874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:47.609659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:47.619256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:47.628697 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:47.630704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:47.632616 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:47.632616 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:47.632616 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:47.634686 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 258/501] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:47.636557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.725897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1b44a-a816-4231-b5e8-6faf9df5490b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.781224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dc667-f250-4e1d-bd99-84d20bb213e1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:47.791537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f36949-ae47-4c3d-99ea-a2cc8914a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.792522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36949-ae47-4c3d-99ea-a2cc8914a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.793014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36949-ae47-4c3d-99ea-a2cc8914a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.793476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36949-ae47-4c3d-99ea-a2cc8914a5e3 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.794793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f36949-ae47-4c3d-99ea-a2cc8914a5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:47.795568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f36949-ae47-4c3d-99ea-a2cc8914a5e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:47.798070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f36949-ae47-4c3d-99ea-a2cc8914a5e3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:47.802589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:47.824167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:47.824862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.825143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.825417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.838968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:47.840484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:47.859225 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:47.860586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:47.862266 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:47.862266 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:47.862266 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:47.862266 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 259/502] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:47.862266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:47.871367 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 244/503] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:47.873602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7b443-5866-4e01-9cc2-64988586b813 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:47.874070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7b443-5866-4e01-9cc2-64988586b813 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:47.874542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7b443-5866-4e01-9cc2-64988586b813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:47.900854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-67c7b443-5866-4e01-9cc2-64988586b813 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:47.902117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-67c7b443-5866-4e01-9cc2-64988586b813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:47.991723 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 245/504] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:47.992745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7b443-5866-4e01-9cc2-64988586b813 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.027842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf2fb9-5d3c-4f55-bae0-c368e9c932b0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.044851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:48.088796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.089818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.090292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.090772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:48.110293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:48.112923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:48.139455 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:48.140559 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:48.141790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:48.141790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:48.141790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:48.141964 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 260/505] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:48.143823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.183872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7b443-5866-4e01-9cc2-64988586b813 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.187281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:48.247706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.248783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.249273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.249930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:48.251007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:48.251676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:48.254195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:48.279997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 246/506] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:48.279997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.321962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e0a10-5204-45d8-af09-d8d47dec569b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.377938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.378797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.380150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.380515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:48.381612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:48.382120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:48.384507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:48.404007 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 261/507] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:48.405179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.422034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:48.447787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b559fd7-45bc-4216-9d4e-0c26657b5d0c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.499508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:48.504963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.505274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.505546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.505810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:48.526746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82ddcc-b5c9-40a3-9c7a-d74a692b3354 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.577706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.578330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.578602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.578872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:48.579521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:48.579814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:48.581239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:48.589132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:48.590749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:48.595007 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 262/508] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:48.595310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.711798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:48.740059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1fb22-91f1-4753-bc90-648f30edc2f2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.747961 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 247/509] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:48.747961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.792452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.793252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.793707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.794074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:48.803550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:48.806035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:48.810251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:48.821855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:48.823781 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 263/510] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:48.825857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.882276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbe27d-3b05-4179-9344-8c15638b4b65 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.922016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.922894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.923251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.923599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:48.935839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:48.937743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:48.947179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c18ad-394b-4744-8a2b-bdac9c809414 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:48.954650 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:48.956329 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:48.958093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:48.958093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:48.958093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:48.958093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 248/511] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:48.958514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:48.994000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:48.994902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:48.995345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:48.995789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.003059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:49.005653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.007407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:49.020798 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:49.022791 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 264/512] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:49.024786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.094643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7509-03ee-4d4a-b317-68ef4811d941 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.101203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:49.147162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.147938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.148293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.148642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.158490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.161755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.162660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42b51-a9d2-4696-8b86-5259ef34a5d5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:49.178284 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:49.180248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 249/513] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:39:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:49.182236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.207906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.208604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.208913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.209178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.221245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.222803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:49.239693 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:49.241595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:49.243233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:49.243233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:49.243233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:49.243233 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 265/514] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:49.243233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.313159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:49.321729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a73be8-fa39-4067-9e50-33de76c773f9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.364186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29693a99-0ee1-4c9e-aac4-6af09832d706 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.365907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.366738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.367099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.367538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.368809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:49.369156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.370802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.381516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:49.394239 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 250/515] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:49.396005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.414356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.415045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.415423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.415760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.416450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:49.416880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.418544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.436170 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 266/516] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:49.437285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.521627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bacdb-5247-4ed3-906a-ed7ca1d6ab12 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.562897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.563909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.564293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.564684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.565603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:49.565987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.567831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88256406-9412-4661-b9b5-8e8963e68bc7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.568280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.595641 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 251/517] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:49.597582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.614758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.615444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.615804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.616125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.617069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:49.617592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.619277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.619467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:49.662233 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 267/518] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:49.663333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.710798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:49.728728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93594414-6267-4b1f-ad4d-383dacbb0e7f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.767836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.768467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.768775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.769044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.769851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:49.770147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.771674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.783746 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 252/519] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:49.784586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:49.834515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5f204-9152-4af6-b7a3-61f5b8f22669 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.858793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:49.894125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.895047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1adae9-e95b-4c2b-91f1-bc5d2da1a1bc None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:49.895210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.895649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.896117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.932563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:49.933211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:49.933474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:49.933754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:49.934412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:49.935875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:49.966376 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 253/520] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:39:49 2022] POST /identity/v3/users => generated 247 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:39:49.967908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:50.002677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:50.005415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:50.061656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:50.161525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc76e63-3a32-419b-9ad0-9a5b2a79d1fa None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:50.201223 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 268/521] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:50.202578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:50.215521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:50.233493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:50.234505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:50.234932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:50.235323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:50.345080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:50.347010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:50.370364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a3d9f-6561-4046-bba0-3df0460a22ce None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:50.409853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:50.419660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:50.420315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:50.420801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:50.421205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:50.477458 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 254/522] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:50.478758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:50.517884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:50.531591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:50.533495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:50.618722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3276f5-d252-4782-a813-acf71a334923 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:50.663796 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 269/523] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:50.664696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:50.677902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:50.679294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:50.679873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:50.680312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:50.692377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:50.694728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:50.715242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:39:50.717069 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:50.718797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 255/524] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:50.720570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:50.815956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e28f0-b0cb-467c-aa73-96a9e86e58e4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:50.857407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:50.858166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:50.858941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:50.859350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:50.859756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:50.882250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:50.884461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbec72-7206-40d7-a18d-64268654f8fa None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:50.884796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:50.911097 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:50.912926 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:50.920393 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:50.920393 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:50.920393 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:50.920393 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 270/525] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:50.920393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:50.946807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:50.947369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:50.947680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:50.947918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:50.962459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:50.966662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:50.993648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:50.994661 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:50.995505 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:50.995505 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:50.995505 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:50.995780 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 256/526] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:50.997765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.032702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:51.066413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bf9f1-27e9-4b55-936d-62314b068499 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.109547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.110338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.110929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.111206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.123067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.124548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.130284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be037aba-e104-4f14-b8e1-88fc280dc076 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:51.142359 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:51.143488 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:51.144360 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:51.144360 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:51.144360 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:51.144360 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 271/527] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:51.145595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.172679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.173519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.173804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.174079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.174782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:51.175086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.176784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.189335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:51.209765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 257/528] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:39:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:51.210721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.323486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef2266-b207-4d31-90f8-56a270e8b4df None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.334073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5ee4c-9ed5-444c-a8b2-ab16008b2660 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.369027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.369646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.369924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.370197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.370855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:51.371157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.372550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.374238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:51.378134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.379447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.380034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.380499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.381364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:51.381854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.383429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.389558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 272/529] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:51.390521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.397030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 258/530] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:51.398114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.476367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:51.529885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4ee9f-96ef-48f3-8959-88f3dbd6dcf2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.537843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484bafb-fc96-4f71-86a1-358d99771f53 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.573792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.574563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.574850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.575120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.575772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:51.576081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.577594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.585915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.585915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.586298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.586589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.587400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=065ad0a36b32453c98ca3ecda6f952a7) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:51.587804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.589610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.591988 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 273/531] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:51.592978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.620689 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 259/532] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/role_assignments?user.id=065ad0a36b32453c98ca3ecda6f952a7&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:51.620689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.664372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:51.724885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc737e-e938-4b89-a076-9d6bb0b1228d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.776094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175364-113a-4650-b803-285770719824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.776094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175364-113a-4650-b803-285770719824 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.776094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175364-113a-4650-b803-285770719824 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.776094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175364-113a-4650-b803-285770719824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.786244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14175364-113a-4650-b803-285770719824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.788799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14175364-113a-4650-b803-285770719824 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-14175364-113a-4650-b803-285770719824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:39:51.805554 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:51.806681 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:51.807723 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:51.816253 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 274/533] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:51.818029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175364-113a-4650-b803-285770719824 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.818632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aa087-e0da-402a-98a5-8442b2ececa5 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.869662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:51.890130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:51.891324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:51.891850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:51.892427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:51.905342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:51.908298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:39:51.924296 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:51.925380 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:51.926789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:51.932829 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 260/534] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:51.932829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:51.971685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175364-113a-4650-b803-285770719824 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:51.991463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:52.017554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:52.018455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.018873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.019282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.028783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:52.030805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:39:52.044494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:52.045606 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:52.046646 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:52.048735 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 275/535] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:52.049887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:52.076294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96b53-84dc-4601-8ea0-62975e761d51 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:52.146062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:52.147134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.147685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.148113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.149004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:52.149466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:52.152010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:52.175101 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 261/536] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:39:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:52.175101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:52.218733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0b46-699b-4674-8324-9d2ec239b58a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:52.232546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:52.266714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:52.267702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.267980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.268253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.268990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:52.269297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:52.270831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:52.293724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 276/537] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:39:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:52.295003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:52.311740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aebba-71e0-4bb6-af7a-9f8322078eed None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:52.344359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:52.367933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:52.369036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.369516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.370662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.371932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:52.372453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:52.375262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:52.403059 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 262/538] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:39:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:52.404199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:52.437820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d8c2-1ad6-4f49-8afb-4b760c65318b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:52.480742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:52.481411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.481693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.482051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.482786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=b324af05ec3e4b35a9bfc188b3b3abec) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:52.483091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:52.485329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:52.500858 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 277/539] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:52 2022] GET /identity/v3/role_assignments?user.id=b324af05ec3e4b35a9bfc188b3b3abec&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:52.502274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:52.650883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54ee0-26bf-4aea-b27f-10d7d28ff4a6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:52.657596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:52.742423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24feb-4b0b-4432-9fea-d74ffcf4011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:52.743064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24feb-4b0b-4432-9fea-d74ffcf4011a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.743342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24feb-4b0b-4432-9fea-d74ffcf4011a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.743613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24feb-4b0b-4432-9fea-d74ffcf4011a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.744364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24feb-4b0b-4432-9fea-d74ffcf4011a None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=ed91b2c83d57419dac50595f57fc3411) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:52.744685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24feb-4b0b-4432-9fea-d74ffcf4011a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:52.746184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24feb-4b0b-4432-9fea-d74ffcf4011a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:52.755152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4958-a9c1-4756-a7e8-3672eac7ad88 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:52.758342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:52.760929 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 263/540] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:52 2022] GET /identity/v3/role_assignments?user.id=ed91b2c83d57419dac50595f57fc3411&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 259 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:52.763021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f425dc-b826-4764-ba0d-9f8d65a41deb None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.763311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f425dc-b826-4764-ba0d-9f8d65a41deb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.763590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f425dc-b826-4764-ba0d-9f8d65a41deb None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.769607 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 264/541] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:52.771192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbf6cf-e057-4d6a-ad79-9bc8e6eeb2a8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.771475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbf6cf-e057-4d6a-ad79-9bc8e6eeb2a8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.771750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbf6cf-e057-4d6a-ad79-9bc8e6eeb2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.815285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7abbf6cf-e057-4d6a-ad79-9bc8e6eeb2a8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:52.817449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7abbf6cf-e057-4d6a-ad79-9bc8e6eeb2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:52.872790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:52.873737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:52.874206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:52.874668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:52.876057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=3c8ed3cf7a5742808029d336ebc784fa) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:52.876591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:52.879804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:52.890592 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 278/542] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:52 2022] GET /identity/v3/role_assignments?user.id=3c8ed3cf7a5742808029d336ebc784fa&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 259 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:52.978847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 265/543] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:53.409008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:53.548755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634209-da8b-4a18-ac52-053c20c98ffd None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:53.592795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:53.593698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:53.594148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:53.594572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:53.617468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:53.623247 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:53.624389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:53.625429 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:53.627829 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 279/544] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:53.634146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbf6cf-e057-4d6a-ad79-9bc8e6eeb2a8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:53.813847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:53.821567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbf6cf-e057-4d6a-ad79-9bc8e6eeb2a8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:53.875161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:53.875789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:53.876064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:53.876329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:53.877033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:53.877330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:53.878785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:53.893318 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 266/545] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:53.899352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:53.951309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:54.104039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad56d8-a251-471a-85aa-d7ff215cacc1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:54.162756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:54.163407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:54.163685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:54.163955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:54.187161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:54.188635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:54.212223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:54.213251 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:54.214109 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:54.214109 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:54.214109 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:54.214109 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 280/546] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:54.221297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:54.380855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7a3d6-acb0-4bfc-aef1-c2242ac34d75 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:54.435472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519efc9b-aa44-405a-994d-a9c3605a9c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:54.436594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efc9b-aa44-405a-994d-a9c3605a9c02 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:54.437125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efc9b-aa44-405a-994d-a9c3605a9c02 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:54.437599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efc9b-aa44-405a-994d-a9c3605a9c02 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:54.438470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519efc9b-aa44-405a-994d-a9c3605a9c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:54.438979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519efc9b-aa44-405a-994d-a9c3605a9c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:54.440568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519efc9b-aa44-405a-994d-a9c3605a9c02 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:54.459164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 267/547] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:54.466227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:54.595650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:54.643034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436705b7-714d-4520-9871-6b8309ad2a62 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:54.669577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e86772-3c93-4f11-95c3-278c7e9b2076 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:54.670218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e86772-3c93-4f11-95c3-278c7e9b2076 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:54.670218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e86772-3c93-4f11-95c3-278c7e9b2076 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:54.671452 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 268/548] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:39:54.686335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e9a41-b74d-4be7-b83e-808ca94b71e9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:54.686741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:54.687254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e9a41-b74d-4be7-b83e-808ca94b71e9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:54.687360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:54.687640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:54.687904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] PATH_INFO: `/v3/users/swift` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:54.688230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e9a41-b74d-4be7-b83e-808ca94b71e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:54.698698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:54.700230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:54.714474 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:54.715480 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:54.716395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 281/549] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:39:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:54.721735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:54.724100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-204e9a41-b74d-4be7-b83e-808ca94b71e9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:54.725700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-204e9a41-b74d-4be7-b83e-808ca94b71e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:54.752409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:54.859286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee491c-d7cb-4f38-aeb6-1244ec7c7918 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:54.866992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 269/550] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:54.896795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:54.897315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:54.897597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:54.897868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:54.898610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:54.898917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:54.900366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:54.902230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:54.922512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 282/551] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 27 09:39:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:39:54.927984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e9a41-b74d-4be7-b83e-808ca94b71e9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:55.132110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e9a41-b74d-4be7-b83e-808ca94b71e9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:55.213883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:55.214726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:55.215151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:55.215572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:55.255370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:55.308492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:55.370077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:55.372559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:55.511265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8a52-6521-47c9-baeb-e8045187c7d6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:55.558760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:55.559427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:55.559706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:55.559976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:55.585163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:55.606138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:55.607920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:55.609686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:55.612097 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 283/552] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:55.622591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:55.658139 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 270/553] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:39:55.684144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:55.803994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:55.882105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:56.073253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691025-6dfa-436d-bf0e-029dccf3705c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:56.143407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fca10-c434-4816-ad11-721ffd4631aa None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:56.151042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce017020-d192-46f9-8fd0-19a94aa4332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:56.152047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce017020-d192-46f9-8fd0-19a94aa4332e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.152504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce017020-d192-46f9-8fd0-19a94aa4332e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.153018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce017020-d192-46f9-8fd0-19a94aa4332e None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.154170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce017020-d192-46f9-8fd0-19a94aa4332e None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:56.154682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce017020-d192-46f9-8fd0-19a94aa4332e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:56.157070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce017020-d192-46f9-8fd0-19a94aa4332e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:56.177442 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 284/554] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 551 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:56.177442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f87e0-0aec-4031-9a2f-4626689cfa06 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.177442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f87e0-0aec-4031-9a2f-4626689cfa06 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.177442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f87e0-0aec-4031-9a2f-4626689cfa06 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.179314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 285/555] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:56.181649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3f94-d848-42d0-98cc-f86f7e024de8 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.182332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3f94-d848-42d0-98cc-f86f7e024de8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.182941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3f94-d848-42d0-98cc-f86f7e024de8 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.185452 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 286/556] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:56.187026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea3f94-d848-42d0-98cc-f86f7e024de8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:56.225618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094caf2-5fc2-41ec-994c-4cac0a1ea762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:56.227109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094caf2-5fc2-41ec-994c-4cac0a1ea762 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.227852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094caf2-5fc2-41ec-994c-4cac0a1ea762 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.228310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094caf2-5fc2-41ec-994c-4cac0a1ea762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.254115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094caf2-5fc2-41ec-994c-4cac0a1ea762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:56.257669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094caf2-5fc2-41ec-994c-4cac0a1ea762 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d094caf2-5fc2-41ec-994c-4cac0a1ea762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:56.279902 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:56.281746 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:56.283411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:56.283411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:56.283411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:56.283411 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 271/557] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 599 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:39:56.283841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fed3e8-0465-47de-a44d-9dc3c61c63af None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.284570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fed3e8-0465-47de-a44d-9dc3c61c63af None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.286324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fed3e8-0465-47de-a44d-9dc3c61c63af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.316416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-47fed3e8-0465-47de-a44d-9dc3c61c63af None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:56.318319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-47fed3e8-0465-47de-a44d-9dc3c61c63af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:56.376896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea3f94-d848-42d0-98cc-f86f7e024de8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:56.400777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:56.432708 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 272/558] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:56.432708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a8875-460c-4d56-8948-71cbca6ec0c3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.432708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a8875-460c-4d56-8948-71cbca6ec0c3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.432708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a8875-460c-4d56-8948-71cbca6ec0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.436157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:56.436990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.437388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.437800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.456744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:56.457202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:56.461833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-948a8875-460c-4d56-8948-71cbca6ec0c3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:56.468691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-948a8875-460c-4d56-8948-71cbca6ec0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:56.489605 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:56.491413 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:56.493164 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:56.493164 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:56.493164 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:56.494928 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 287/559] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:56.494928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:56.512144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:56.587882 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 273/560] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:56.590717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0b5ec-f31d-4193-861a-fd96a9dfc7fb None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.591280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0b5ec-f31d-4193-861a-fd96a9dfc7fb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.591836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0b5ec-f31d-4193-861a-fd96a9dfc7fb None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.612703 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 274/561] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:39:56 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:39:56.615521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0b5ec-f31d-4193-861a-fd96a9dfc7fb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:56.718816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f2971-0b8a-49cd-984f-4d8c2671e6aa None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:56.785812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e00aa-b016-4cb9-b2ae-1f7ed30a4551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:56.786866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e00aa-b016-4cb9-b2ae-1f7ed30a4551 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.787294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e00aa-b016-4cb9-b2ae-1f7ed30a4551 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.787758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e00aa-b016-4cb9-b2ae-1f7ed30a4551 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.788940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e00aa-b016-4cb9-b2ae-1f7ed30a4551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:56.789479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e00aa-b016-4cb9-b2ae-1f7ed30a4551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:56.793080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e00aa-b016-4cb9-b2ae-1f7ed30a4551 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:56.819883 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 288/562] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:56.819883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cd93d-865b-4f73-9a19-619feac6a0c6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.819883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cd93d-865b-4f73-9a19-619feac6a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.819883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cd93d-865b-4f73-9a19-619feac6a0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.852107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-449cd93d-865b-4f73-9a19-619feac6a0c6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:39:56.854235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-449cd93d-865b-4f73-9a19-619feac6a0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:39:56.894319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0b5ec-f31d-4193-861a-fd96a9dfc7fb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:56.897463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:56.984176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:56.984485 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 289/563] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:39:56.985168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:56.985605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:56.985763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cd93d-865b-4f73-9a19-619feac6a0c6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:56.986046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:56.986830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:56.987190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:56.988751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:57.016693 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 275/564] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:57.016693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.045304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:57.140454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cd93d-865b-4f73-9a19-619feac6a0c6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.185042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.186026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.186467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.186905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.196100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:57.196331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b52a0-2c72-4944-9ecd-5d800e3a3d63 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.197879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:39:57.211679 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:57.213750 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:57.215569 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:57.219942 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 290/565] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:57.221249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.243334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:57.266676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.267760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.268252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.268775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.289308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:57.300244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:57.301872 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:57.303097 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 276/566] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:57.306646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.386316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b6e57-a6c8-40b6-813c-1609da29d8fc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.439364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.440451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.440954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.441424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.451453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:57.455777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:57.465254 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:57.466487 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:57.467726 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 291/567] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:57.469152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.489363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee9435-b0da-4817-b0de-535ee02e1f2f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.543598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.544229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.544518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.544893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.553585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:57.555338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:57.568001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:57.570183 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 277/568] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:57.571530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.600936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:57.641645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfdd2f-b645-4dbf-aa59-16c612c3df17 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.683969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.685204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.685709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.686160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.687399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:57.687945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:57.690608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:57.691029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71cfb-f150-49f5-a22d-e0c431efbdae None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.717843 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 292/569] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:57.717843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.730900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.731534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.731852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.732088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.732829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:57.733141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:57.734611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:57.746519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 278/570] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:57.747175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.750296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:57.885829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfd1e-0c60-448c-ac2d-115e711d4187 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.897353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c7ee2-cef5-4d10-ab63-5a2c23dcc98b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:57.931422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.932314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.932806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.933264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.937285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:57.947050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:57.948003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:57.948446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:57.948935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:57.950000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:57.950486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:57.950687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:39:57.952917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:39:57.960228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:57.961283 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:57.962989 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 293/571] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:57.964592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:57.968975 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 279/572] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:57.970488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:58.043025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:58.094146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a83d66-ee46-406d-a9f3-89ca7d77bde3 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.108371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349d2ee-1381-4c0e-8b3a-7be16874c5eb None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.137834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:58.138372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:58.138622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:58.138896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:58.139629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:58.139956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:58.141438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:58.162884 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 294/573] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:58.163753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:58.168130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:58.169102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:58.169591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:58.170076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:58.171394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=c6863dfc2c0143a88060bbf5f0e146df) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:58.171932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:58.174495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:58.185143 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 280/574] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:39:57 2022] GET /identity/v3/role_assignments?user.id=c6863dfc2c0143a88060bbf5f0e146df&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:58.186587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:58.250299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:58.331150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409fe9c-55ea-4080-aa40-a911eea0911e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.381216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99836177-f266-4d5f-a5a5-2e1df705312e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.384090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:58.384924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:58.385262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:58.385603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:58.401972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:58.404121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:58.424212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:58.426629 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:58.428315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:58.428315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:58.428315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:58.432950 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 295/575] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:58.434850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:58.437619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:58.454976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:58.456175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:58.456898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:58.457494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:58.458733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:58.459318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:58.461896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:58.476051 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 281/576] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:39:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:58.477353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:58.599515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d65fb2-b5dc-4b37-9473-1c5be54b587c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.634304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8be5-b5b3-4eeb-978e-504c324d86d6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.642688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:58.665655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:58.666195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:58.666486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:58.666763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:58.680066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:58.681597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:58.688375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:58.689426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:58.689899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:58.690393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:58.700042 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:58.701979 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:58.703721 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:58.703721 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:58.703721 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:58.704308 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 282/577] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:58.705894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:58.713813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:58.715766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:58.742638 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:58.744580 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:58.746092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:58.746092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:58.746092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:58.755693 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 296/578] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:58.755693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:58.796202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:58.887771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d567547-d486-4575-ac1d-0676ea4ec4da None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.930817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:58.931727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:58.931727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:58.931992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:58.932705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:58.933027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:58.934439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:58.946879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1d9fe-0717-496c-a800-0796a55dac6b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:58.949609 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 283/579] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:58.951123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.006735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.007384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.007661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.007930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.020055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.021791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:39:59.044264 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:59.045748 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:39:59.046960 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:39:59.046960 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:39:59.046960 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:59.046960 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 297/580] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:59.047418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.064723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:59.094695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b17d7f-9ea3-4f5b-8afe-c80e1a96ea9a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:59.096464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:59.143746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.145028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.145468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.145892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.146949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:59.147432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.149663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.166995 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 284/581] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:59.167647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.228169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120f949-8436-4acd-9933-97f9d27a1a2d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:59.284219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.285110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.285758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.285995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.286772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:59.287265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.288517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.303820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 298/582] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:59.303820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.311257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee788a3c-e1f3-4f85-8b43-726844b87538 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:59.373055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.373688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.373979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.374252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.383500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.385117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.396788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:59.399716 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:59.402651 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 285/583] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:59.404495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.434072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:59.445936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d2c1d-afdf-4903-93b1-c269895a7847 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:59.508715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.509686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.510138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.510619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.511783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:59.512272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.514939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.538610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 299/584] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:59.539569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.550235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c295-4660-48bf-a37f-9cfd7ab06a1e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:59.596881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.597856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.598261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.598624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.614675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.616152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:39:59.631629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:59.632777 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:39:59.633685 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 286/585] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:59.634193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.711346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3bc98-184e-4386-929f-f5e492836d70 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:59.765240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:59.772570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.773320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.773671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.774020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.778489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:39:59.783728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.786409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.803566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e97f7-d372-4573-a1fc-54bf3971eb24 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:39:59.807928 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:39:59.809025 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:39:59.810092 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:39:59.818079 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 300/586] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:39:59.818079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.850606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:39:59.851522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:39:59.851808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:39:59.852185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:39:59.853095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:39:59.853404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:39:59.855050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:39:59.876273 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 287/587] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:39:59.881581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:39:59.978974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546d112-e8a3-420e-941e-f68d73e0bead None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.002545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de322a-e92d-4146-a147-b9fe96e7ec68 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.021367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.022740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.023358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.023698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] PATH_INFO: `/v3/users/glance` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.033861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.035949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:00.049575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:00.058729 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:00.059845 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 301/588] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:00.060695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.060800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.060881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.060881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.060881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.060881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:00.060881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.060881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.070761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:00.077758 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 288/589] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:39:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:00.080018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.130944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:00.197203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd81274-55ee-4086-a027-d36ff2b5a930 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.228749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822956f-a25d-4ed0-830a-d984b1c2f0c3 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.243181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.244214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.244644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.245194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.246445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:00.246984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.249771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.270959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.276725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 302/590] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:40:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:00.276862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.276862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.277015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.277104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.295430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.297063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:00.314943 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:00.320725 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:00.321831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:00.321831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:00.321831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:00.321831 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 289/591] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:00.321831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.397586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:00.427542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209bf75-8260-434d-867e-e3333f40735f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.438538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:00.470577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.471031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d94fa-decf-46dd-b87c-47513fadb274 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.471216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.471484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.471733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.473129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:00.473453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.474955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.503380 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 303/592] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 27 09:40:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:00.503380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.520126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.521034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.521372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.521711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/065ad0a36b32453c98ca3ecda6f952a7/roles/d2b916599fc849669bff95d9ac632b79` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.545588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=065ad0a36b32453c98ca3ecda6f952a7, role_id=d2b916599fc849669bff95d9ac632b79)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.547681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.572696 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 290/593] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:00 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/065ad0a36b32453c98ca3ecda6f952a7/roles/d2b916599fc849669bff95d9ac632b79 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:40:00.572696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.655486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150dfb-eea4-429f-8f5d-fd392ca26844 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.709788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.710702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.711117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.711584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.728755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.730545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.754863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be96d3a-81ee-492f-8abc-197632f3fa9f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:00.755620 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:00.756689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:00.757604 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:00.757604 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:00.757604 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:00.760009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:00.766081 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 304/594] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:00.767667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.813800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:00.814718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.815553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.815969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.816673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:00.817592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:00.818080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:00.821031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:00.838763 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 291/595] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:00.839884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:00.950829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea490a-800d-4f0a-a399-2de8ef015558 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.985813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41e2d7-9f9a-4ba8-98ae-301e45ef7ccc None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:00.991771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:00.992428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:00.992734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:00.993008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.006046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.007531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:01.027703 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:01.028766 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:01.029694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:01.029694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:01.029694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:01.029878 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 305/596] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:01.031141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.049405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86305d6-8e3c-4fb9-8f85-d37c4bdb8321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.050062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86305d6-8e3c-4fb9-8f85-d37c4bdb8321 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.050462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86305d6-8e3c-4fb9-8f85-d37c4bdb8321 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.050841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86305d6-8e3c-4fb9-8f85-d37c4bdb8321 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.051610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86305d6-8e3c-4fb9-8f85-d37c4bdb8321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:01.052399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86305d6-8e3c-4fb9-8f85-d37c4bdb8321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.053615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86305d6-8e3c-4fb9-8f85-d37c4bdb8321 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.069722 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 292/597] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:01.070659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080038a0-5430-4efa-8ef4-6e666e46e7a7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.070941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080038a0-5430-4efa-8ef4-6e666e46e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.071229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080038a0-5430-4efa-8ef4-6e666e46e7a7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.072824 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 293/598] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:01.077505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080038a0-5430-4efa-8ef4-6e666e46e7a7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.111959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:01.153873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:01.198839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcab96a-585b-4530-aef6-38390218ebb5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:01.225932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080038a0-5430-4efa-8ef4-6e666e46e7a7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:01.242674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.243660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.244117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.244684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.254528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.256233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:01.269558 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:01.270856 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:01.271886 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:01.278546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 306/599] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:01.280061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.287583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7994-92cf-4dd0-8399-49b03b326bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.288526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7994-92cf-4dd0-8399-49b03b326bce None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.289076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7994-92cf-4dd0-8399-49b03b326bce None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.289574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7994-92cf-4dd0-8399-49b03b326bce None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.290760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7994-92cf-4dd0-8399-49b03b326bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:01.291301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7994-92cf-4dd0-8399-49b03b326bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.294001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7994-92cf-4dd0-8399-49b03b326bce None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.310193 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 294/600] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:01.311264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa87d-154b-4c19-add8-4b9c4fef6ec7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.311558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa87d-154b-4c19-add8-4b9c4fef6ec7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.311838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa87d-154b-4c19-add8-4b9c4fef6ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.330981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b4faa87d-154b-4c19-add8-4b9c4fef6ec7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:01.332216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4faa87d-154b-4c19-add8-4b9c4fef6ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:01.422646 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 295/601] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:01.423418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa87d-154b-4c19-add8-4b9c4fef6ec7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.442255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:01.469184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a511400-3084-455f-88c9-bcd18ca076e5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:01.510828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:01.519667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.520606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.521074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.521540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.531987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.533766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:01.550962 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:01.551948 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:01.552869 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:01.555640 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 307/602] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:01.556499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.568648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa87d-154b-4c19-add8-4b9c4fef6ec7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:01.615641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.617220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.617751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.618251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.619617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:01.620521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.622508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.646321 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 296/603] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:01.647053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.703488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a26be3-8dcf-4e24-b7c4-0a7629fc98d2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:01.753246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.755880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:01.756913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.757256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.757675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.768138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.770284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:01.787920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:01.790931 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 308/604] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:01.792607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.810969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e682e-4118-4e11-b090-0bfcd96a8927 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:01.866023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.866574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.866835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.867108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:01.867809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:01.868109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:01.868790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:01.869805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:01.894006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 297/605] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:01.895573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:01.934211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2f1c8-5a25-4cf0-96a6-e17f86771a30 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:01.990158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:01.990946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:01.991234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:01.991486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/b324af05ec3e4b35a9bfc188b3b3abec/roles/d2b916599fc849669bff95d9ac632b79` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.010983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=b324af05ec3e4b35a9bfc188b3b3abec, role_id=d2b916599fc849669bff95d9ac632b79)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:02.012475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:02.028863 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 309/606] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:01 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/b324af05ec3e4b35a9bfc188b3b3abec/roles/d2b916599fc849669bff95d9ac632b79 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:40:02.030024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:02.068236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196eff-87c1-4513-a9d0-cb8d679651a8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:02.138378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:02.150020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:02.150500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:02.150854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:02.151261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.168925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:02.174288 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:02.175993 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:02.178317 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:02.181367 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 298/607] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:02.182675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:02.202223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:02.208579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be701a-5f42-4773-a83c-96e63b9a6203 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:02.281504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:02.282645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:02.283041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:02.283532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.284538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:02.285089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:02.287449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:02.302465 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 310/608] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:02.303391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:02.322650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673816f0-a0f0-44fe-99f9-ca05060294e5 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:02.374268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:02.375436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:02.375916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:02.376381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/3c8ed3cf7a5742808029d336ebc784fa/roles/d2b916599fc849669bff95d9ac632b79` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.396515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=3c8ed3cf7a5742808029d336ebc784fa, role_id=d2b916599fc849669bff95d9ac632b79)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:02.398502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:02.421808 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 299/609] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:02 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/3c8ed3cf7a5742808029d336ebc784fa/roles/d2b916599fc849669bff95d9ac632b79 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:40:02.422720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:02.441526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6ba23-9779-41a7-9953-eff720d82cd5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:02.483187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:02.508575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:02.509939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:02.510702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:02.511172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.512046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:02.512542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:02.514439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:02.527962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 311/610] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:02.536274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:02.572185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:02.592242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a6d6-7dfa-41d2-adda-ae340c72d93c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:02.678666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:02.679678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:02.679990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:02.680486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/ed91b2c83d57419dac50595f57fc3411/roles/d2b916599fc849669bff95d9ac632b79` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.682804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5257dd-8a36-438a-a1f9-01709ea218d0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:02.707804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=ed91b2c83d57419dac50595f57fc3411, role_id=d2b916599fc849669bff95d9ac632b79)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:02.710406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:02.730652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:02.731309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:02.731598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:02.731718 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 300/611] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:02 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/ed91b2c83d57419dac50595f57fc3411/roles/d2b916599fc849669bff95d9ac632b79 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:40:02.731914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.747544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:02.749227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:02.771567 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:02.772690 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:02.773665 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:02.773665 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:02.773665 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:02.775750 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 312/612] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:02.781604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:02.914224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:02.932150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3a377-33dd-4a4e-b45d-9be0d59bf220 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:02.974769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:02.975412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:02.975693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:02.975974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:02.976677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:02.976995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:02.978470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:02.996371 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 301/613] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:03.001965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:03.100600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:03.136750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849705d5-fbbf-4479-9df0-58d9e852e046 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:03.190159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f0645-9d05-4f12-ad6b-9897bcb9f6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:03.190876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0645-9d05-4f12-ad6b-9897bcb9f6bd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:03.191323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0645-9d05-4f12-ad6b-9897bcb9f6bd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:03.191737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0645-9d05-4f12-ad6b-9897bcb9f6bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:03.205998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f0645-9d05-4f12-ad6b-9897bcb9f6bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:03.207811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f0645-9d05-4f12-ad6b-9897bcb9f6bd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-cf4f0645-9d05-4f12-ad6b-9897bcb9f6bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:03.225870 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:03.226896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:03.227834 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 313/614] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:03.233187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:03.331409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb505050-9d36-41ea-997e-dee8af61b568 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:03.331752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb505050-9d36-41ea-997e-dee8af61b568 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:03.332061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb505050-9d36-41ea-997e-dee8af61b568 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:03.333467 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 314/615] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:40:03.350746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd879e3-2416-40d8-b13a-cad5e6d7a8ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:03.351063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd879e3-2416-40d8-b13a-cad5e6d7a8ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:03.351295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd879e3-2416-40d8-b13a-cad5e6d7a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:03.375893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cab55-9cf5-4670-8666-c4ccddaddcd2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:03.393829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4dd879e3-2416-40d8-b13a-cad5e6d7a8ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:03.395117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4dd879e3-2416-40d8-b13a-cad5e6d7a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:03.419990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:03.420968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:03.421431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:03.422244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:03.423198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:03.423963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:03.425694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:03.450005 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 302/616] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 27 09:40:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:03.457124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:03.473970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:03.575219 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 315/617] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:03.626526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddcc76-e38d-49e7-a8b9-85f058174802 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:03.674256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:03.674929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:03.675207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:03.675484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:03.696592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:03.698147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:03.718533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:03.720540 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:03.722929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:03.722929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:03.722929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:03.722929 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 303/618] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:03.726124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd879e3-2416-40d8-b13a-cad5e6d7a8ee None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:03.825273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:03.865836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd879e3-2416-40d8-b13a-cad5e6d7a8ee None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:03.908148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:03.908880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:03.909147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:03.909428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:03.910265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:03.910956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:03.912559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:03.928314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 316/619] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:03.934685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:04.102305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:04.191487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e7eb4-cfbb-403b-8c84-00770b81f211 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:04.250551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:04.251290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:04.251509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:04.251779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:04.266002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:04.270390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:04.273095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:04.291938 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:04.292999 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:04.293908 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:04.301930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 304/620] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:04.306371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:04.530938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eb114-a6ae-447b-9aa3-712d41740867 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:04.617104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:04.623528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b66fbb-ca0b-43a8-b6ee-4b05c6cbec0e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:04.647128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:04.648236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:04.648766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:04.649241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:04.670723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:04.688931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:04.694895 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:04.696744 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:04.698404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:04.706087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:04.706739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:04.707265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:04.707651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:04.708398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:04.708719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:04.710358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:04.711691 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 317/621] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:04.714922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:04.744107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 305/622] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:04.746615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:05.040740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53689e2a-00fa-4683-a39e-33ae953f4fc4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:05.049360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79f7e7-5087-406f-a876-6d0f93331813 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:05.114975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:05.115659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:05.115939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:05.116212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:05.117160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:05.117514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:05.117898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:05.118908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:05.119598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:05.120520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:05.121372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/c6863dfc2c0143a88060bbf5f0e146df/roles/d2b916599fc849669bff95d9ac632b79` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:05.145765 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 318/623] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:05.155925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:05.175746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=c6863dfc2c0143a88060bbf5f0e146df, role_id=d2b916599fc849669bff95d9ac632b79)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:05.178751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:05.229020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 306/624] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:04 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/c6863dfc2c0143a88060bbf5f0e146df/roles/d2b916599fc849669bff95d9ac632b79 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:40:05.283531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:05.408539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef06dbd-2675-4549-9dbc-4654ea00250f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:05.496307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fad3f3-0574-4e81-937c-06053cdbde5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:05.497945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad3f3-0574-4e81-937c-06053cdbde5b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:05.498743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad3f3-0574-4e81-937c-06053cdbde5b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:05.499513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad3f3-0574-4e81-937c-06053cdbde5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:05.531604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fad3f3-0574-4e81-937c-06053cdbde5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:05.534818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fad3f3-0574-4e81-937c-06053cdbde5b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-01fad3f3-0574-4e81-937c-06053cdbde5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:05.576691 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:05.578497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:05.580215 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:05.580215 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:05.580215 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:05.580215 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 319/625] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:05.583176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:05.658104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:05.824054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eb1cc-21db-4ce5-9316-6ec4665cb399 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:05.824407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eb1cc-21db-4ce5-9316-6ec4665cb399 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:05.824612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eb1cc-21db-4ce5-9316-6ec4665cb399 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:05.826433 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 320/626] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:05.827070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1686-adb7-4453-95be-f12615347f0c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:05.832307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f97686-53fb-4598-a460-4dc749eab129 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:05.832624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f97686-53fb-4598-a460-4dc749eab129 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:05.832901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f97686-53fb-4598-a460-4dc749eab129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:05.866966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b4f97686-53fb-4598-a460-4dc749eab129 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:05.868959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4f97686-53fb-4598-a460-4dc749eab129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:05.896649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:05.897696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:05.898172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:05.898637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:05.899495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:05.900005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:05.901642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:05.922436 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 307/627] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:05.929687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:06.024341 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 321/628] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:40:06.108002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e78eb-4b77-4ec3-a7d8-1ed4ebfb3b07 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.108381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e78eb-4b77-4ec3-a7d8-1ed4ebfb3b07 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.108890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e78eb-4b77-4ec3-a7d8-1ed4ebfb3b07 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.113487 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 322/629] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:06.119260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:06.121343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7ba92-8282-48f7-86d7-a4cf5a75e71c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.121835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7ba92-8282-48f7-86d7-a4cf5a75e71c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.122305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7ba92-8282-48f7-86d7-a4cf5a75e71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.143161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89135e34-0816-41b7-872f-9a16d903e6a8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:06.154221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-84f7ba92-8282-48f7-86d7-a4cf5a75e71c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:06.156209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84f7ba92-8282-48f7-86d7-a4cf5a75e71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:06.204694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:06.257926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0009d3-bfab-46a2-8e15-5005049b7c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:06.259051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0009d3-bfab-46a2-8e15-5005049b7c04 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.259527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0009d3-bfab-46a2-8e15-5005049b7c04 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.259994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0009d3-bfab-46a2-8e15-5005049b7c04 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.284819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0009d3-bfab-46a2-8e15-5005049b7c04 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:06.293649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0009d3-bfab-46a2-8e15-5005049b7c04 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:06.314985 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 323/630] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-af0009d3-bfab-46a2-8e15-5005049b7c04 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:06.341935 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:06.342976 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 09:40:06.343926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:06.345024 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 308/631] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:06.353422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7ba92-8282-48f7-86d7-a4cf5a75e71c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:06.354843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbe97-397a-4988-a9cc-d9127229aaa6 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.355285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbe97-397a-4988-a9cc-d9127229aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.355561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbe97-397a-4988-a9cc-d9127229aaa6 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.357583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 309/632] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:06.369108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bfd36-832b-4ce3-b720-f44580abe11f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.369416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bfd36-832b-4ce3-b720-f44580abe11f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.369574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bfd36-832b-4ce3-b720-f44580abe11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.400260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c09bfd36-832b-4ce3-b720-f44580abe11f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:06.401905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c09bfd36-832b-4ce3-b720-f44580abe11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:06.481879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7ba92-8282-48f7-86d7-a4cf5a75e71c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:06.496779 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 310/633] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:06.497926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bfd36-832b-4ce3-b720-f44580abe11f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:06.520649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:06.521690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.522173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.522616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.523593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:06.524154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:06.526165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:06.548332 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 324/634] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Sep 27 09:40:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:06.554990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:06.592614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:06.721330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:06.755010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bfd36-832b-4ce3-b720-f44580abe11f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:06.762742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecee172-d589-4c3d-a698-4c15ef0e0863 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:06.805358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:06.806162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.806533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.806899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.878920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:06.880082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:06.880082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:06.880373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:06.884916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:06.886508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:06.892333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:06.897941 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:06.900106 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:06.902087 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 311/635] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:06.916163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:06.917659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.003964 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 325/636] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 09:40:07.005054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.084722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab633b-30b4-4670-a9ba-9b3854ed19a0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.125402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.126787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.127464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.128136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.130517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:07.131337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:07.134163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:07.142518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55969d-9b38-4138-b886-77dc57cf4f69 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.151015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 312/637] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:07.152999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.161150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:07.167449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:07.189807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.190478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.190771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.191040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.204613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:07.210028 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:07.211769 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 326/638] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:07.213772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.331996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791074fe-4f21-4ed0-89fc-7fb14e0ad1f6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.349269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e7ae-921a-4ae5-b93d-36e3f2d4c710 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.389161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.390260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.390707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.391146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.406354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:07.406619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.407273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.407554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.407821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.408546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:07.423820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:07.432712 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:07.434538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 327/639] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:07.439568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:07.440345 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:07.441322 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:07.442245 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:07.442245 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:07.442245 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:07.442245 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 313/640] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:07.442245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.498579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:07.539199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:07.593013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e8d51-033b-427b-9813-1026c976de3c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.619880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83850e34-c2ad-4dc5-9440-a7bfeb1b39b2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.643953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.645613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.646265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.646818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.661038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:07.662335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.662650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:07.663500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.663924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.664337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.665197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:07.665676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:07.667342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:07.677325 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 314/641] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:07.678497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:07.681448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:07.682568 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:07.683704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:07.683704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:07.683704 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:07.683704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 328/642] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:07.683704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.811249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37da66e-e4ba-471b-8402-d1bce6524a1e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.818420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:07.836325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:07.839869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9318ed-6113-4eda-967d-e59eecfdaac6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:07.866465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.867341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.867781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.868270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.869310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:07.869795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:07.871428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:07.886594 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 315/643] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:07.886594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:07.892321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:07.893237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:07.893698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:07.894160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:07.895397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:07.895819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:07.897975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:07.919891 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 329/644] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:07.921239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.033403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648847a-51ba-4a70-b07e-26e27f116437 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.082836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.083734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.084170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.084619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.100636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.102543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.108987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ccb3e-f13a-4b40-863b-aa7298f6871e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:08.121173 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:08.123027 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:08.125078 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:08.125078 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:08.125078 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:08.125078 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 316/645] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:08.126472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.166935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.168081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.168491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.168949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.169774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:08.170222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.171853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.194959 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 330/646] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:08.194959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.200380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:08.208752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:08.321973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25f42-21a8-4618-a594-1c7d5db92cd7 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.349137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e7b60-cb23-41cf-92bc-3c0b9b13541e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.380027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.380721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.381005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.381279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.390142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.390953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.391090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.391449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.394891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.396913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.400242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.401940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:08.414484 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:08.415527 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:08.421939 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:08.421939 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:08.421939 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:08.422115 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:08.422947 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:08.423938 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:08.424370 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 317/647] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:08.424370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.424506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 331/648] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:08.424506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.503203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:08.548904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bffc0-b3b9-4ad1-a98e-7911710e3a86 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.549204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:08.581131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424613de-e10a-4d11-a5a7-4a96c4d80f51 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.596690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.597880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.598151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.598417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.610596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.612142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:08.630305 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:08.632324 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:08.634272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:08.636083 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 332/649] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:08.637052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.645535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.646165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.646438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.647372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.648053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:08.648348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.650248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.672512 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 318/650] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:08.672512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.802760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d67c5f-08e0-4994-b12a-ced116bd507b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.829344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed36b0-f59b-4731-9563-8a94cc2729ba None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:08.851293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:08.867565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.868482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.868837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.869052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.869723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:08.870025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.871810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.871976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:08.872449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:08.872749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:08.873017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:08.873714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:08.874019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:08.875467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:08.889766 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 319/651] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:08.896330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 333/652] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:08.896330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.896704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:08.900487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:09.025818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28139d-c950-40ee-8af7-05a6944f720b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:09.037085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa64975-72a7-4ee5-a9fd-eea5ba416497 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:09.084016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:09.085036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.085647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.086147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.087747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:09.088053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:09.090563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:09.100440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:09.101593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.102046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.102497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] PATH_INFO: `/v3/users/glance` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.117032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:09.119795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:09.123747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 320/653] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:09.125279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:09.147362 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:09.148547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:09.149797 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 334/654] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:09.150956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:09.217346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:09.232225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:09.280483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1bf0f-c6e7-4992-b6ca-3b1e7f0b11e9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:09.309035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd0499-616d-48ee-8748-fe1ef96cf548 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:09.322227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cc8f5-474c-4a89-ab0b-cc1ac4a8c751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:09.322864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cc8f5-474c-4a89-ab0b-cc1ac4a8c751 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.323141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cc8f5-474c-4a89-ab0b-cc1ac4a8c751 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.323412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cc8f5-474c-4a89-ab0b-cc1ac4a8c751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.324186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cc8f5-474c-4a89-ab0b-cc1ac4a8c751 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=065ad0a36b32453c98ca3ecda6f952a7) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:09.324481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cc8f5-474c-4a89-ab0b-cc1ac4a8c751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:09.326074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cc8f5-474c-4a89-ab0b-cc1ac4a8c751 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:09.333637 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 321/655] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:09 2022] GET /identity/v3/role_assignments?user.id=065ad0a36b32453c98ca3ecda6f952a7&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:09.335028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0856134-02c9-4fb5-a3b5-225373a976e1 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.335313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0856134-02c9-4fb5-a3b5-225373a976e1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.335583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0856134-02c9-4fb5-a3b5-225373a976e1 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.337001 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 322/656] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:09.337853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0856134-02c9-4fb5-a3b5-225373a976e1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:09.361796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:09.362470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.362746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.363049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.375785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:09.378707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:09.403718 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:09.406581 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:09.409448 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 335/657] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:09.411432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:09.512544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0856134-02c9-4fb5-a3b5-225373a976e1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:09.569076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:09.582908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:09.583618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.583958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.584193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.587474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556e95d-a182-4e05-bdd9-cb46508e8da3 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:09.612956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:09.654846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ea483-a6d9-4bfc-8354-f2e417350fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:09.655738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ea483-a6d9-4bfc-8354-f2e417350fec None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.656213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ea483-a6d9-4bfc-8354-f2e417350fec None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.656741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ea483-a6d9-4bfc-8354-f2e417350fec None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.658134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ea483-a6d9-4bfc-8354-f2e417350fec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:09.659211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ea483-a6d9-4bfc-8354-f2e417350fec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:09.663072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ea483-a6d9-4bfc-8354-f2e417350fec None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:09.699044 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 336/658] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 27 09:40:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:09.700942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161d52-a162-44da-a4e1-96fcd944677a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:09.701363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161d52-a162-44da-a4e1-96fcd944677a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:09.701769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161d52-a162-44da-a4e1-96fcd944677a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:09.707666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:09.709398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:09.726352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-69161d52-a162-44da-a4e1-96fcd944677a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:09.727566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69161d52-a162-44da-a4e1-96fcd944677a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:09.861053 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 337/659] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:09.862479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69161d52-a162-44da-a4e1-96fcd944677a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:09.891449 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 323/660] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:40:09.893175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:09.954872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:10.053277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:10.065613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69161d52-a162-44da-a4e1-96fcd944677a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:10.116944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9153a-c73c-4870-9e97-ef7d245ba932 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:10.137275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:10.138482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:10.139063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:10.140142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:10.141178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:10.141829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:10.144828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:10.179603 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 338/661] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Sep 27 09:40:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:10.181400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:10.191467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:10.192963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:10.193656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:10.194337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:10.337254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:10.340408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:10.365696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bae2fd-07ab-4aee-bb10-9398f84dbb6f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:10.390431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:10.428282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:10.429890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:10.430431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:10.431003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:10.451008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:10.453079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:10.473658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:10.475749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:10.477891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:10.477891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:10.477891 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:10.478270 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 324/662] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:40:10.478270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:10.483999 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 339/663] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:10.483999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:10.493971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:10.638683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27d241-59c0-4940-980b-cab82c5498d2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:10.676092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e9ce-d81c-46c5-b05c-049698490538 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:10.697947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:10.698737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:10.699129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:10.699554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:10.718187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:10.724700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:10.726819 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:10.728545 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:10.730653 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:10.734308 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 340/664] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:10.735701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:10.748107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:10.749090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:10.749528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:10.750616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:10.858821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:10.885524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:10.888103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:10.915203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eed1ad-24ea-415a-b551-f49c8930e7eb None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:10.961182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:10.962328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:10.962769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:10.963159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:10.980860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:10.983240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:11.008144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:11.010192 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:11.012538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:11.012538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:11.012538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:11.012538 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 341/665] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:11.012538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.053250 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 325/666] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:40:11.054279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.098853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:11.193098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb80269-d062-44c0-9231-da1134a0c3a2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:11.200689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fb04-38bd-4f4f-b8c0-71c9a88e9363 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:11.235882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:11.239158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:11.240343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:11.240811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:11.241237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:11.242933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:11.243404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:11.245849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:11.268137 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 326/667] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:11.268137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.272572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:11.273553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:11.273985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:11.274522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:11.275522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:11.275986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:11.278388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:11.305745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 342/668] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:11.305745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.429493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:11.452073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653e218-c7a1-4da7-991c-d80148cb2e75 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:11.456356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c51a11-e14d-43bf-b024-33dec7e56b07 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:11.495026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:11.495674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:11.495960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:11.496211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:11.508905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:11.510409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:11.524526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:11.526038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:11.526688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:11.527421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:11.528269 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:11.530249 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:11.532072 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:11.532072 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:11.532072 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:11.532072 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 343/669] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:11.532072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.533651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:11.534189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:11.537652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:11.561093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 327/670] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:11.562635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.575024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:11.713830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebedac8-e4ce-4c29-835a-36e344695d97 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:11.717800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc56dd-f50c-4c34-93de-445bd3080932 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:11.755344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:11.755986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:11.756259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:11.756530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:11.765054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:11.766540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:11.778927 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:11.781224 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 328/671] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:11.783043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.792766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:11.793839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:11.794331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:11.794809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:11.798328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:11.816132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:11.818947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:11.852315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:11.853511 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:11.854670 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:11.854670 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:11.854670 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:11.854670 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 344/672] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:11.856358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:11.961201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12943b-b5e3-4609-bdfc-11319e65f03c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:11.996583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:12.024474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76184750-458a-45a4-b05d-02364348b797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.025558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76184750-458a-45a4-b05d-02364348b797 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.026036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76184750-458a-45a4-b05d-02364348b797 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.026516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76184750-458a-45a4-b05d-02364348b797 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.027699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76184750-458a-45a4-b05d-02364348b797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:12.028251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76184750-458a-45a4-b05d-02364348b797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.030860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76184750-458a-45a4-b05d-02364348b797 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.032294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3257be-58e9-4b3c-8961-fb2074a53b76 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.057812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 329/673] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:12.057812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76184750-458a-45a4-b05d-02364348b797 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.094757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.095402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.096143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.096143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.104785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.106249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:12.133212 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:12.135039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 345/674] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:12.137776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.152187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:12.193993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76184750-458a-45a4-b05d-02364348b797 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.234988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.236198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.236815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.237788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.238797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:12.239438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.242210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.269988 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 330/675] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:12.269988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.331551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa12252-b38d-4141-b41f-172c60fb3230 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.372832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.373447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.373715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.373977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.374616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:12.374907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.376264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.389548 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 346/676] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:12.390397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.393500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:12.431431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eaa108-8c27-4e43-bfa9-97aeb856d6e6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.461668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:12.474534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.475440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.475834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.476219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.488839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.490965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:12.506148 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:12.508019 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 331/677] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:12.510000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.522083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0233719-f5d4-4cf7-aa2b-4e11efb618d6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.560571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb13db-6d1a-4d1b-9a33-bb6e23cfca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.561218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb13db-6d1a-4d1b-9a33-bb6e23cfca0a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.561493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb13db-6d1a-4d1b-9a33-bb6e23cfca0a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.561767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb13db-6d1a-4d1b-9a33-bb6e23cfca0a None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.562469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb13db-6d1a-4d1b-9a33-bb6e23cfca0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:12.562832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb13db-6d1a-4d1b-9a33-bb6e23cfca0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.564237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb13db-6d1a-4d1b-9a33-bb6e23cfca0a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.577763 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 347/678] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:12.579099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb08147-de76-4b8a-b5b0-665122fede08 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.579389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb08147-de76-4b8a-b5b0-665122fede08 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.579691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb08147-de76-4b8a-b5b0-665122fede08 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.581273 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 348/679] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:12.586581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb08147-de76-4b8a-b5b0-665122fede08 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.641222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c16ea3-977a-4df6-9666-f7a639edc952 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.675300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:12.690588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.691565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.692610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.693003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.693850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=3c8ed3cf7a5742808029d336ebc784fa) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:12.694230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.695795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.703545 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 332/680] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/role_assignments?user.id=3c8ed3cf7a5742808029d336ebc784fa&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:12.704744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.730657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb08147-de76-4b8a-b5b0-665122fede08 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.787422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:12.802489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.803173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.803447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.803716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.814076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.815623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:12.832471 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:12.834417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 349/681] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:12.835844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:12.874268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01925fd8-84a5-4c74-b063-e7fe5b2cd815 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:12.916900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f853f-4e35-4936-96c6-9bc90dd6e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:12.917550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f853f-4e35-4936-96c6-9bc90dd6e6aa None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.917820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f853f-4e35-4936-96c6-9bc90dd6e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.918113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f853f-4e35-4936-96c6-9bc90dd6e6aa None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.918811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f853f-4e35-4936-96c6-9bc90dd6e6aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:12.919098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f853f-4e35-4936-96c6-9bc90dd6e6aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:12.920616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f853f-4e35-4936-96c6-9bc90dd6e6aa None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:12.939181 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 333/682] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:12.942679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab501-047e-4ee2-b25f-4c42f14bb32f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:12.943627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab501-047e-4ee2-b25f-4c42f14bb32f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:12.944207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab501-047e-4ee2-b25f-4c42f14bb32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:12.980226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-857ab501-047e-4ee2-b25f-4c42f14bb32f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:12.983324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-857ab501-047e-4ee2-b25f-4c42f14bb32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:13.020270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812e3be-d417-46d9-a5b8-1556ede83e65 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:13.067344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:13.083831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:13.084520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:13.084830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:13.085140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:13.085888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=ed91b2c83d57419dac50595f57fc3411) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:13.086192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:13.087680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:13.095952 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 350/683] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:12 2022] GET /identity/v3/role_assignments?user.id=ed91b2c83d57419dac50595f57fc3411&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:13.101134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:13.117222 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 334/684] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:13.119628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab501-047e-4ee2-b25f-4c42f14bb32f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:13.209487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:13.347226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab501-047e-4ee2-b25f-4c42f14bb32f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:13.378278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bf599-a053-4baa-b907-3778e1bfc753 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:13.454335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:13.454983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:13.455262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:13.455518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:13.456281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:13.456583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:13.458516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:13.462341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:13.463618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:13.464359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:13.465018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:13.466567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=b324af05ec3e4b35a9bfc188b3b3abec) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:13.467253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:13.470247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:13.484446 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 351/685] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 27 09:40:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:13.492448 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 335/686] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:13 2022] GET /identity/v3/role_assignments?user.id=b324af05ec3e4b35a9bfc188b3b3abec&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 608 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:13.493832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:13.564964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:13.676893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a203163-c8d1-4501-a46d-34d0715540f9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:13.739488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:13.740505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:13.741006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:13.741464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:13.824822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:13.884179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:13.886717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:13.970597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:14.064498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd842fc9-2fa9-4be8-9687-ed1f95a2ecd6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:14.071460 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 352/687] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 09:40:14.079157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:14.082293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:14.117114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:14.117773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:14.118649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:14.118649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] PATH_INFO: `/v3/services/network` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:14.119151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:14.120979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:14.144146 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:14.146109 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:14.147809 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:14.155002 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 336/688] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:13 2022] GET /identity/v3/services/network => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:14.164176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:14.382439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda55653-6fd4-4976-9e66-19e07b2f5b17 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:14.460564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:14.461154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:14.464326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a9016-5f94-4df5-a003-6ef030337519 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:14.466179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:14.466453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:14.484644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:14.486229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:14.498090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:14.511877 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:14.513764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:14.515405 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:14.515405 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:14.515405 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:14.515405 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 353/689] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:14.525221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:14.572568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:14.574325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:14.575095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:14.575821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:14.577292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] RBAC: Adding query filter params (name=network) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:14.578162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:14.584864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:14.599973 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 337/690] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:14.607232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:14.739054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:14.855550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1683-9a3e-4936-88b5-75747d1c5abb None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:14.930750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eebaec-69e7-4f94-af9a-eef0bcd3fe9d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:14.944395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56746567-9546-49b6-a020-0823f1107238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:14.945065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746567-9546-49b6-a020-0823f1107238 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:14.945335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746567-9546-49b6-a020-0823f1107238 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:14.945601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746567-9546-49b6-a020-0823f1107238 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:14.946316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56746567-9546-49b6-a020-0823f1107238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:14.946615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56746567-9546-49b6-a020-0823f1107238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:14.948010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56746567-9546-49b6-a020-0823f1107238 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:14.976518 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 354/691] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:14.984341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56746567-9546-49b6-a020-0823f1107238 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:15.017713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:15.018599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:15.019066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:15.019574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:15.020712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] RBAC: Adding query filter params (type=network) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:15.021215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:15.023625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:15.035139 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 338/692] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:15.155685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:15.246361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56746567-9546-49b6-a020-0823f1107238 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:15.295527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:15.296178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:15.296487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:15.296792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:15.305044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:15.306508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:15.319826 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:15.321610 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:15.323343 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 355/693] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:15.329805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:15.496608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:15.501274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3bf6-f8a2-42b9-84ef-462a6e3f295b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:15.545229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9224-c1ba-4051-938a-8a643bf005c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:15.545934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9224-c1ba-4051-938a-8a643bf005c8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:15.546149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9224-c1ba-4051-938a-8a643bf005c8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:15.546416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9224-c1ba-4051-938a-8a643bf005c8 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:15.547146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9224-c1ba-4051-938a-8a643bf005c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:15.547540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9224-c1ba-4051-938a-8a643bf005c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:15.549064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9224-c1ba-4051-938a-8a643bf005c8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:15.564745 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 339/694] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:15.573641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:15.837794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa054be-d839-48fc-98b7-59c431cbf425 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:15.882151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:15.917998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476737d-3d4f-46d8-8de4-490f7ee43c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:15.918998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476737d-3d4f-46d8-8de4-490f7ee43c7e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:15.919474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476737d-3d4f-46d8-8de4-490f7ee43c7e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:15.919945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476737d-3d4f-46d8-8de4-490f7ee43c7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:15.921329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476737d-3d4f-46d8-8de4-490f7ee43c7e None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=c6863dfc2c0143a88060bbf5f0e146df) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:15.921858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476737d-3d4f-46d8-8de4-490f7ee43c7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:15.924562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476737d-3d4f-46d8-8de4-490f7ee43c7e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:15.940581 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 356/695] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:15 2022] GET /identity/v3/role_assignments?user.id=c6863dfc2c0143a88060bbf5f0e146df&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 608 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:16.436545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad247bd-9439-4d3a-9c50-b06aee5b1490 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:16.438098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad247bd-9439-4d3a-9c50-b06aee5b1490 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:16.438847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad247bd-9439-4d3a-9c50-b06aee5b1490 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:16.442134 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 340/696] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:16.450505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea170-76e3-4d61-8c62-527afe7260fa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:16.450960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea170-76e3-4d61-8c62-527afe7260fa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:16.451393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea170-76e3-4d61-8c62-527afe7260fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:16.478492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4a5ea170-76e3-4d61-8c62-527afe7260fa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:16.480136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4a5ea170-76e3-4d61-8c62-527afe7260fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:16.632838 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 357/697] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:17.210155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad247bd-9439-4d3a-9c50-b06aee5b1490 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:17.477694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad247bd-9439-4d3a-9c50-b06aee5b1490 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:17.491435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3ed84-c83f-4d5e-b171-1350c9a652c9 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:17.492261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3ed84-c83f-4d5e-b171-1350c9a652c9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:17.492415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3ed84-c83f-4d5e-b171-1350c9a652c9 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:17.497310 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 358/698] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:40:17.503524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16df199-553b-4c0d-9cb7-28c0b57c0c88 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:17.503990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16df199-553b-4c0d-9cb7-28c0b57c0c88 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:17.504355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16df199-553b-4c0d-9cb7-28c0b57c0c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:17.524726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a16df199-553b-4c0d-9cb7-28c0b57c0c88 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:17.525903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a16df199-553b-4c0d-9cb7-28c0b57c0c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:17.554213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42979caa-b3dc-4677-ab92-9ccc50c3b7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:17.556059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42979caa-b3dc-4677-ab92-9ccc50c3b7ad None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:17.559442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42979caa-b3dc-4677-ab92-9ccc50c3b7ad None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:17.559887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42979caa-b3dc-4677-ab92-9ccc50c3b7ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:17.579335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42979caa-b3dc-4677-ab92-9ccc50c3b7ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-42979caa-b3dc-4677-ab92-9ccc50c3b7ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:17.584509 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:17.586869 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:17.588633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 341/699] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:17.609614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a5a-20f3-446b-b3c2-c7980ecffbfb None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:17.609614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a5a-20f3-446b-b3c2-c7980ecffbfb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:17.609614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a5a-20f3-446b-b3c2-c7980ecffbfb None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:17.609614 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 342/700] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:17.609614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96378a5a-20f3-446b-b3c2-c7980ecffbfb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:17.635165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 359/701] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:40:17.636501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504bd24-1426-4fb5-9949-ecfe1bcd0d70 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:17.636810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504bd24-1426-4fb5-9949-ecfe1bcd0d70 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:17.637080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504bd24-1426-4fb5-9949-ecfe1bcd0d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:17.655936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d504bd24-1426-4fb5-9949-ecfe1bcd0d70 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:17.657287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:17.657482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d504bd24-1426-4fb5-9949-ecfe1bcd0d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:17.702138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:17.773198 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 360/702] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:17.831572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96378a5a-20f3-446b-b3c2-c7980ecffbfb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:17.904413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:17.905842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:17.906502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:17.907193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:17.908587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:17.909337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:17.913092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:17.936835 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 343/703] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:17.941019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504bd24-1426-4fb5-9949-ecfe1bcd0d70 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:18.128846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504bd24-1426-4fb5-9949-ecfe1bcd0d70 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:18.185694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:18.186422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:18.186698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:18.186974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:18.203098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:18.204611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:18.226544 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:18.228892 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:18.230612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:18.230612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:18.230612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:18.230612 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 361/704] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:18.237906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:18.305842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:18.356619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:18.389717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2242f3-9ef7-4214-a423-d7e02f190b99 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:18.397261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:18.436912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:18.437888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:18.438326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:18.438740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:18.439569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:18.440835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:18.442487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:18.542610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f457efb-b341-4872-813c-c1f8f5dee5db None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:18.588003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:18.589030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:18.589461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:18.589878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:18.590702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:18.592380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:18.606932 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:18.608019 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:18.608920 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:18.611973 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 362/705] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Sep 27 09:40:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:18.613253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:18.644893 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 344/706] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:18.646385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:18.730302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:18.751474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d39fe-1d73-4f17-85c7-dd2ee664c7ef None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:18.803457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:18.804267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:18.804643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:18.805040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:18.817044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:18.818779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:18.836582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:18.838381 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:18.840046 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:18.840046 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:18.840046 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:18.844061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 363/707] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:18.845102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:18.896970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a4c40-bf7f-47fe-a7f9-911aa0de4715 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:18.971682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec48143-814d-4b23-908e-d38fc141d4b0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:18.972607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:18.973665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:18.974249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:18.974715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:18.975787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:18.976297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:18.978820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:18.987527 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 345/708] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 27 09:40:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:18.989156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:18.995297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:19.010013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:19.010659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.010931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.011193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:19.020041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:19.021599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:19.023603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:19.034943 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:19.037272 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 364/709] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:40:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:19.040749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:19.160292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ca185-b3f1-4a33-a947-44ca1d44cf15 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:19.166425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d021b9a-8e63-48ce-9cb6-4fd3d9c268a0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:19.207399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:19.208033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.208297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda48491-cc7b-4024-847c-f3cf58e496b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:19.208413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.208570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:19.209248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:19.209428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda48491-cc7b-4024-847c-f3cf58e496b6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.209537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:19.209874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda48491-cc7b-4024-847c-f3cf58e496b6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.210350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda48491-cc7b-4024-847c-f3cf58e496b6 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:19.210941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:19.211514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda48491-cc7b-4024-847c-f3cf58e496b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:19.212043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda48491-cc7b-4024-847c-f3cf58e496b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:19.214613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda48491-cc7b-4024-847c-f3cf58e496b6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:19.217576 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 365/710] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 27 09:40:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:19.218628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:19.248584 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 346/711] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:19.248584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb2c7b-dba0-493f-a654-4b9d403cc5c3 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.248584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb2c7b-dba0-493f-a654-4b9d403cc5c3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.248584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb2c7b-dba0-493f-a654-4b9d403cc5c3 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:19.248584 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 347/712] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:19.251224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb2c7b-dba0-493f-a654-4b9d403cc5c3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:19.361024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:19.369694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:19.441894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d48295-b3e3-4456-b2dd-23424794741b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:19.469205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb2c7b-dba0-493f-a654-4b9d403cc5c3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:19.509089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f8467-3222-4808-9d6a-112969380b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:19.510645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f8467-3222-4808-9d6a-112969380b65 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.511552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f8467-3222-4808-9d6a-112969380b65 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.512052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f8467-3222-4808-9d6a-112969380b65 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:19.513517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f8467-3222-4808-9d6a-112969380b65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:19.514137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f8467-3222-4808-9d6a-112969380b65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:19.516758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f8467-3222-4808-9d6a-112969380b65 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:19.552705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:19.552705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.552705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.552705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:19.553579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:19.554917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:19.555708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 366/713] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:40:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:19.557774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adde385-c748-49d2-80b7-4bb619aca9af None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.558089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adde385-c748-49d2-80b7-4bb619aca9af None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.558547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adde385-c748-49d2-80b7-4bb619aca9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:19.588295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 348/714] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:40:19 2022] POST /identity/v3/users => generated 254 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:19.589466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:19.602705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3adde385-c748-49d2-80b7-4bb619aca9af None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:19.604963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3adde385-c748-49d2-80b7-4bb619aca9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:19.834180 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 367/715] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:19.910454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5369b-0f81-4a3b-82f8-791133b7c06b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:19.962158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:19.994358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:19.995288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:19.995749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:19.996223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:20.014099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a387-e54b-4ecc-b3e7-b70a99c259ed None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:20.014601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a387-e54b-4ecc-b3e7-b70a99c259ed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:20.015028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a387-e54b-4ecc-b3e7-b70a99c259ed None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:20.024481 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 368/716] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:20.026558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64189aec-bd04-445d-9bed-de4965b6c0b9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:20.027007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64189aec-bd04-445d-9bed-de4965b6c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:20.027452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64189aec-bd04-445d-9bed-de4965b6c0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:20.060351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-64189aec-bd04-445d-9bed-de4965b6c0b9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:20.062556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-64189aec-bd04-445d-9bed-de4965b6c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:20.134290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:20.136944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:20.152148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:20.214088 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 369/717] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 09:40:20.290008 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 349/718] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 09:40:20.298124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64189aec-bd04-445d-9bed-de4965b6c0b9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:20.541771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64189aec-bd04-445d-9bed-de4965b6c0b9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:20.543170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:20.605676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:20.638583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:20.639619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:20.640064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:20.640701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:20.661057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:20.663647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:20.690586 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:20.693172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:20.694864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:20.694864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:20.694864 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:20.698319 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 370/719] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:20.701469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:20.758976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:20.849345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204eb93-34de-4a8a-8ca6-e82804bff080 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:20.886210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa5d9d-45f6-4c19-bf4a-816c66c576ef None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:20.953490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:20.954718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:20.954718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:20.954718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:20.955390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:20.956609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:20.957284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:20.957685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:20.958150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:20.958619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:20.959730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:20.960237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:20.962778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:20.987056 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 371/720] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:20.988274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:21.006256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 350/721] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:40:20 2022] POST /identity/v3/services => generated 240 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:40:21.007636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:21.180081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:21.198927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf697c5-d7c3-46a1-ad67-75aa343d906c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:21.277837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:21.279096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:21.279642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:21.280176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:21.301718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:21.311142 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:21.313309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:21.315202 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:21.317630 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 372/722] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:21.329816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:21.349097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:21.409556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efdb98-7405-4b7d-bf72-731b6f4aaa54 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:21.503527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:21.504449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:21.506037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:21.506453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:21.530695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:21.536632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:21.573962 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:21.575004 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:21.575958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:21.576941 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 351/723] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 570 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:21.594870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:21.650942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd8e9-8634-4fe4-bdba-fa91b7ab9ee6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:21.725261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:21.726756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:21.727434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:21.728134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:21.729523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:21.730252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:21.733066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:21.750212 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 373/724] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:21.760046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:21.806377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:21.912755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0c392-59ce-41e9-ba85-365135bd674d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:21.995962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:21.998711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:21.999484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:22.000698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:22.001749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:22.002427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:22.007085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:22.020849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:22.033021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 352/725] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:22.040913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:22.049205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4e928-7bb2-43dc-8e19-1683092d0b81 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:22.143526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:22.145969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:22.146927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:22.147585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:22.153404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8907ba-8bd6-4d71-af4c-ecde388fb2f1 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:22.176985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:22.180352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:22.196340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bafb1-7faa-42af-9f88-057837eded08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:22.197145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bafb1-7faa-42af-9f88-057837eded08 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:22.197535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bafb1-7faa-42af-9f88-057837eded08 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:22.197863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bafb1-7faa-42af-9f88-057837eded08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:22.198695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bafb1-7faa-42af-9f88-057837eded08 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=3c8ed3cf7a5742808029d336ebc784fa) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:22.199111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bafb1-7faa-42af-9f88-057837eded08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:22.200814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bafb1-7faa-42af-9f88-057837eded08 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:22.207359 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 353/726] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:22 2022] GET /identity/v3/role_assignments?user.id=3c8ed3cf7a5742808029d336ebc784fa&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:22.215014 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:22.217163 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:22.219340 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:22.219340 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:22.219340 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:22.219718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3ec92-855a-44cf-9581-47b5abf3e2b5 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:22.219718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3ec92-855a-44cf-9581-47b5abf3e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:22.219718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3ec92-855a-44cf-9581-47b5abf3e2b5 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:22.221052 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 354/727] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:22 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:22.223908 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 374/728] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 464 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:22.232809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937c3be-c6a0-44e1-8d44-bdb931d2ab8d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:22.232809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937c3be-c6a0-44e1-8d44-bdb931d2ab8d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:22.232809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937c3be-c6a0-44e1-8d44-bdb931d2ab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:22.238727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:22.252469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9937c3be-c6a0-44e1-8d44-bdb931d2ab8d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:22.253855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9937c3be-c6a0-44e1-8d44-bdb931d2ab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:22.437246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:22.441411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:22.485461 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 355/729] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:22 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 09:40:22.536542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f1416-0f54-4226-b935-8888d947b7bb None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:22.581236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:22.582313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:22.582751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:22.583178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:22.584005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:22.584445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:22.586017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:22.600374 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 375/730] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:22.608956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937c3be-c6a0-44e1-8d44-bdb931d2ab8d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:22.727068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937c3be-c6a0-44e1-8d44-bdb931d2ab8d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:22.765804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:22.766454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:22.766726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:22.766992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] PATH_INFO: `/v3/users/swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:22.776941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:22.778526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:22.793818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:22.794852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:22.796248 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 356/731] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:22.801479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:22.902851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:22.965161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a0053-5b84-4c9f-8c53-38d6ad409525 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:22.978160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:22.992557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:23.011801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b7d63-b3b3-4a59-b0e5-9193b199a583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:23.012782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b7d63-b3b3-4a59-b0e5-9193b199a583 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:23.013206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b7d63-b3b3-4a59-b0e5-9193b199a583 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:23.013763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b7d63-b3b3-4a59-b0e5-9193b199a583 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:23.015018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b7d63-b3b3-4a59-b0e5-9193b199a583 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:23.015751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b7d63-b3b3-4a59-b0e5-9193b199a583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:23.018983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b7d63-b3b3-4a59-b0e5-9193b199a583 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:23.038463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193d91a-8754-4dc1-9dcf-be59203b3460 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:23.040924 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 376/732] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 27 09:40:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:23.042515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668c23c-268e-4eab-aff7-9edbbbca8578 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:23.043010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668c23c-268e-4eab-aff7-9edbbbca8578 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:23.043468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668c23c-268e-4eab-aff7-9edbbbca8578 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:23.045550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 377/733] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:23.046673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668c23c-268e-4eab-aff7-9edbbbca8578 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:23.092728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5df1b6-4f09-4b29-8fd4-f1c65354fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:23.093373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5df1b6-4f09-4b29-8fd4-f1c65354fb9b None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:23.093728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5df1b6-4f09-4b29-8fd4-f1c65354fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:23.094016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5df1b6-4f09-4b29-8fd4-f1c65354fb9b None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:23.094692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5df1b6-4f09-4b29-8fd4-f1c65354fb9b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:23.096083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5df1b6-4f09-4b29-8fd4-f1c65354fb9b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:23.128785 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 357/734] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:40:22 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:23.132936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4d493-8314-45ac-a3d6-899fb40fe316 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:23.134853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4d493-8314-45ac-a3d6-899fb40fe316 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:23.135535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4d493-8314-45ac-a3d6-899fb40fe316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:23.207172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-66b4d493-8314-45ac-a3d6-899fb40fe316 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:23.213795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66b4d493-8314-45ac-a3d6-899fb40fe316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:23.319844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668c23c-268e-4eab-aff7-9edbbbca8578 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:23.368019 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 358/735] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:23.392308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:23.393561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:23.394088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:23.394630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:23.474024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:23.521060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:23.524026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:23.771381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4d493-8314-45ac-a3d6-899fb40fe316 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:23.892760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:23.932307 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 378/736] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 886 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:40:23.938284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:24.040710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4d493-8314-45ac-a3d6-899fb40fe316 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:24.107440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:24.108465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:24.108950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:24.109370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:24.125325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:24.131459 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:24.133077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:24.134535 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 359/737] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:24.146479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:24.189044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:24.237310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce1aa-9706-4f00-a451-5b12a9981830 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:24.296802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:24.297782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:24.298209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:24.298617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:24.312291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:24.314214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:24.332940 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:24.334171 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:24.335275 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:24.335275 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:24.335275 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:24.337216 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 379/738] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:24.343306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:24.460531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c54407-ae39-4c5d-bbf8-37667ddfea4d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:24.491149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:24.496688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6808ab-a189-49a3-9464-c1b88131afc2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:24.500991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:24.501903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:24.502579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:24.502987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:24.503847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:24.504297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:24.505915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:24.510759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:24.525077 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 380/739] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:24.527228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:24.638469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5150d-dd69-4b60-baed-05ae16d4cc99 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:24.657863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308695d-834f-4587-91b2-e579574e460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:24.658913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308695d-834f-4587-91b2-e579574e460d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:24.659404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308695d-834f-4587-91b2-e579574e460d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:24.660288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308695d-834f-4587-91b2-e579574e460d None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:24.661959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308695d-834f-4587-91b2-e579574e460d None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:24.662517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308695d-834f-4587-91b2-e579574e460d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:24.664163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308695d-834f-4587-91b2-e579574e460d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:24.679811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:24.680261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:24.680533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:24.680825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:24.683140 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 360/740] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 538 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:24.690571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308695d-834f-4587-91b2-e579574e460d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:24.696424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:24.697945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:24.719879 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:24.720875 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:24.721794 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:24.729188 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 381/741] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:24.731937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:24.851371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:24.890718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308695d-834f-4587-91b2-e579574e460d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:24.977786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246ce9e-6a70-4597-9a2f-60055fed8648 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:24.979905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46e42e-7c11-44a8-8ed9-280ede3d2e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:24.980686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e42e-7c11-44a8-8ed9-280ede3d2e76 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:24.980987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e42e-7c11-44a8-8ed9-280ede3d2e76 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:24.981261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e42e-7c11-44a8-8ed9-280ede3d2e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:24.994017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46e42e-7c11-44a8-8ed9-280ede3d2e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:24.995691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46e42e-7c11-44a8-8ed9-280ede3d2e76 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-9b46e42e-7c11-44a8-8ed9-280ede3d2e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:25.015816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:25.016906 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:25.018440 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:25.018440 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:25.018440 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:25.018440 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 361/742] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:25.019010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cee50-65c5-4131-a0b0-6cc49562adc8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.019392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cee50-65c5-4131-a0b0-6cc49562adc8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.019675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cee50-65c5-4131-a0b0-6cc49562adc8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.025193 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 362/743] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:25.026439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cee50-65c5-4131-a0b0-6cc49562adc8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:25.029901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5655e-feca-4c0b-b857-8ded62a88c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:25.030968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5655e-feca-4c0b-b857-8ded62a88c18 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.031458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5655e-feca-4c0b-b857-8ded62a88c18 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.031933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5655e-feca-4c0b-b857-8ded62a88c18 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.033239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5655e-feca-4c0b-b857-8ded62a88c18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:25.033797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5655e-feca-4c0b-b857-8ded62a88c18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:25.037703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5655e-feca-4c0b-b857-8ded62a88c18 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:25.056679 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 382/744] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:25.058757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e67964-5e73-4977-a7a9-93fe3281e6f1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.059264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e67964-5e73-4977-a7a9-93fe3281e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.059730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e67964-5e73-4977-a7a9-93fe3281e6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.063105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:25.083341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-60e67964-5e73-4977-a7a9-93fe3281e6f1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:25.085302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-60e67964-5e73-4977-a7a9-93fe3281e6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:25.190641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cee50-65c5-4131-a0b0-6cc49562adc8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:25.200930 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 383/745] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:25.203176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e67964-5e73-4977-a7a9-93fe3281e6f1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:25.235852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:25.236887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.237375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.237843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.239152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:25.240309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:25.242293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:25.257344 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 363/746] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:25.266368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:25.340795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:25.370165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:25.415485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ce5b6-9e29-445f-990e-40309078ec77 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:25.516871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e67964-5e73-4977-a7a9-93fe3281e6f1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:25.522966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1911473-4c9b-496b-92f6-1e3dfb38779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:25.523788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911473-4c9b-496b-92f6-1e3dfb38779d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.524071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911473-4c9b-496b-92f6-1e3dfb38779d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.524345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911473-4c9b-496b-92f6-1e3dfb38779d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.533280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1911473-4c9b-496b-92f6-1e3dfb38779d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:25.534751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1911473-4c9b-496b-92f6-1e3dfb38779d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-c1911473-4c9b-496b-92f6-1e3dfb38779d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:25.549304 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:25.550346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 364/747] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:40:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8572-ccd0-4e37-989c-89026ba1444c None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.553070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8572-ccd0-4e37-989c-89026ba1444c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.553793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8572-ccd0-4e37-989c-89026ba1444c None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.555070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 365/748] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:25.558696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8572-ccd0-4e37-989c-89026ba1444c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:25.584074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be0869-c478-4ab0-bc49-1809062d4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:25.584858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be0869-c478-4ab0-bc49-1809062d4c45 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.585235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be0869-c478-4ab0-bc49-1809062d4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.585652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be0869-c478-4ab0-bc49-1809062d4c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.586785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be0869-c478-4ab0-bc49-1809062d4c45 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=c6863dfc2c0143a88060bbf5f0e146df) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:25.587195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be0869-c478-4ab0-bc49-1809062d4c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:25.589274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be0869-c478-4ab0-bc49-1809062d4c45 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:25.596701 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 384/749] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:25 2022] GET /identity/v3/role_assignments?user.id=c6863dfc2c0143a88060bbf5f0e146df&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:25.598466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fde7db-ae5d-4c78-ba9b-d8e369335a08 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.598826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fde7db-ae5d-4c78-ba9b-d8e369335a08 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.604914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fde7db-ae5d-4c78-ba9b-d8e369335a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.621739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-36fde7db-ae5d-4c78-ba9b-d8e369335a08 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:25.624052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36fde7db-ae5d-4c78-ba9b-d8e369335a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:25.726480 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 385/750] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:25.727769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fde7db-ae5d-4c78-ba9b-d8e369335a08 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:25.769672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:25.787546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8572-ccd0-4e37-989c-89026ba1444c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:25.859058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:25.860061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.860435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.860847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.862073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:25.862498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:25.864717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:25.875668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:25.891951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 366/751] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 27 09:40:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:25.899415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:25.907305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fde7db-ae5d-4c78-ba9b-d8e369335a08 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:25.970131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:25.970833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:25.971105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:25.971416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] PATH_INFO: `/v3/services/network` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:25.972060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:25.973782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:25.995432 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:25.997222 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:25.998089 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 386/752] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:25 2022] GET /identity/v3/services/network => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:26.006703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:26.033349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e4f3-f4a2-4a42-81ec-74a369afcd46 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:26.074758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:26.076185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.076639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.077136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.157373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:26.159430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:26.169239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:26.238322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:26.251591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429fd7-89f8-4ae2-8f5a-a9745a8abce5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:26.290522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 367/753] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:40:26.297525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:26.364887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ef1c2-60bb-47ed-9e61-b760acbd13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:26.365645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ef1c2-60bb-47ed-9e61-b760acbd13e0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.365930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ef1c2-60bb-47ed-9e61-b760acbd13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.366237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ef1c2-60bb-47ed-9e61-b760acbd13e0 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.366876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ef1c2-60bb-47ed-9e61-b760acbd13e0 None admin] RBAC: Adding query filter params (name=network) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:26.367181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ef1c2-60bb-47ed-9e61-b760acbd13e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:26.368866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ef1c2-60bb-47ed-9e61-b760acbd13e0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:26.373841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 387/754] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:26.375212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c305d8a-5751-410f-a25c-08b6aa83ac14 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.375495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c305d8a-5751-410f-a25c-08b6aa83ac14 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.375763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c305d8a-5751-410f-a25c-08b6aa83ac14 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.380760 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 388/755] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:26.382030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c305d8a-5751-410f-a25c-08b6aa83ac14 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:26.409608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:26.418341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cf11f-6382-406b-83b5-39697bf6ac8a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:26.458609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a412fd-e16d-4be5-b2a3-08ecf11e7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:26.459265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a412fd-e16d-4be5-b2a3-08ecf11e7175 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.459562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a412fd-e16d-4be5-b2a3-08ecf11e7175 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.459858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a412fd-e16d-4be5-b2a3-08ecf11e7175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.472585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a412fd-e16d-4be5-b2a3-08ecf11e7175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:26.475665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a412fd-e16d-4be5-b2a3-08ecf11e7175 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-87a412fd-e16d-4be5-b2a3-08ecf11e7175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:26.503503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:26.505639 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:26.507634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:26.507634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:26.507634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:26.508859 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 368/756] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:26.512054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c449a-1bfb-48f9-ba99-68702e6e3bf1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.512784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c449a-1bfb-48f9-ba99-68702e6e3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.513465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c449a-1bfb-48f9-ba99-68702e6e3bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.532224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c305d8a-5751-410f-a25c-08b6aa83ac14 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:26.544677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d36c449a-1bfb-48f9-ba99-68702e6e3bf1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:26.546782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d36c449a-1bfb-48f9-ba99-68702e6e3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:26.580140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:26.580798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.581061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.581329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.582096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] RBAC: Adding query filter params (type=network) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:26.582331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:26.583779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:26.590442 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 389/757] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:26.591314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:26.631360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:26.662626 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 369/758] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:26.663274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c449a-1bfb-48f9-ba99-68702e6e3bf1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:26.770487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:26.794283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74767b8a-5986-4ecd-877b-a488a094e290 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:26.800819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c449a-1bfb-48f9-ba99-68702e6e3bf1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:26.847637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:26.848484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.849186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.849186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.850052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:26.850459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:26.852134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:26.863015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:26.864476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:26.865196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:26.865955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:26.870164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 370/759] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:26.871604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:26.891861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:26.908039 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:26.909827 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:26.911784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:26.916292 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 390/760] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:26.918040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.016932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6ba18-c0d0-4e53-9219-dc7ef5caa1cc None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.059427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.059966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.060295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.060890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.061660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] RBAC: Adding query filter params (interface=public, service_id=b1fdcb2b5ffd452e8341578357a5aa02, region_id=RegionOne) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:27.062964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.064378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.080716 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 371/761] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:40:26 2022] GET /identity/v3/endpoints?service_id=b1fdcb2b5ffd452e8341578357a5aa02&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:27.080716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.083110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:27.085878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:27.108802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5319b9-39d7-4aaf-8ca6-8b58b9708f23 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.157700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.158346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.158627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.158893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.167220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.168714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:27.181175 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:27.182787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 391/762] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:27.184582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.218410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f5322-886e-465e-bcf1-a962837a0557 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.261963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.262621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.262906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.263177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.264253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:27.264679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.266516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.279209 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 372/763] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:27.280078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.340606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb392d4b-b6f6-4c5b-8d08-f95134d952a4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.375218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:27.385081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.386069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.386394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.386759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.387482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.389094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:27.400637 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:27.402790 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:27.404498 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:27.407903 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 392/764] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:27.409073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.423838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:27.432526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111fd31-5017-4815-a9d8-c76f3d147707 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.472330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.473603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.473888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.474166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.474751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.476142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.482423 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 373/765] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:27.483294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.566208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7feab-b7ef-471c-be15-3f094597d439 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.649244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.650169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.650583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.650986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.652540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:27.652540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.655050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.687460 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 393/766] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:27.689079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.691376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0511852-464a-4a8b-9353-1bc82fb15b38 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.730405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:27.743747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.744894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.745381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.745871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.761184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.763212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:27.783006 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:27.784117 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:27.790116 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:27.790116 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:27.790116 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:27.790116 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 374/767] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:27.790116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.813987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:27.871344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15447f0f-95d3-49d0-9330-cac566bdcc0c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:27.928435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:27.930264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:27.931209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:27.935634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:27.936057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:27.936303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:27.937793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:27.949123 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 394/768] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:27.950354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:27.980180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba2694-2f67-420d-854e-fb0a739051c4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:28.057374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:28.058371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.058644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.058909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.059649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=9afc5552272c4928be73cb202945c8ba) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:28.059943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:28.061602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:28.068340 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 375/769] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/role_assignments?user.id=9afc5552272c4928be73cb202945c8ba&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:28.069243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:28.107984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2aef66-536a-4b17-9bf8-b26a74981b7c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:28.162115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:28.169582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:28.170847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.171334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.171825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.172022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:28.173268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:28.174030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:28.176118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:28.190200 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 395/770] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:28.197845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:28.298281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cff5a-f43c-49d2-9e4c-01b9555ce309 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:28.369713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd632e-2e43-4ebe-9e3c-c7a3d9cf8bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:28.370364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd632e-2e43-4ebe-9e3c-c7a3d9cf8bfe None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.370645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd632e-2e43-4ebe-9e3c-c7a3d9cf8bfe None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.370916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd632e-2e43-4ebe-9e3c-c7a3d9cf8bfe None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.371570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd632e-2e43-4ebe-9e3c-c7a3d9cf8bfe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:28.371876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd632e-2e43-4ebe-9e3c-c7a3d9cf8bfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:28.373336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd632e-2e43-4ebe-9e3c-c7a3d9cf8bfe None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:28.379624 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 376/771] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 27 09:40:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:28.381170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8d8f3-b81f-4452-ae69-2d520bbc9ddb None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.381457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8d8f3-b81f-4452-ae69-2d520bbc9ddb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.381729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8d8f3-b81f-4452-ae69-2d520bbc9ddb None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.387351 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 377/772] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:28.388319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8d8f3-b81f-4452-ae69-2d520bbc9ddb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:28.414610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95d749-71dc-445c-8fd7-6282715f5862 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:28.463621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73834f04-9a3c-4fb6-81c2-b73b50447cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:28.464625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73834f04-9a3c-4fb6-81c2-b73b50447cec None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.465088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73834f04-9a3c-4fb6-81c2-b73b50447cec None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.465545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73834f04-9a3c-4fb6-81c2-b73b50447cec None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.475595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73834f04-9a3c-4fb6-81c2-b73b50447cec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:28.477419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73834f04-9a3c-4fb6-81c2-b73b50447cec None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-73834f04-9a3c-4fb6-81c2-b73b50447cec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:28.491005 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:28.493040 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 396/773] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:40:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0d4f7-4bbf-4a55-84f6-1d60c28ef89a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.494851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0d4f7-4bbf-4a55-84f6-1d60c28ef89a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.496208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0d4f7-4bbf-4a55-84f6-1d60c28ef89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.515830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-50a0d4f7-4bbf-4a55-84f6-1d60c28ef89a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:28.517082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-50a0d4f7-4bbf-4a55-84f6-1d60c28ef89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:28.571418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8d8f3-b81f-4452-ae69-2d520bbc9ddb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:28.585057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:28.589081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:28.630425 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 397/774] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:28.630425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0d4f7-4bbf-4a55-84f6-1d60c28ef89a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:28.648343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:28.649067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.649347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.649617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.650347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] RBAC: Adding query filter params (interface=public, service_id=847f5c6b1ea54d509ebca33e99614228, region_id=RegionOne) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:28.650651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:28.652062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:28.663218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 378/775] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:40:28 2022] GET /identity/v3/endpoints?service_id=847f5c6b1ea54d509ebca33e99614228&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:28.669833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:28.802379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0d4f7-4bbf-4a55-84f6-1d60c28ef89a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:28.842983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:28.843712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.843989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.844263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.845042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:28.845347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:28.848710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:28.867912 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 398/776] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:40:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:28.877765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:28.880603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e07e-207c-4ef1-bf61-14f9b2176b76 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:28.920616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:28.921686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:28.922474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:28.923457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:28.924452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:28.926178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:28.933245 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 379/777] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:40:28 2022] GET /identity/v3/services => generated 772 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:28.956331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:29.032679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c707bb5-9f32-4e9d-b4c1-5aa8fd37c09e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:29.084958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:29.085690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:29.085968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:29.086235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:29.095933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:29.099905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:29.101513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:29.132725 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:29.134238 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:29.135152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:29.135152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:29.135152 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:29.135152 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 399/778] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:29.140518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:29.160159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:29.275640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef59c23-e7eb-428c-8c7f-aa4d948eb549 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:29.320611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:29.321350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:29.321679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:29.321952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:29.330291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f84846-d8f6-41f7-8ce2-b8798dbdac5e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:29.335826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:29.340156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:29.342007 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:29.342965 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:29.343901 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 380/779] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:29.350053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:29.389818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:29.390427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:29.390703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:29.390972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:29.391622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:29.391925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:29.393336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:29.424723 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 400/780] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:29.430624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:29.540831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:29.545773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a23b7-01b3-4505-b9b0-59c743d3e6f4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:29.625222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afb1b5-a5b6-4dcf-a766-eb3a9d2eb152 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:29.649425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:29.650081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:29.650357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:29.650625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:29.651307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:29.651604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:29.653094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:29.663929 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 381/781] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:29.669866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:29.690367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:29.691118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:29.691408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:29.691680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:29.707182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:29.709100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:29.736871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:29.737874 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:29.738829 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:29.739809 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 401/782] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:29.746575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:29.774739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:29.837072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba824677-f46e-4134-8ad4-0224e51a856d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:29.879049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:29.880107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:29.880541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:29.881115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:29.901817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:29.903316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:29.924994 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:29.926677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:29.927584 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:29.927584 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:29.927584 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:29.927584 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 382/783] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:29.933273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:29.949056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:29.982038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df0453-ec0f-4ee8-9dfb-dedfbc68efd4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:30.071613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:30.072956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.073568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.074060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.075013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:30.075496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:30.077209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:30.106221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7925152-3121-4c25-9b58-37879188e6b8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:30.109922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 402/784] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:30.116313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:30.184166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:30.184820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.185101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.185371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.186030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:30.186335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:30.187725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:30.218965 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 383/785] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:30.236495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:30.236737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:30.271554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b25ce-be76-4acb-95d9-040f9939a716 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:30.313552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:30.314280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.314554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.314821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/3c8ed3cf7a5742808029d336ebc784fa/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.316962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=3c8ed3cf7a5742808029d336ebc784fa, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:30.319002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:30.338376 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 403/786] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:30 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/3c8ed3cf7a5742808029d336ebc784fa/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:40:30.409177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:30.430128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a213a-6385-44d9-9fdc-4f93844854cd None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:30.469333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9da908-9465-4ca8-bdf7-dd2c51d1e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:30.470460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da908-9465-4ca8-bdf7-dd2c51d1e6ed None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.470795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da908-9465-4ca8-bdf7-dd2c51d1e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.471124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da908-9465-4ca8-bdf7-dd2c51d1e6ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.480574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9da908-9465-4ca8-bdf7-dd2c51d1e6ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:30.482259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9da908-9465-4ca8-bdf7-dd2c51d1e6ed None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-ca9da908-9465-4ca8-bdf7-dd2c51d1e6ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:30.496229 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:30.497513 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:30.499374 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 384/787] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:30.505207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:30.626992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1987d-d16e-4d3d-b10b-87de1ee24071 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:30.662240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:30.669913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:30.670583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.670857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.671120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.671851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:30.672149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:30.673737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:30.690119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f7c5-ccd2-46c7-970f-8f80c7914a29 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.690544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f7c5-ccd2-46c7-970f-8f80c7914a29 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.690705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f7c5-ccd2-46c7-970f-8f80c7914a29 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.692104 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 385/788] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:30.696542 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 404/789] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 27 09:40:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:30.702454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:30.702878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53550362-05f9-4049-954e-e541a83f8a59 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.703165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53550362-05f9-4049-954e-e541a83f8a59 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.703444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53550362-05f9-4049-954e-e541a83f8a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.749723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-53550362-05f9-4049-954e-e541a83f8a59 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:30.750942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-53550362-05f9-4049-954e-e541a83f8a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:30.835861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb14f3-3833-46a9-a7fe-ae9c10ed2c88 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:30.837674 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 386/790] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:30.882397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:30.883075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:30.883352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:30.883632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:30.902452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:30.904089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:30.922548 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:30.924784 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:30.927325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:30.927325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:30.927325 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:30.927325 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 405/791] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:30.931828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53550362-05f9-4049-954e-e541a83f8a59 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:31.020610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:31.082506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53550362-05f9-4049-954e-e541a83f8a59 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:31.134456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:31.135092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.135375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.135645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.136309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:31.136614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:31.138062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:31.158180 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 387/792] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:31.164560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:31.237747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:31.263836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:31.301989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617893a-f66e-4333-9dfd-16f49a00f26e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:31.342168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:31.343216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.343692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.344162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.357113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:31.359824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:31.380459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:31.381923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:31.382868 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:31.383304 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 406/793] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:31.389799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:31.479812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e4d4-26f6-4f55-86a2-97806bf9a05b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:31.504787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c8a94-31ed-4a56-8a8e-86a5e39cb65c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:31.505647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:31.506498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:31.529998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfebb4-09ac-44a5-94cd-faf080c5790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:31.531230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfebb4-09ac-44a5-94cd-faf080c5790f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.531719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfebb4-09ac-44a5-94cd-faf080c5790f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.532380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfebb4-09ac-44a5-94cd-faf080c5790f None admin] PATH_INFO: `/v3/services/network` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.533870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfebb4-09ac-44a5-94cd-faf080c5790f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:31.536536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfebb4-09ac-44a5-94cd-faf080c5790f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-8cbfebb4-09ac-44a5-94cd-faf080c5790f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:31.548501 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:31.549885 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 09:40:31.550995 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:31.551831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffe070-a481-4f59-a054-15cb64042128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:31.551831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffe070-a481-4f59-a054-15cb64042128 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.551831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffe070-a481-4f59-a054-15cb64042128 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.551831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffe070-a481-4f59-a054-15cb64042128 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.552110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffe070-a481-4f59-a054-15cb64042128 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:31.552110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffe070-a481-4f59-a054-15cb64042128 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:31.553473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffe070-a481-4f59-a054-15cb64042128 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:31.559119 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 388/794] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:31 2022] GET /identity/v3/services/network => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:31.559119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d9590-feb6-4578-a27e-789e95cdda6f None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.559610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d9590-feb6-4578-a27e-789e95cdda6f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.560093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d9590-feb6-4578-a27e-789e95cdda6f None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.567816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 389/795] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:31.568908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d9590-feb6-4578-a27e-789e95cdda6f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:31.576871 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 407/796] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:31.578635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d78c40-3aa1-4048-90e5-13d6187e4796 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.579139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d78c40-3aa1-4048-90e5-13d6187e4796 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.579607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d78c40-3aa1-4048-90e5-13d6187e4796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.614723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-84d78c40-3aa1-4048-90e5-13d6187e4796 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:31.616214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84d78c40-3aa1-4048-90e5-13d6187e4796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:31.697914 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 408/797] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:31.698488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d78c40-3aa1-4048-90e5-13d6187e4796 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:31.787810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d9590-feb6-4578-a27e-789e95cdda6f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:31.838752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:31.839800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.840284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.840810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.842218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:31.842770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:31.844377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:31.850474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 390/798] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:31.857241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:31.921572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:31.937412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d78c40-3aa1-4048-90e5-13d6187e4796 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:31.944283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:31.989758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:31.990911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:31.991404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:31.991877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/c6863dfc2c0143a88060bbf5f0e146df/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:31.995464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=c6863dfc2c0143a88060bbf5f0e146df, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:31.998055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:32.026301 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 409/799] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:31 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/c6863dfc2c0143a88060bbf5f0e146df/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:40:32.066454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921fab-d0f0-48e3-a03f-b26a7cd351d4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:32.099836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:32.137724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:32.138851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:32.139404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:32.139882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:32.140857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] RBAC: Adding query filter params (type=network) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:32.141497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:32.144364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:32.156937 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 391/800] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:32.168037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:32.267656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02262559-fd7c-4b7c-804d-f3c9377cae90 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:32.320316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20545b9-c4ce-42de-9b80-25a8ad3e7710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:32.321069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20545b9-c4ce-42de-9b80-25a8ad3e7710 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:32.321348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20545b9-c4ce-42de-9b80-25a8ad3e7710 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:32.321806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20545b9-c4ce-42de-9b80-25a8ad3e7710 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:32.323608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f140-f20e-4c9b-b4de-1a02b5a4fc10 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:32.337293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20545b9-c4ce-42de-9b80-25a8ad3e7710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:32.342125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-a20545b9-c4ce-42de-9b80-25a8ad3e7710 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:32.344927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:32.345973 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:32.347840 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 410/801] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:32.348604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a7d61-6ea9-41b2-8326-b4336b61fe12 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:32.348604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a7d61-6ea9-41b2-8326-b4336b61fe12 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:32.352900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a7d61-6ea9-41b2-8326-b4336b61fe12 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:32.357187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 411/802] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:32.358676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a7d61-6ea9-41b2-8326-b4336b61fe12 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:32.369417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc3b3-57a6-429a-af57-26884ca931fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:32.370096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc3b3-57a6-429a-af57-26884ca931fd None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:32.370333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc3b3-57a6-429a-af57-26884ca931fd None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:32.370957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc3b3-57a6-429a-af57-26884ca931fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:32.371441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc3b3-57a6-429a-af57-26884ca931fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:32.372921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc3b3-57a6-429a-af57-26884ca931fd None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:32.384016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:32.415170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 392/803] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:40:32 2022] POST /identity/v3/endpoints => generated 337 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:32.416748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4c4fd-f179-443c-96c0-21a0d66ede5e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:32.417058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4c4fd-f179-443c-96c0-21a0d66ede5e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:32.417339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4c4fd-f179-443c-96c0-21a0d66ede5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:32.464152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a8d4c4fd-f179-443c-96c0-21a0d66ede5e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:32.467670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8d4c4fd-f179-443c-96c0-21a0d66ede5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:32.579238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a7d61-6ea9-41b2-8326-b4336b61fe12 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:32.631659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:32.632427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:32.632750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:32.633027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:32.633780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:32.634084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:32.635628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:32.645936 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 412/804] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:32.654438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:32.663466 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 393/805] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:32.763314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:32.808551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b29fd-a55a-40a8-afc1-8676296601fe None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:32.849871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:32.850450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:32.850728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:32.851000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:32.864514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:32.866130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:32.883104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:32.884137 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:32.885077 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:32.885077 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:32.885077 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:32.885077 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 413/806] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:32.890374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4c4fd-f179-443c-96c0-21a0d66ede5e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.025830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.029546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:33.038940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4c4fd-f179-443c-96c0-21a0d66ede5e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.103876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.104436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.104753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.105022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.105722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:33.106057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.107574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.124865 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 394/807] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:33.127612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.175059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc7d18-15e7-49dc-9c17-148e4a66be23 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.202031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:33.217877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.218611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.218962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.219307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.219957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.221452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:33.233688 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:33.234779 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 09:40:33.235612 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:33.240230 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 414/808] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:33.241345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.257873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870c857-3c43-4e87-89d5-9e666ca5e37e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.296754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.297870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.298454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.298918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.309275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.311258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:33.324526 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:33.325618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:33.326648 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:33.329452 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 395/809] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:33.337645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.363683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:33.368241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b26ba-1f28-4df0-aae8-797263e89cdc None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.406710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.407322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.407622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.408238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.409107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:33.409426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.411443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.422679 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 415/810] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:33.424217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.469732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1896e-8029-4d4e-a92f-68d1ea95a14c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.488615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:33.507838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2334c-0eb1-4ac7-b5ab-72eef86d2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.508580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2334c-0eb1-4ac7-b5ab-72eef86d2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.508917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2334c-0eb1-4ac7-b5ab-72eef86d2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.509164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2334c-0eb1-4ac7-b5ab-72eef86d2cf5 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.509954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2334c-0eb1-4ac7-b5ab-72eef86d2cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:33.510261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2334c-0eb1-4ac7-b5ab-72eef86d2cf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.511700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2334c-0eb1-4ac7-b5ab-72eef86d2cf5 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.536433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 396/811] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:33.536433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b625b-fb7b-49f5-9878-26b86b081adf None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.536433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b625b-fb7b-49f5-9878-26b86b081adf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.536433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b625b-fb7b-49f5-9878-26b86b081adf None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.536433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 397/812] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:33.538092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b625b-fb7b-49f5-9878-26b86b081adf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.550927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa884e4-a288-4f97-ae37-5f449a33a9e0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.591610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7f03c-3cb4-4911-b19a-14c938049bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.592241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7f03c-3cb4-4911-b19a-14c938049bc4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.592519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7f03c-3cb4-4911-b19a-14c938049bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.592824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7f03c-3cb4-4911-b19a-14c938049bc4 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.593465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7f03c-3cb4-4911-b19a-14c938049bc4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:33.593768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7f03c-3cb4-4911-b19a-14c938049bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.595148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7f03c-3cb4-4911-b19a-14c938049bc4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.606325 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 416/813] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:33.607829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0de437-3207-4c7d-9154-b9be7796f4c6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.608118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0de437-3207-4c7d-9154-b9be7796f4c6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.608882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0de437-3207-4c7d-9154-b9be7796f4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.627999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1f0de437-3207-4c7d-9154-b9be7796f4c6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:33.629240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1f0de437-3207-4c7d-9154-b9be7796f4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:33.640071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:33.668636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b625b-fb7b-49f5-9878-26b86b081adf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.708687 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 417/814] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:33.708687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0de437-3207-4c7d-9154-b9be7796f4c6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.721034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.721979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.722354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.722663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.741338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.742819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:33.759937 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:33.760973 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:33.761852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:33.761852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:33.761852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:33.764880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:33.765847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 398/815] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:33.767361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:33.842744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0de437-3207-4c7d-9154-b9be7796f4c6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.892108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.892687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.892969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.893244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.893947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.896101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.900960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42713f8e-6fc3-4c91-8ec5-fc9cd2c9ed98 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:33.926667 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 418/816] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:40:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:33.944522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:33.961772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:33.962423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:33.962813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:33.963102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:33.963789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:33.964101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:33.965797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:33.987049 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 399/817] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:33.988601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:34.058672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:34.117815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:34.122647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee8613-c8e7-46c9-91be-79c27274feb3 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:34.166923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:34.167503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.167781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.168059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.176755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:34.178256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:34.190639 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:34.191993 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:34.193161 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:34.193699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 419/818] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:34.202110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:34.220485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817120d-a4dd-4847-a785-f469d2a01d64 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:34.272555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:34.274064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.274800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.275215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.291642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:34.298233 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:34.299411 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:34.300610 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:34.309933 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 400/819] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:34.312159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:34.343811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb850c-c529-4ce7-8cbc-1de641c07295 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:34.371013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:34.391139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:34.391856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.392086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.392452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.393337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:34.393685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:34.395619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:34.417438 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 420/820] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:34.419064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:34.441679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0844-6815-488f-b749-e00454cc4a10 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:34.443852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:34.482708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:34.483678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.484103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.484532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.485514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:34.486054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:34.487657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:34.498507 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 401/821] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:34.507141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:34.531838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40988e-77ca-42b3-8187-84faf7672f6c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:34.576432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:34.577225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.577635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.577902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/9afc5552272c4928be73cb202945c8ba/roles/d2b916599fc849669bff95d9ac632b79` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.595875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=9afc5552272c4928be73cb202945c8ba, role_id=d2b916599fc849669bff95d9ac632b79)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:34.597909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:34.624044 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 421/822] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:34 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/9afc5552272c4928be73cb202945c8ba/roles/d2b916599fc849669bff95d9ac632b79 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:40:34.641223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b281d3-befc-4307-b7c0-08b51fc78f75 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:34.661439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:34.687884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e24370-ff5a-4175-b034-e15ad3281278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:34.688540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24370-ff5a-4175-b034-e15ad3281278 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.688797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24370-ff5a-4175-b034-e15ad3281278 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.689078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e24370-ff5a-4175-b034-e15ad3281278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.703184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e24370-ff5a-4175-b034-e15ad3281278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:34.704799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e24370-ff5a-4175-b034-e15ad3281278 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-14e24370-ff5a-4175-b034-e15ad3281278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:34.722834 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:34.723884 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:34.725287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:34.725287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:34.725287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:34.725287 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 402/823] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:34.730557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:34.819626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:34.856621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea3ac-8804-4315-8f44-8ca3b2e93824 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.857070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea3ac-8804-4315-8f44-8ca3b2e93824 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.857214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea3ac-8804-4315-8f44-8ca3b2e93824 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.857846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbbebf-50ce-44a4-a4fc-182004c7ef25 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:34.859016 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 403/824] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:34.865402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad324b-7e34-4432-ae16-d96c086536ee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.865793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad324b-7e34-4432-ae16-d96c086536ee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.865959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad324b-7e34-4432-ae16-d96c086536ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.901203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-90ad324b-7e34-4432-ae16-d96c086536ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:34.902534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90ad324b-7e34-4432-ae16-d96c086536ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:34.906907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:34.907681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:34.907962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:34.908238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:34.909109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:34.909416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:34.911405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:34.927976 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 422/825] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:34.936647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:35.006016 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 404/826] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:35.055883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b108001-2534-4cc7-aebf-478cca39939a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:35.086946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:35.093534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:35.094465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:35.094942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:35.095429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:35.104521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:35.106329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:35.119521 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:35.120524 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:40:35.121635 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:35.123791 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 423/827] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:40:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:35.130351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad324b-7e34-4432-ae16-d96c086536ee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:35.233879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:35.250897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad324b-7e34-4432-ae16-d96c086536ee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:35.299388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:35.300215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:35.300632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:35.301140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:35.302518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:35.303050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:35.305548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:35.341448 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 405/828] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:40:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:35.350153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:35.385717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:35.494978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f098-c6e3-4069-8c67-10b139f8acfa None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:35.526790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c3291-7f3c-4263-acac-5c6be815ea6f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:35.543813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:35.544438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:35.544730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:35.544996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:35.573947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:35.575101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:35.575628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:35.576112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:35.590554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:35.597831 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:35.598950 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:40:35.599797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:35.602777 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 406/829] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:35.608278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:35.623321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:35.627349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:35.640781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:35.642810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:35.738794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b6b9f-8d09-429e-acae-597e22782130 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:35.776015 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 424/830] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:40:35.779063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:35.780043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:35.780514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:35.780998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:35.781874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:35.783128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:35.785299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:35.787700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:35.798747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 407/831] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:35.806593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:35.885124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:35.920812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:35.924491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8655dd-732c-4666-a60a-2869d4673c88 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:35.933873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624356d2-9939-4aa3-8b3e-0f411855a443 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:35.962420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:35.963026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:35.963299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:35.963567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:35.976265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:35.979853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:35.993539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:35.994385 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:35.995326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:35.996348 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:35.996348 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:35.996348 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:35.997588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:35.998197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:35.998716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.000556 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 408/832] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:36.001648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.014298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:36.016246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:36.033765 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:36.034841 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:36.037104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:36.037104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:36.037104 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:36.044749 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 425/833] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:36.045787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.136402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590945fa-b0f1-4002-8df2-c808857d3ed3 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:36.160636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:36.185545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:36.186983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.188227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.189145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.190561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:36.191238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:36.194158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:36.195218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e849-2c3d-4f03-afed-ce7d956b7c4a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:36.212374 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 409/834] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:36.221725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.247131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:36.248142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.248628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.249149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.250308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:36.251108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:36.254203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:36.285507 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 426/835] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:36.291729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.303400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:36.384578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7033dfc-4501-431f-b1f5-99fa6cd58e44 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:36.432844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:36.433576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.433956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.434323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] PATH_INFO: `/v3/users/swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.443896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:36.445700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:36.460277 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:36.461577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 09:40:36.462744 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:36.464756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7060e50-b201-4532-b988-8b23a29e43eb None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:36.469244 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 410/836] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:36.470827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.512603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:36.513613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.514036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.514528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.523549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:36.525255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:36.531249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:36.540978 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:36.542100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:36.543219 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:36.550037 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 427/837] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:36.553080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.628142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b503f8-b3a9-498a-b166-9048f07f6939 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:36.643730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:36.673371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21601ff-01f4-4a50-aa03-077604e0ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:36.674176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21601ff-01f4-4a50-aa03-077604e0ebf9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.674555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21601ff-01f4-4a50-aa03-077604e0ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.674955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21601ff-01f4-4a50-aa03-077604e0ebf9 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.675883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21601ff-01f4-4a50-aa03-077604e0ebf9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:36.676339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21601ff-01f4-4a50-aa03-077604e0ebf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:36.678004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21601ff-01f4-4a50-aa03-077604e0ebf9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:36.698682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dbef5-16a8-45f1-8760-848aab01bca8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:36.705442 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 411/838] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:36.707184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130892-e06f-436e-a9c1-823602280994 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.707480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130892-e06f-436e-a9c1-823602280994 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.707767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130892-e06f-436e-a9c1-823602280994 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.709230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 412/839] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:36.710061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18130892-e06f-436e-a9c1-823602280994 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.745120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ce1c1-9018-4d9a-9c62-882e0d87a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:36.745723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ce1c1-9018-4d9a-9c62-882e0d87a743 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.745993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ce1c1-9018-4d9a-9c62-882e0d87a743 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.746258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ce1c1-9018-4d9a-9c62-882e0d87a743 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.746962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ce1c1-9018-4d9a-9c62-882e0d87a743 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:36.747262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ce1c1-9018-4d9a-9c62-882e0d87a743 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:36.748757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ce1c1-9018-4d9a-9c62-882e0d87a743 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:36.763175 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 428/840] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:36.764497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b493b-f838-4439-9f2e-296a40e90f10 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.764811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b493b-f838-4439-9f2e-296a40e90f10 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.765080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b493b-f838-4439-9f2e-296a40e90f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.785168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9f9b493b-f838-4439-9f2e-296a40e90f10 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:36.786788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9f9b493b-f838-4439-9f2e-296a40e90f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:36.850514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:36.858293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18130892-e06f-436e-a9c1-823602280994 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:36.873663 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 429/841] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:36.875002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b493b-f838-4439-9f2e-296a40e90f10 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:36.907871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:36.909312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:36.909770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:36.910358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:36.953257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:37.000245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.002133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.013987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b493b-f838-4439-9f2e-296a40e90f10 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.062066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.062912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.063134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.063470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.064279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=3c8ed3cf7a5742808029d336ebc784fa) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:37.064591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.066515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.073225 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 430/842] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/role_assignments?user.id=3c8ed3cf7a5742808029d336ebc784fa&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:37.129753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:37.135719 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 413/843] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:40:37.141695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.237739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.258032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a2a8-f7ad-4fb9-a5c1-0517d3e52b4e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.282310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:37.311038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f0ca5-ee22-42b9-8c5c-20a7815209b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.311702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f0ca5-ee22-42b9-8c5c-20a7815209b1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.311975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f0ca5-ee22-42b9-8c5c-20a7815209b1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.312240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f0ca5-ee22-42b9-8c5c-20a7815209b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.324863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f0ca5-ee22-42b9-8c5c-20a7815209b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.326340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f0ca5-ee22-42b9-8c5c-20a7815209b1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-296f0ca5-ee22-42b9-8c5c-20a7815209b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:37.341811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:37.343031 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:37.344090 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:37.344090 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:37.344090 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:37.344090 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 431/844] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:37.344438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4520-fa3a-4e53-aa17-d785155eb292 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.344438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4520-fa3a-4e53-aa17-d785155eb292 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.344633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4520-fa3a-4e53-aa17-d785155eb292 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.346109 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 432/845] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:37.349683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d4520-fa3a-4e53-aa17-d785155eb292 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.373908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf2d91-c4fc-40ed-b392-5a259d84fe8b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.416065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b7ff7-6dfc-434c-ab22-8f5b4133be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.416780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b7ff7-6dfc-434c-ab22-8f5b4133be4a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.417062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b7ff7-6dfc-434c-ab22-8f5b4133be4a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.417337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b7ff7-6dfc-434c-ab22-8f5b4133be4a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.417936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b7ff7-6dfc-434c-ab22-8f5b4133be4a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.419331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b7ff7-6dfc-434c-ab22-8f5b4133be4a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-9b3b7ff7-6dfc-434c-ab22-8f5b4133be4a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:37.431796 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:37.432925 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:37.433789 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:37.437126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 414/846] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:37.439295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead35f-f63b-467b-aa1d-b6a7a066122a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.439701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead35f-f63b-467b-aa1d-b6a7a066122a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.439991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead35f-f63b-467b-aa1d-b6a7a066122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.459318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-22ead35f-f63b-467b-aa1d-b6a7a066122a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:37.460520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-22ead35f-f63b-467b-aa1d-b6a7a066122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:37.468088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d4520-fa3a-4e53-aa17-d785155eb292 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.506113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.506644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.506902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.507269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.507972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:37.508286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.509688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.524306 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 433/847] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:37.525907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:37.529396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.538676 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 415/848] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:37.539417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead35f-f63b-467b-aa1d-b6a7a066122a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.564540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:37.654161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db05bc9-96b4-4cd3-9e5c-4c952e6774df None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.664775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead35f-f63b-467b-aa1d-b6a7a066122a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.696692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.697809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.698437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.698864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.699778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:37.700190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.702887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.705653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.706474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.707307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.707723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.712812 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 434/849] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:37.717882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.720151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.721825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:37.735933 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:37.736952 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:37.737878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:37.742338 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 416/850] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:37.743681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.804221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:37.839904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:37.854136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fadf18a-8041-439b-b553-f61bd3207bc5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.881299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800570d9-969a-49f3-a8e3-9455cb58d38d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:37.897813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.898428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.898699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.898963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.899615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:37.899956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.901392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.909115 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 435/851] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:37.909115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:37.940239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:37.942070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:37.942901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:37.943501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:37.945475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:37.946062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:37.948976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:37.970628 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 417/852] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:37.977904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:38.046297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158a05f-0b4c-44e6-b5f7-ab7837ae55bb None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:38.123963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:38.125112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:38.125628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:38.126131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:38.126644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c78d7-fdf7-46fd-9450-2d8f417482a1 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:38.144091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:38.148831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:38.155080 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:38.156642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:38.158144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 436/853] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:38.161289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:38.185108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:38.186504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:38.187136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:38.187595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:38.188492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=c6863dfc2c0143a88060bbf5f0e146df) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:38.188973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:38.190637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:38.194969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:38.198982 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 418/854] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:37 2022] GET /identity/v3/role_assignments?user.id=c6863dfc2c0143a88060bbf5f0e146df&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:38.291324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad39c6c-3be5-4e4d-8826-8639d1f906b2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:38.334141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:38.334788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:38.335065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:38.335336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:38.336124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:38.340648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:38.340648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:38.348906 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 437/855] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:40:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:38.354642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:38.486586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff99692-ae9d-4794-8d3e-a5466dd2a81b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:38.527727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:38.528634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:38.529135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:38.529555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:38.543577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:38.545519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:38.563354 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:38.564629 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:38.565814 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:38.565814 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:38.565814 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:38.568395 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 419/856] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:38.573692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:38.669568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:38.694497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c2f46-2959-463f-b0fc-bafe787546ac None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:38.731539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:38.732152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:38.732426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:38.732739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:38.733365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:38.733660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:38.735036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:38.752859 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 438/857] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:38.755291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:38.789052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:38.914978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa0a4-e0d6-4140-a679-1223bf33341a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:38.976233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:38.977300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:38.977779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:38.978247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:38.993142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:38.995962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:39.018238 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:39.020077 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:39.023828 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:39.025832 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 420/858] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:40:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:39.031129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:39.213115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9e133-e9ce-4ed6-a634-cc75934f2cce None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:39.279396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c57470-8119-476c-bf1a-a2ea69f9861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:39.280417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c57470-8119-476c-bf1a-a2ea69f9861a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:39.280934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c57470-8119-476c-bf1a-a2ea69f9861a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:39.281406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c57470-8119-476c-bf1a-a2ea69f9861a None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:39.282783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c57470-8119-476c-bf1a-a2ea69f9861a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:39.283306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c57470-8119-476c-bf1a-a2ea69f9861a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:39.286477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c57470-8119-476c-bf1a-a2ea69f9861a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:39.302013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:39.313980 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 439/859] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 27 09:40:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:39.323517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:39.477412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:39.483339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb72f46-4c9d-4cd7-8ba4-6814765f560c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:39.521852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:39.522480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:39.522759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:39.523032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:39.612807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:39.612807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:39.703279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:39.709542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ffe6-2c8f-4633-a236-ad7d669636f7 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:39.710335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ffe6-2c8f-4633-a236-ad7d669636f7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:39.710335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ffe6-2c8f-4633-a236-ad7d669636f7 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:39.712370 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 440/860] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:40:39.717906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d5bb4-4b2c-4cdf-baf6-1d2479307410 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:39.718164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d5bb4-4b2c-4cdf-baf6-1d2479307410 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:39.718442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d5bb4-4b2c-4cdf-baf6-1d2479307410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:39.726928 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 421/861] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:40:39.746525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:39.754405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf8d5bb4-4b2c-4cdf-baf6-1d2479307410 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:39.756694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf8d5bb4-4b2c-4cdf-baf6-1d2479307410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:39.885105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7f09-f795-466e-8f06-9b3711b67f58 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:39.907216 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 441/862] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:39.944522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:39.945588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:39.946067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:39.946487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:39.966196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:39.968682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:39.996978 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:39.999429 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:40.001563 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:40.001563 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:40.001563 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:40.004680 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 422/863] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:40.008234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d5bb4-4b2c-4cdf-baf6-1d2479307410 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:40.147054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d5bb4-4b2c-4cdf-baf6-1d2479307410 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:40.177288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:40.186548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:40.187087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:40.187368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:40.187640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:40.188575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:40.188974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:40.190379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:40.204140 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 442/864] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:40.210078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:40.261648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:40.275266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:40.343505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbbb5c-9ee1-4f5e-a198-6bd2d38b3cc9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:40.396309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:40.397043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:40.397371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:40.397582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:40.407921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:40.409528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:40.427895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b1d89-a5ee-4183-9f90-1fe84a3815c7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:40.428252 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:40.429959 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:40.431295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 423/865] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:40.436116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:40.472988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:40.473719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:40.473994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:40.474264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:40.474861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:40.476272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:40.488732 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:40.490788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:40.491872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 443/866] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Sep 27 09:40:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:40.496100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:40.571254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:40.572923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:40.574339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea1aae-1980-467e-811c-0454dca35b83 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:40.634536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:40.635534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:40.636023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:40.636398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:40.637552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:40.638404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:40.641335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:40.648854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe65c9-7be6-41b8-9166-91dafdacd84b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:40.668556 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 424/867] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:40.670294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:40.690653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:40.692038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:40.692949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:40.693376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:40.694517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:40.695059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:40.697736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:40.708736 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 444/868] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:40.714507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:40.811641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04b36f-b328-4c5a-94f0-280198a5ef21 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:40.843620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebda135-845d-428e-b5d1-82e426e6476f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:40.850352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f3acc-09b3-4d3e-9531-3370988349df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:40.851188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f3acc-09b3-4d3e-9531-3370988349df None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:40.851571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f3acc-09b3-4d3e-9531-3370988349df None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:40.851944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f3acc-09b3-4d3e-9531-3370988349df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:40.852874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f3acc-09b3-4d3e-9531-3370988349df None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=9afc5552272c4928be73cb202945c8ba) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:40.853335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f3acc-09b3-4d3e-9531-3370988349df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:40.854980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f3acc-09b3-4d3e-9531-3370988349df None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:40.861609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:40.868114 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 425/869] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:40 2022] GET /identity/v3/role_assignments?user.id=9afc5552272c4928be73cb202945c8ba&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:40.887686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ae7bc-9a4b-4055-94ce-4483f5fe6544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:40.888593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae7bc-9a4b-4055-94ce-4483f5fe6544 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:40.889066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae7bc-9a4b-4055-94ce-4483f5fe6544 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:40.889503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae7bc-9a4b-4055-94ce-4483f5fe6544 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:40.890316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ae7bc-9a4b-4055-94ce-4483f5fe6544 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:40.890765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ae7bc-9a4b-4055-94ce-4483f5fe6544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:40.892361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ae7bc-9a4b-4055-94ce-4483f5fe6544 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:40.899879 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 445/870] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:41.051136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4983-5b59-4df2-9846-9934dca2d199 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:41.051958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4983-5b59-4df2-9846-9934dca2d199 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:41.051958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4983-5b59-4df2-9846-9934dca2d199 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:41.053579 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 426/871] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:41.060510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2552750-2477-4c14-a8fc-e983dfd3802f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:41.061081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2552750-2477-4c14-a8fc-e983dfd3802f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:41.061621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2552750-2477-4c14-a8fc-e983dfd3802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:41.089473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b2552750-2477-4c14-a8fc-e983dfd3802f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:41.091093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b2552750-2477-4c14-a8fc-e983dfd3802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:41.101047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcf943-a92e-4520-985d-da5117168ae4 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:41.102208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcf943-a92e-4520-985d-da5117168ae4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:41.102208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcf943-a92e-4520-985d-da5117168ae4 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:41.103824 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 427/872] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:41.109987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d816e-37a3-47a7-a22a-f5c760473ec2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:41.110357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d816e-37a3-47a7-a22a-f5c760473ec2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:41.110733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d816e-37a3-47a7-a22a-f5c760473ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:41.132144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5f1d816e-37a3-47a7-a22a-f5c760473ec2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:41.133670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5f1d816e-37a3-47a7-a22a-f5c760473ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:41.155765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:41.184704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 446/873] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:41.215422 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 428/874] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:41.512046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2552750-2477-4c14-a8fc-e983dfd3802f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:41.553840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d816e-37a3-47a7-a22a-f5c760473ec2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:41.641200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2552750-2477-4c14-a8fc-e983dfd3802f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:41.654150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:41.672555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d816e-37a3-47a7-a22a-f5c760473ec2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:41.694246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:41.695714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:41.696202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:41.696706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:41.697966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:41.701479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:41.715297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:41.716057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:41.716505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:41.716811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:41.717610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:41.719384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:41.729241 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:41.731076 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:41.732713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 429/875] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:41.736766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 447/876] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:40:41 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:41.744162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:41.771993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:41.879894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e60b-7ec1-4e53-947c-72def3afa950 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:41.932561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c554d-2aa0-49c5-b1b0-4057ed40878c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:41.933216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c554d-2aa0-49c5-b1b0-4057ed40878c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:41.933490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c554d-2aa0-49c5-b1b0-4057ed40878c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:41.933754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c554d-2aa0-49c5-b1b0-4057ed40878c None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:41.934385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c554d-2aa0-49c5-b1b0-4057ed40878c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:41.934681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c554d-2aa0-49c5-b1b0-4057ed40878c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:41.936066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c554d-2aa0-49c5-b1b0-4057ed40878c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:41.941981 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 430/877] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:41.947724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:42.062767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ebd3f-3992-464c-9e8b-cff7deff5697 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:42.102111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af64b19-6926-4fc4-8016-2011753a90db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:42.102833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af64b19-6926-4fc4-8016-2011753a90db None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:42.103243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af64b19-6926-4fc4-8016-2011753a90db None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:42.103518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af64b19-6926-4fc4-8016-2011753a90db None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:42.104188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af64b19-6926-4fc4-8016-2011753a90db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:42.104586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af64b19-6926-4fc4-8016-2011753a90db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:42.106266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af64b19-6926-4fc4-8016-2011753a90db None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:42.116846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 448/878] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:43.457285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41040639-f0fb-45ab-9a89-8511d0417cc7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:43.457864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41040639-f0fb-45ab-9a89-8511d0417cc7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:43.457864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41040639-f0fb-45ab-9a89-8511d0417cc7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:43.459462 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 431/879] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:43.465155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f60c6-5490-4435-aa4a-d8d0d1a99e2a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:43.465640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f60c6-5490-4435-aa4a-d8d0d1a99e2a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:43.465743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f60c6-5490-4435-aa4a-d8d0d1a99e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:43.486004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8a4f60c6-5490-4435-aa4a-d8d0d1a99e2a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:43.487268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8a4f60c6-5490-4435-aa4a-d8d0d1a99e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:43.518845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:43.570323 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 449/880] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:43.654298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693825-4057-4ffb-bd36-011c14420558 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:43.654633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693825-4057-4ffb-bd36-011c14420558 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:43.654834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693825-4057-4ffb-bd36-011c14420558 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:43.656403 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 432/881] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:43.663191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94915f27-9c11-4467-885b-fcae5fecf3da None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:43.663847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94915f27-9c11-4467-885b-fcae5fecf3da None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:43.664327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94915f27-9c11-4467-885b-fcae5fecf3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:43.685605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-94915f27-9c11-4467-885b-fcae5fecf3da None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:43.687435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-94915f27-9c11-4467-885b-fcae5fecf3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:43.769736 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 450/882] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:43.996582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65693825-4057-4ffb-bd36-011c14420558 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:44.090114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:44.122337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65693825-4057-4ffb-bd36-011c14420558 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:44.126443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94915f27-9c11-4467-885b-fcae5fecf3da None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:44.188691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506ac1e-50e7-467b-a7c4-435d4b6b4358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:44.188691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506ac1e-50e7-467b-a7c4-435d4b6b4358 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.188691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506ac1e-50e7-467b-a7c4-435d4b6b4358 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.188691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506ac1e-50e7-467b-a7c4-435d4b6b4358 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.189133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506ac1e-50e7-467b-a7c4-435d4b6b4358 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:44.189706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506ac1e-50e7-467b-a7c4-435d4b6b4358 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:44.236697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 433/883] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:40:43 2022] POST /identity/v3/services => generated 269 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:44.389499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:44.407431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94915f27-9c11-4467-885b-fcae5fecf3da None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:44.459706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924492ce-f6a5-418e-9293-12d7f5f43ce8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.460244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924492ce-f6a5-418e-9293-12d7f5f43ce8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.460244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924492ce-f6a5-418e-9293-12d7f5f43ce8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.461876 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 434/884] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:40:44.470874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30a6ee-a09d-4a06-865d-142d621fa3bb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.470874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30a6ee-a09d-4a06-865d-142d621fa3bb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.471497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30a6ee-a09d-4a06-865d-142d621fa3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.485079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:44.488963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.490153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.490678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.503755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ca30a6ee-a09d-4a06-865d-142d621fa3bb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:44.505533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ca30a6ee-a09d-4a06-865d-142d621fa3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:44.513289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:44.518396 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:44.519563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:44.521337 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 451/885] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 27 09:40:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:44.526133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:44.617485 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 435/886] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:44.656382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fc992-6e0f-467d-9b56-319aadfb1caa None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:44.695933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1334962-a30c-4247-9265-9eab44e4f135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:44.697021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334962-a30c-4247-9265-9eab44e4f135 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.697522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334962-a30c-4247-9265-9eab44e4f135 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.698000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334962-a30c-4247-9265-9eab44e4f135 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.699000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1334962-a30c-4247-9265-9eab44e4f135 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:44.699535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1334962-a30c-4247-9265-9eab44e4f135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:44.701264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1334962-a30c-4247-9265-9eab44e4f135 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:44.712045 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 452/887] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 27 09:40:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:44.718469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30a6ee-a09d-4a06-865d-142d621fa3bb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:44.762013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d2c85-6681-4728-82a7-356e254e8dfa None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.762352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d2c85-6681-4728-82a7-356e254e8dfa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.762576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d2c85-6681-4728-82a7-356e254e8dfa None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.764229 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 453/888] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:44.770293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ac22-de28-41c1-8f44-be385707c772 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.770693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ac22-de28-41c1-8f44-be385707c772 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.770919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ac22-de28-41c1-8f44-be385707c772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.792552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3173ac22-de28-41c1-8f44-be385707c772 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:44.793815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3173ac22-de28-41c1-8f44-be385707c772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:44.809083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:44.809520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:44.849268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30a6ee-a09d-4a06-865d-142d621fa3bb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:44.889955 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 454/889] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:40:44.891377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:44.892047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:44.892324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:44.892598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:44.909473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:44.910988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:44.928397 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:44.929454 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:44.930353 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:44.930353 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:44.930353 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:44.930524 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 436/890] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:44.936100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173ac22-de28-41c1-8f44-be385707c772 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:44.991243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.063004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173ac22-de28-41c1-8f44-be385707c772 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.106064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.107047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.107578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.108056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.108971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:45.109485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.109838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced14257-c2f3-470d-a2f3-e188c412eda8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.111398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.122207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:45.126289 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 455/891] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:45.132779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.141363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:45.157844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.158493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.158773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.159047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.159978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.162237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:45.173287 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:45.174325 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:45.175182 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:45.179629 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 437/892] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:45.181151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.252940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb9643-e40d-42d5-a4a2-687efffd5f5b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.296190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.296887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.297168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.297437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.297689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da3231-d4b7-4220-84ac-492104a0063b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.305853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.307307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:45.319971 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:45.321228 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 456/893] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:45.322619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.338716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.339337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.339614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.339884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.340519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:45.340858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.342231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.353107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 438/894] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:45.353107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.395414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:45.406805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:45.440029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef09652-dc1b-490e-b9a6-8fbbb705d822 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.460447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fd9e2-5809-4d14-ac9c-c347143cf97a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.478469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.479089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.479367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.479636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.480291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.482103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.497216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.497848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.498122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.498394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.500909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:45.501062 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 457/895] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:40:45 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:45.501513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.501608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.501680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.527807 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 439/896] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:45.528977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.664780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bb8ab-e622-49af-9302-daa782e5cf76 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.668687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc3415-e34c-4493-8bfd-8eeeb2237a09 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.676097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:45.691863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:45.716853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.717651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.717940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.718220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.721369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.722561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.723066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.723536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.724387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:45.724944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.726742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.732993 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 458/897] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:45.739894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.809236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.811397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.870800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf9c4-7e5e-4dc3-a4a2-17da293f2870 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:45.913817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:45.914436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:45.914714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:45.914988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:45.915692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] RBAC: Adding query filter params (interface=public, service_id=c57860bd9ab44017a7292998503eea05, region_id=RegionOne) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:45.915999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:45.917416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:45.926499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 459/898] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/endpoints?service_id=c57860bd9ab44017a7292998503eea05&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:45.932206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.947582 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 440/899] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:40:45.952960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:45.963736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:45.988272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:46.046884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2df3a-53f6-4f92-a244-47ccd995eb9d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:46.071121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa1c1e-83c5-4b70-9ba5-f5f5973028b9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:46.084166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:46.084758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:46.085140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:46.085513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:46.086393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:46.087708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:46.099792 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 460/900] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:40:46.115275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:46.116195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:46.116620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:46.117083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:46.131604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:46.133521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:46.151003 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:46.152244 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:46.153368 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:46.153368 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:46.153368 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:46.155260 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 441/901] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:46.161437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:46.281642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee8131-3347-4926-ade8-655da77c8062 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:46.293287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:46.327134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:46.327780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:46.328065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:46.328330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:46.329048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:46.329353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:46.330791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:46.345785 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 461/902] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:46.352041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:46.417081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:46.464620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcd588-d59c-4537-8d83-17e7c6e1f632 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:46.499611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:46.500256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:46.500535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:46.500839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:46.508129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:46.509685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:46.523075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:46.524041 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:46.525408 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 442/903] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:46.531069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:46.648927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7fc5-716f-4ed8-a655-bc9133b5aaf7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:46.684873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95223ec8-2cde-42f7-aeb0-5b7e739113e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:46.685878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95223ec8-2cde-42f7-aeb0-5b7e739113e4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:46.686372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95223ec8-2cde-42f7-aeb0-5b7e739113e4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:46.686836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95223ec8-2cde-42f7-aeb0-5b7e739113e4 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:46.687726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95223ec8-2cde-42f7-aeb0-5b7e739113e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:46.688237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95223ec8-2cde-42f7-aeb0-5b7e739113e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:46.689875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95223ec8-2cde-42f7-aeb0-5b7e739113e4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:46.703222 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 462/904] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:46.709393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:46.828678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc434d98-f77b-497b-b8b6-adfa7f8b09f6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:46.849467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:46.869942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ec94c-4ba2-41fa-8f84-5830858076a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:46.870788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ec94c-4ba2-41fa-8f84-5830858076a7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:46.871203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ec94c-4ba2-41fa-8f84-5830858076a7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:46.871607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ec94c-4ba2-41fa-8f84-5830858076a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:46.872471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ec94c-4ba2-41fa-8f84-5830858076a7 None admin] RBAC: Adding query filter params (role.id=e851bb42f132413f967ccd21e7bc8f37, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=9afc5552272c4928be73cb202945c8ba) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:46.872962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ec94c-4ba2-41fa-8f84-5830858076a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:46.874563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ec94c-4ba2-41fa-8f84-5830858076a7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:46.881186 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 443/905] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:46 2022] GET /identity/v3/role_assignments?user.id=9afc5552272c4928be73cb202945c8ba&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=e851bb42f132413f967ccd21e7bc8f37 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:47.002332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691afa-ec4f-4b21-b8bb-c41f9d6178fd None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:47.002809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691afa-ec4f-4b21-b8bb-c41f9d6178fd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:47.003180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691afa-ec4f-4b21-b8bb-c41f9d6178fd None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:47.005179 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 463/906] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:47.011204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e58d-2c57-435e-8697-00afbf6c5c1b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:47.011635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e58d-2c57-435e-8697-00afbf6c5c1b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:47.012018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e58d-2c57-435e-8697-00afbf6c5c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:47.036727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3889e58d-2c57-435e-8697-00afbf6c5c1b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:47.038019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3889e58d-2c57-435e-8697-00afbf6c5c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:47.119836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 444/907] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:47.446753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691afa-ec4f-4b21-b8bb-c41f9d6178fd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:47.480011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:47.565270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691afa-ec4f-4b21-b8bb-c41f9d6178fd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:47.606776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:47.606776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:47.606776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:47.606776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:47.607253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:47.607338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:47.617117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:47.618296 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:47.619295 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:47.623326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 464/908] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Sep 27 09:40:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:47.625269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889e58d-2c57-435e-8697-00afbf6c5c1b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:47.737109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889e58d-2c57-435e-8697-00afbf6c5c1b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:47.754359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:47.781025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:47.782002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:47.782426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:47.782840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:47.783610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:47.784062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:47.786621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:47.792560 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 445/909] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:47.800692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:47.897182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:47.905973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fab2b-c026-4791-a8f0-6a11af22c3ba None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:47.941960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:47.942657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:47.942934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:47.943203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:47.943840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:47.944138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:47.945540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:47.954804 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 465/910] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:47.961824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:48.074041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8aa36-41df-4eb4-8d44-93f8997dc62a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:48.111673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d0e4b-40f8-4dc3-9a8e-2903a1d8eff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:48.112300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d0e4b-40f8-4dc3-9a8e-2903a1d8eff7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.112579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d0e4b-40f8-4dc3-9a8e-2903a1d8eff7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.113273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d0e4b-40f8-4dc3-9a8e-2903a1d8eff7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.113815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d0e4b-40f8-4dc3-9a8e-2903a1d8eff7 None admin] RBAC: Adding query filter params (interface=public, service_id=f4234abd43d14b2da0299f53ad1ec281, region_id=RegionOne) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:48.114124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d0e4b-40f8-4dc3-9a8e-2903a1d8eff7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:48.115585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d0e4b-40f8-4dc3-9a8e-2903a1d8eff7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:48.121855 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 446/911] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:40:47 2022] GET /identity/v3/endpoints?service_id=f4234abd43d14b2da0299f53ad1ec281&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:48.128652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:48.208933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207777-f054-42c8-937e-f5d220fa9377 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.209329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207777-f054-42c8-937e-f5d220fa9377 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.209504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207777-f054-42c8-937e-f5d220fa9377 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.211106 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 447/912] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:48.218369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed474e9-c15c-4964-b45f-0c081886b7b2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.218791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed474e9-c15c-4964-b45f-0c081886b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.219178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed474e9-c15c-4964-b45f-0c081886b7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.236559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f5bea-f232-4319-abaa-44792d0e4430 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:48.240605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3ed474e9-c15c-4964-b45f-0c081886b7b2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:48.241993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3ed474e9-c15c-4964-b45f-0c081886b7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:48.272721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:48.273339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.273614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.273884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.274490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:48.276690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:48.281219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:48.283766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 466/913] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:40:48 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:40:48.325888 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 448/914] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:40:48.656742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:48.681604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:48.767980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ad8be-0833-4692-8cd8-fc4a978eff17 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:48.777409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30830ce-983c-419a-9b60-4ecbcb3bab8b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.777913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30830ce-983c-419a-9b60-4ecbcb3bab8b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.777988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30830ce-983c-419a-9b60-4ecbcb3bab8b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.779708 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 449/915] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:40:48.785533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09820d-28d8-4807-a6b9-6b778ae4c2b1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.785802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09820d-28d8-4807-a6b9-6b778ae4c2b1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.786076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09820d-28d8-4807-a6b9-6b778ae4c2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.805539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5e09820d-28d8-4807-a6b9-6b778ae4c2b1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:48.806726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e09820d-28d8-4807-a6b9-6b778ae4c2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:48.806940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:48.807525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.807798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.808066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.808823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:48.810297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:48.820174 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:48.821428 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:48.822257 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 467/916] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:48.827750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:48.899190 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 450/917] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:40:48.945058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:48.950491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544032-202e-47d0-aa05-dab99dc85943 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:48.993853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:48.994495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:48.994769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:48.995041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:48.995688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:48.995988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:48.997430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:49.003386 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 468/918] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:49.009490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09820d-28d8-4807-a6b9-6b778ae4c2b1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:49.118566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:49.121591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09820d-28d8-4807-a6b9-6b778ae4c2b1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:49.160419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:49.160922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.161543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.161873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.162517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:49.162857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:49.164535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:49.170383 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 451/919] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:49.176220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:49.238500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:49.287727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599727-7e35-4a29-a240-ca74c56b0d03 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:49.336895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:49.337518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.337792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.338064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.338770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] RBAC: Adding query filter params (interface=public, service_id=ff6364b0d1c843deb1979b57f2182ca8, region_id=RegionOne) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:49.339074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:49.340460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:49.351183 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 469/920] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:40:49 2022] GET /identity/v3/endpoints?service_id=ff6364b0d1c843deb1979b57f2182ca8&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:49.352156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:49.368241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:49.376728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a14a9-504b-48bf-ba83-08083293a017 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:49.420947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:49.421991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.422552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.423127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.423962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:49.425663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:49.436327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:49.437432 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 09:40:49.438262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:49.447224 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 452/921] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:49.448747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:49.468392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f109a85-b205-4338-bd77-c21a5b75f1e1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:49.482891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:49.505385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706af3-0ebf-4fff-9c05-ada9e5c08490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:49.506022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706af3-0ebf-4fff-9c05-ada9e5c08490 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.506287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706af3-0ebf-4fff-9c05-ada9e5c08490 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.506538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706af3-0ebf-4fff-9c05-ada9e5c08490 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.507184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706af3-0ebf-4fff-9c05-ada9e5c08490 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:49.508555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706af3-0ebf-4fff-9c05-ada9e5c08490 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:49.516723 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 470/922] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:40:49 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:40:49.516929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ea792-dc78-4a2a-9a19-283d11883813 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.517171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ea792-dc78-4a2a-9a19-283d11883813 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.517436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ea792-dc78-4a2a-9a19-283d11883813 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.518673 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 471/923] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:49.531140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb24f8-2795-4fa1-9f43-feef23e1a13e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.531415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb24f8-2795-4fa1-9f43-feef23e1a13e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.531686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb24f8-2795-4fa1-9f43-feef23e1a13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.554488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-36bb24f8-2795-4fa1-9f43-feef23e1a13e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:49.555998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36bb24f8-2795-4fa1-9f43-feef23e1a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:49.576322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cd382-d5fa-48bf-8e7d-bad1f5ad01e6 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:49.640026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:49.641377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.641868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.642634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.644196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:49.645067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:49.647597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:49.654357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 453/924] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:49.656710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 472/925] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:40:49.666188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:49.779538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:49.790263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65b87-0853-49ca-b68d-88dc3a607c29 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:49.825963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:49.826578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.826853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.827120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.827819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:49.828123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:49.829569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:49.835679 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 454/926] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Sep 27 09:40:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:49.841443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb24f8-2795-4fa1-9f43-feef23e1a13e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:49.853669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:49.952084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb24f8-2795-4fa1-9f43-feef23e1a13e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:49.989252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:49.989875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:49.990155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:49.990426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:49.991103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:49.992488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:50.016589 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 473/927] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:40:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:40:50.020234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:50.155580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13affea-8b6c-447b-ac1b-520ca11b6852 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:50.209061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:50.209803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:50.210082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:50.210350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:50.223033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:50.228940 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:50.229975 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:50.231503 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 455/928] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 27 09:40:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:50.237863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:50.317626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:50.377429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1fec3-0e9c-4069-a72a-d70918b3c685 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:50.417493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:50.418027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:50.418306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:50.418577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:50.419258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:50.419560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:50.420968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:50.431282 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 474/929] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 27 09:40:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:50.437210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:50.454128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:50.554548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997725f-8ad7-42f0-b826-92781745bfa1 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:50.596084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:50.597144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:50.597685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:50.598306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:50.610212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:50.612040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:50.628303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:50.629611 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:50.630500 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:50.630500 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:50.630500 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:50.630500 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 456/930] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:50.635835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:50.767668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772d9d-60d7-47e7-8756-922c71d6e6f1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:50.793573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:50.807755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e42b6-200e-44a8-a51b-4ccd1f4722de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:50.808385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e42b6-200e-44a8-a51b-4ccd1f4722de None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:50.808792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e42b6-200e-44a8-a51b-4ccd1f4722de None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:50.809070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e42b6-200e-44a8-a51b-4ccd1f4722de None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:50.809942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e42b6-200e-44a8-a51b-4ccd1f4722de None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:50.810452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e42b6-200e-44a8-a51b-4ccd1f4722de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:50.811921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e42b6-200e-44a8-a51b-4ccd1f4722de None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:50.826040 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 475/931] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:50.832482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:50.904332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:50.935134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7eb76-4508-4a20-97af-65316bc9e027 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:50.936172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7eb76-4508-4a20-97af-65316bc9e027 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:50.936646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7eb76-4508-4a20-97af-65316bc9e027 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:50.938375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 476/932] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:50.944434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d4be1-6b29-46e3-b2c5-93baaec5077d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:50.944785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d4be1-6b29-46e3-b2c5-93baaec5077d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:50.945012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d4be1-6b29-46e3-b2c5-93baaec5077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:50.954513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4da1e-f5c7-4f20-9809-afb5ac2cbdd9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:50.966305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8a9d4be1-6b29-46e3-b2c5-93baaec5077d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:50.967528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8a9d4be1-6b29-46e3-b2c5-93baaec5077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:51.006510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.008031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.008614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.009138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.021189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.023722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:51.040261 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:51.041355 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:51.042418 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 457/933] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:40:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:51.047058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.063017 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 477/934] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:51.171830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76dc5-332e-4952-a078-3349dc30a7fe None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.199255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:51.215421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.216061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.216338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.216602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.217395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:51.217692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.219269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.241397 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 458/935] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:51.247368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d4be1-6b29-46e3-b2c5-93baaec5077d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.368098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d4be1-6b29-46e3-b2c5-93baaec5077d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.397425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:51.400092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.415364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.416423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.416936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.417429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.429813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.431644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:51.447633 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:51.448689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:51.449611 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:51.449611 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:51.449611 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:51.451825 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 478/936] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:51.460012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.522135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c7eb4-1dc0-4490-b8c7-9e7126fadb3b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.562793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.563448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.563726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.563985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.564572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.565992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.566578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2fa52-45c2-403e-8bad-336404ccd7c7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:51.576699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:51.577722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:51.578602 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 459/937] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:51.586369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.612340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.613129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.613404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.613669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.614303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:51.614602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.615968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.634683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 479/938] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:51.635560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.654539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:51.672073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:51.714460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb2a1e-2cea-4028-94ec-45d29c406aaa None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.752452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.754309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.754309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.754309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.754894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:51.755274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.756754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.766917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 460/939] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:51.768531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.771646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e3a6a-e542-4e32-8481-2b0817faae0c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.815095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.815734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.816005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.816338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.823980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.825447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:51.837662 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:51.838709 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:51.840139 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:51.843685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 480/940] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:51.844587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:51.926358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbbd8a-e2a7-45a4-ae39-7741383fb8b0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.937988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:51.943349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:51.965491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d469f1-56d7-4cb4-a797-a29d83abe27b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:51.975936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:51.976560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:51.976883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:51.977155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:51.977791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:51.978094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:51.979479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:51.985068 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 461/941] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:40:51.993312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:52.005473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:52.006595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.006879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.007149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.007844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:52.008145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:52.010102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:52.025329 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 481/942] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:52.031363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:52.108772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf6efe-b910-47dc-852e-ded461197cd9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:52.145552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca8b11-c80e-4f1c-95e4-6924aa281543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:52.146590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8b11-c80e-4f1c-95e4-6924aa281543 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.147139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8b11-c80e-4f1c-95e4-6924aa281543 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.147608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8b11-c80e-4f1c-95e4-6924aa281543 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.148588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca8b11-c80e-4f1c-95e4-6924aa281543 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:52.150466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca8b11-c80e-4f1c-95e4-6924aa281543 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:52.156302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa0f84-3b44-4c83-833a-79a10774b8b2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:52.180060 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 462/943] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:40:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:52.218441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9aa11-d5bd-408b-8731-be08d169fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:52.219119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9aa11-d5bd-408b-8731-be08d169fc43 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.219393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9aa11-d5bd-408b-8731-be08d169fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.219659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9aa11-d5bd-408b-8731-be08d169fc43 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/9afc5552272c4928be73cb202945c8ba/roles/e851bb42f132413f967ccd21e7bc8f37` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.222026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9aa11-d5bd-408b-8731-be08d169fc43 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=9afc5552272c4928be73cb202945c8ba, role_id=e851bb42f132413f967ccd21e7bc8f37)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:52.223881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9aa11-d5bd-408b-8731-be08d169fc43 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:52.248871 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 482/944] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:40:52 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/9afc5552272c4928be73cb202945c8ba/roles/e851bb42f132413f967ccd21e7bc8f37 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:40:52.272508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3ac26-9351-4c45-80e2-60bb3f48c913 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.273222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3ac26-9351-4c45-80e2-60bb3f48c913 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.273698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3ac26-9351-4c45-80e2-60bb3f48c913 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.276964 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 463/945] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:52.282364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513e8b2-7250-4f32-a240-02f7f434669c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.282697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513e8b2-7250-4f32-a240-02f7f434669c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.282912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513e8b2-7250-4f32-a240-02f7f434669c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.290629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:52.307778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d513e8b2-7250-4f32-a240-02f7f434669c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:52.309329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d513e8b2-7250-4f32-a240-02f7f434669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:52.403291 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 483/946] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:52.631943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff324b-7eea-41c9-902f-bc36480d8440 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.632287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff324b-7eea-41c9-902f-bc36480d8440 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.632494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff324b-7eea-41c9-902f-bc36480d8440 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.634147 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 464/947] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:52.639611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe04125-1b50-42bb-b52f-6c855467065d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.639886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe04125-1b50-42bb-b52f-6c855467065d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.640162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe04125-1b50-42bb-b52f-6c855467065d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.659424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7fe04125-1b50-42bb-b52f-6c855467065d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:52.660910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7fe04125-1b50-42bb-b52f-6c855467065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:52.735155 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 484/948] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:52.741534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff324b-7eea-41c9-902f-bc36480d8440 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:52.796300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:52.866450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff324b-7eea-41c9-902f-bc36480d8440 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:52.904037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:52.905018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:52.905444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:52.905861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:52.906747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:52.908278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:52.918207 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:52.919448 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:52.920544 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:52.922965 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 465/949] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:52.929686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe04125-1b50-42bb-b52f-6c855467065d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:52.993616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:53.034514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe04125-1b50-42bb-b52f-6c855467065d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:53.070649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:53.071279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:53.071556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:53.071823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:53.072461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:53.072793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:53.074169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:53.079665 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 485/950] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:53.085693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:53.089624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:53.197779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d7f98-aff2-44ec-b14d-ad01f3f625a5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:53.206705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:53.217983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb830d-feb4-4a80-aedb-d60c499cb363 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:53.226806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:53.236498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:53.237397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:53.237875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:53.238323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:53.239195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:53.240905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:53.243803 ncdv-master-1 devstack@keystone.service[104132]: WARNING keystone.server.flask.common [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 27 09:40:53.265303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.sql.core [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 27 09:40:53.265303 ncdv-master-1 devstack@keystone.service[104132]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 09:40:53.265303 ncdv-master-1 devstack@keystone.service[104132]: [parameters: {'id': '3964d03481d94294bc6ee6d83b0f55ee', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 27 09:40:53.265303 ncdv-master-1 devstack@keystone.service[104132]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104132) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 09:40:53.266547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:53.267186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:53.267460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:53.267725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:53.268492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:53.268832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:53.270568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:53.276869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 466/951] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3964d03481d94294bc6ee6d83b0f55ee.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3964d03481d94294bc6ee6d83b0f55ee. Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application conn.query(q) Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:40:53.289055 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result.read() Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 27 09:40:53.290257 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application next(self.gen) Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application yield resource Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application next(self.gen) Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise self.value Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application session.commit() Sep 27 09:40:53.291772 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.session.flush() Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application compat.raise_( Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 09:40:53.292719 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 09:40:53.293661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application conn.query(q) Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result.read() Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:40:53.294459 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [parameters: {'id': '3964d03481d94294bc6ee6d83b0f55ee', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:53.296510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:53.298029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3964d03481d94294bc6ee6d83b0f55ee. Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 486/952] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Sep 27 09:40:53 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:53.299974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:53.302694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:53.420302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa22170-0086-43d1-ab88-66857692b8e7 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:53.420562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93032-718e-40c2-bfd6-c6505ffad032 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:53.464898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d25390-455e-4536-aaea-5e5368c1d534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:53.465893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:53.466142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d25390-455e-4536-aaea-5e5368c1d534 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:53.466673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d25390-455e-4536-aaea-5e5368c1d534 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:53.466903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:53.466988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:53.467317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:53.467631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d25390-455e-4536-aaea-5e5368c1d534 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:53.468696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d25390-455e-4536-aaea-5e5368c1d534 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:53.470561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d25390-455e-4536-aaea-5e5368c1d534 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:53.475015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:53.477267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:53.490547 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:53.491589 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:53.492591 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:53.496730 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 487/953] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:53.497726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:53.505129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 467/954] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:40:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:40:53.555760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:53.627231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda234b-7179-4b7b-974d-192c35bb703b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:53.676508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d57d3-5850-4075-acc2-4f9c249c892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:53.677935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d57d3-5850-4075-acc2-4f9c249c892b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:53.678223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d57d3-5850-4075-acc2-4f9c249c892b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:53.678473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d57d3-5850-4075-acc2-4f9c249c892b None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:53.679159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d57d3-5850-4075-acc2-4f9c249c892b None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:53.679460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d57d3-5850-4075-acc2-4f9c249c892b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:53.680949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d57d3-5850-4075-acc2-4f9c249c892b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:53.703330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 488/955] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:54.898068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3abd20-26cf-4d74-acbe-97290f2e4967 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:54.898585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3abd20-26cf-4d74-acbe-97290f2e4967 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:54.898585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3abd20-26cf-4d74-acbe-97290f2e4967 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:54.900155 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 468/956] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:54.906609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81101c-d16b-48b9-81cc-f706aa2776ff None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:54.907736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81101c-d16b-48b9-81cc-f706aa2776ff None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:54.908217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81101c-d16b-48b9-81cc-f706aa2776ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:54.928921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ff81101c-d16b-48b9-81cc-f706aa2776ff None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:54.930863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ff81101c-d16b-48b9-81cc-f706aa2776ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:55.009732 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 489/957] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:55.117845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e8bd5-fc41-44f1-8d8f-d9ad95d5dba4 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:55.118662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e8bd5-fc41-44f1-8d8f-d9ad95d5dba4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:55.119326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e8bd5-fc41-44f1-8d8f-d9ad95d5dba4 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:55.122363 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 469/958] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:55.134461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5af4-7e45-45a0-8ade-7385c0d91409 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:55.134768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5af4-7e45-45a0-8ade-7385c0d91409 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:55.135106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5af4-7e45-45a0-8ade-7385c0d91409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:55.158137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-460e5af4-7e45-45a0-8ade-7385c0d91409 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:55.159281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-460e5af4-7e45-45a0-8ade-7385c0d91409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:55.188470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:55.240685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 490/959] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:55.397774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e8bd5-fc41-44f1-8d8f-d9ad95d5dba4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:55.419493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:55.552529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e8bd5-fc41-44f1-8d8f-d9ad95d5dba4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:55.599139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:55.599872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:55.600250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:55.600533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] PATH_INFO: `/v3/services/compute` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:55.601172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:55.602593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:55.613677 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:55.615366 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:55.616236 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 470/960] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:55.620780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e5af4-7e45-45a0-8ade-7385c0d91409 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:55.624800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:55.725181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:55.750117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e5af4-7e45-45a0-8ade-7385c0d91409 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:55.755508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd5b0-a6f7-4e78-ad7d-722218e334ab None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:55.791749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:55.792403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:55.792700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:55.792967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:55.804415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:55.805191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:55.805538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:55.805899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:55.806268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:40:55.806674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] RBAC: Adding query filter params (name=compute) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:55.807041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:55.808514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:40:55.812563 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:55.813853 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:40:55.814945 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:55.819415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 471/961] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:55.820390 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 491/962] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 27 09:40:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:40:55.822144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:55.829553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:55.860961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:55.980808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdba98-46b2-4cd2-9fcb-e7a50c47079c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:55.999661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57011af-2c6e-413d-9f6f-9200d433a023 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:56.034415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:56.035629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.036122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.036975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.037916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:56.038432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:56.040165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:56.051355 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 472/963] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 27 09:40:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:56.058338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:56.060819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692b95c-91f9-4b52-ab38-bc803d5d41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:56.061471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692b95c-91f9-4b52-ab38-bc803d5d41cd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.061777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692b95c-91f9-4b52-ab38-bc803d5d41cd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.062048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692b95c-91f9-4b52-ab38-bc803d5d41cd None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.062713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692b95c-91f9-4b52-ab38-bc803d5d41cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:56.063016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692b95c-91f9-4b52-ab38-bc803d5d41cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:56.065494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692b95c-91f9-4b52-ab38-bc803d5d41cd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:56.077586 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 492/964] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:40:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:56.077800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:56.232398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a47a2-dc5e-4e24-9d4f-1e82a644b112 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.232701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a47a2-dc5e-4e24-9d4f-1e82a644b112 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.232860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a47a2-dc5e-4e24-9d4f-1e82a644b112 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.234527 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 493/965] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:56.234869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda78e9-6727-4039-ac1b-5262d5188120 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:56.241764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb375c-21d8-48a4-8f60-9f059dedfd86 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.242343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb375c-21d8-48a4-8f60-9f059dedfd86 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.242931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb375c-21d8-48a4-8f60-9f059dedfd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.268002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6ffb375c-21d8-48a4-8f60-9f059dedfd86 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:56.269968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6ffb375c-21d8-48a4-8f60-9f059dedfd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:56.292192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:56.292902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.293213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.293629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.314713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:56.316849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:56.340815 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:56.341852 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:56.342779 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:56.342779 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:56.342779 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:56.344576 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 473/966] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:56.353815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:56.399922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 494/967] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:40:56.441133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:56.512521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ba5d8-6f41-42b5-8c64-f9f9411e90da None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:56.572062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:56.572729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.573010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.573298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.573984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:56.574285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:56.575754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:56.586233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad205a80-f053-4189-9fb7-e065d5ae337c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.586603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad205a80-f053-4189-9fb7-e065d5ae337c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.586774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad205a80-f053-4189-9fb7-e065d5ae337c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.588435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 495/968] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:40:56.590715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 474/969] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:56.594842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7c19c-e72d-4877-8cff-eccb75989983 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.595412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7c19c-e72d-4877-8cff-eccb75989983 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.595868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7c19c-e72d-4877-8cff-eccb75989983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.598608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:56.629717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5ce7c19c-e72d-4877-8cff-eccb75989983 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:56.631060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5ce7c19c-e72d-4877-8cff-eccb75989983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:56.640430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:56.717932 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 496/970] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:56.737645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dddb1-f43f-43be-9498-356014fde64e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:56.755780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:56.775866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:56.776290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7c19c-e72d-4877-8cff-eccb75989983 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:56.776556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.777200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.777482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.788045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:56.789633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:56.802912 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:56.803956 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:56.804992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 475/971] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:40:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:56.813872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:56.895393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7c19c-e72d-4877-8cff-eccb75989983 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:56.938321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:56.939078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.939338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:56.939739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:56.940386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:56.942782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:56.946633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd490-c641-4e19-b874-4cae3a7721d5 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:56.953361 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:56.954473 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:56.955315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 497/972] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:40:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:56.960863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:56.973100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:56.997337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:56.998931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:56.999625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.000070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.001710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:57.002199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.004265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.037286 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 476/973] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Sep 27 09:40:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:57.038553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.075507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:57.091374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156ce04-dbd1-4478-9e20-85dc808d0827 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.127615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.128222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.128699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.129053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.129695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:57.129987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.131362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.136165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 498/974] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:57.137042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.159048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64a4e4-2496-4c77-86e6-666836e5305a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.198579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.199214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.199493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.199753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.225719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:57.259979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f36603-0940-446d-89b3-a654d95904b0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.281573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.283488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.299928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.300507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.300806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.301074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.301725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.303168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.323276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.sql.core [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 27 09:40:57.323276 ncdv-master-1 devstack@keystone.service[104132]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 27 09:40:57.323276 ncdv-master-1 devstack@keystone.service[104132]: [parameters: {'user_id': 'ccdd88628ac94157a4787a3b4db74e62', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 27 09:40:57.323276 ncdv-master-1 devstack@keystone.service[104132]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104132) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 09:40:57.329235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application conn.query(q) Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result.read() Sep 27 09:40:57.331172 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 09:40:57.332788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application next(self.gen) Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application yield resource Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application next(self.gen) Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise self.value Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application session.commit() Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 09:40:57.334267 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.session.flush() Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application compat.raise_( Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 09:40:57.335715 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:40:57.336905 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application conn.query(q) Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application result.read() Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:40:57.337855 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ccdd88628ac94157a4787a3b4db74e62', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:57.338710 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 27 09:40:57.339575 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 499/975] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Sep 27 09:40:57 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:57.340526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.413736 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 477/976] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:40:57.419118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.452023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f60e6-8837-43b4-b357-db68bc3a63a6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.484483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:57.490564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.491487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.491853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.492331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.493153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:57.493721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.495596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.509187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 500/977] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:57.509187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.533448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98027a82-dacd-4192-acb9-a526586b07b9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.570189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.570744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.571017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.571283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.573927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:57.582050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.583620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:40:57.596143 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:57.597485 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 09:40:57.598710 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:57.599894 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 478/978] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:57.600795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.635609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90486b-dc87-45d9-a7ec-07c9b039f89f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.674865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.675563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.675947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.676269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.687926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.689415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:40:57.705700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:57.706809 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:40:57.707925 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:40:57.707925 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:40:57.707925 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:40:57.707925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 501/979] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:57.707925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.720248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f277ff7-c13a-489c-9b3c-84267166e6d1 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.753896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.754515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.754788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.755054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.755760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] RBAC: Adding query filter params (interface=admin, service_id=ff6364b0d1c843deb1979b57f2182ca8, region_id=RegionOne) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:57.756059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.757455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.762687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:57.762989 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 479/980] 10.222.0.26 () {56 vars in 1249 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/endpoints?service_id=ff6364b0d1c843deb1979b57f2182ca8&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:57.763614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.824023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:57.828091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e780bd-7848-4ef7-9a57-1ae4885545fd None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.870033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.870966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.871303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.871574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.872433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:57.872982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.876086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.881286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e38b-1a4b-43f9-8038-25b46ed22c7a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:57.910703 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 502/981] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:57.911910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:57.939248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:57.939886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:57.940159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:57.940422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:57.943026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:57.943026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:57.943026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:57.962613 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 480/982] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:57.963659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:58.058577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a4f77-024c-416a-89af-f904ce9b3242 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:58.069468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:58.082532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ca1d-8504-4155-a943-1f37ca6cb03d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:58.103628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:58.110274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:58.112351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:58.113629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:58.114089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:58.115178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:58.116933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:58.125806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:58.126313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 503/983] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/services => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:40:58.132716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:58.132716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:58.132716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:58.141932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:58.143958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:40:58.158156 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:40:58.159154 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:40:58.160094 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:40:58.163546 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 481/984] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:40:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:40:58.168557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:58.298533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cba95-baaf-4033-8ef8-dd93607dcc7d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:58.343198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad468f70-60aa-42f0-b697-f41060d1e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:58.344064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad468f70-60aa-42f0-b697-f41060d1e981 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:58.344630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad468f70-60aa-42f0-b697-f41060d1e981 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:58.345170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad468f70-60aa-42f0-b697-f41060d1e981 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:58.346055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad468f70-60aa-42f0-b697-f41060d1e981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:58.346513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad468f70-60aa-42f0-b697-f41060d1e981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:58.348065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad468f70-60aa-42f0-b697-f41060d1e981 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:58.361445 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 504/985] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:58.367467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:58.481209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efdcd1-bf37-48cf-b20b-ceb61aa9ddc3 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:58.531335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6f9bb-519b-4b8d-af8c-ae57ec53e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:58.532350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f9bb-519b-4b8d-af8c-ae57ec53e20e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:58.532864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f9bb-519b-4b8d-af8c-ae57ec53e20e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:58.533341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f9bb-519b-4b8d-af8c-ae57ec53e20e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:58.534595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6f9bb-519b-4b8d-af8c-ae57ec53e20e None admin] RBAC: Adding query filter params (role.id=e851bb42f132413f967ccd21e7bc8f37, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=9afc5552272c4928be73cb202945c8ba) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:40:58.535171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6f9bb-519b-4b8d-af8c-ae57ec53e20e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:58.538355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6f9bb-519b-4b8d-af8c-ae57ec53e20e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:58.552699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 482/986] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:40:58 2022] GET /identity/v3/role_assignments?user.id=9afc5552272c4928be73cb202945c8ba&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=e851bb42f132413f967ccd21e7bc8f37 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:40:58.727716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210b02e-bab9-4218-ad8e-8fc21ce25c82 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:58.728116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210b02e-bab9-4218-ad8e-8fc21ce25c82 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:58.728478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210b02e-bab9-4218-ad8e-8fc21ce25c82 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:58.730268 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 505/987] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:40:58.736377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855bfcb-e263-4296-b5fe-ef5f240596f6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:58.736862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855bfcb-e263-4296-b5fe-ef5f240596f6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:58.737238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855bfcb-e263-4296-b5fe-ef5f240596f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:58.758594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1855bfcb-e263-4296-b5fe-ef5f240596f6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:40:58.759852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1855bfcb-e263-4296-b5fe-ef5f240596f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:40:58.807884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:58.846988 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 483/988] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:40:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:40:59.174863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210b02e-bab9-4218-ad8e-8fc21ce25c82 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:40:59.214757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:40:59.316113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210b02e-bab9-4218-ad8e-8fc21ce25c82 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:40:59.365208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36308b20-7cde-4709-bf57-fa4952f9d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:40:59.365860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308b20-7cde-4709-bf57-fa4952f9d4b7 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:40:59.366140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308b20-7cde-4709-bf57-fa4952f9d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:40:59.366416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308b20-7cde-4709-bf57-fa4952f9d4b7 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:40:59.367116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36308b20-7cde-4709-bf57-fa4952f9d4b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:40:59.368571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36308b20-7cde-4709-bf57-fa4952f9d4b7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:40:59.395512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 506/989] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:40:59 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:41:00.832148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2d29-f747-49dc-b81c-e0d297b0e816 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:00.832987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2d29-f747-49dc-b81c-e0d297b0e816 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:00.832987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2d29-f747-49dc-b81c-e0d297b0e816 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:00.834301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 484/990] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:00.840887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86080fb-362c-410e-9fe2-0e1bc1979145 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:00.841474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86080fb-362c-410e-9fe2-0e1bc1979145 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:00.841474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86080fb-362c-410e-9fe2-0e1bc1979145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:00.864412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a86080fb-362c-410e-9fe2-0e1bc1979145 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:00.866295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a86080fb-362c-410e-9fe2-0e1bc1979145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:00.969235 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 507/991] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:01.201800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc308858-e90f-4b10-8e8f-cd95b9a72bb7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:01.202064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc308858-e90f-4b10-8e8f-cd95b9a72bb7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:01.202210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc308858-e90f-4b10-8e8f-cd95b9a72bb7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:01.203786 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 485/992] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:41:01.209605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a002-c191-45b5-b18f-4da749d0a574 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:01.209909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a002-c191-45b5-b18f-4da749d0a574 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:01.210158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a002-c191-45b5-b18f-4da749d0a574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:01.230272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4ae6a002-c191-45b5-b18f-4da749d0a574 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:01.231487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4ae6a002-c191-45b5-b18f-4da749d0a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:01.245089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:01.322416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 508/993] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:01.325847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc308858-e90f-4b10-8e8f-cd95b9a72bb7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:01.469797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc308858-e90f-4b10-8e8f-cd95b9a72bb7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:01.511791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:01.512438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:01.512767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:01.513044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:01.513649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:01.515032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:01.525123 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:01.526797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 09:41:01.528213 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:01.531543 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 486/994] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Sep 27 09:41:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:01.533355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a002-c191-45b5-b18f-4da749d0a574 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:01.683075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a002-c191-45b5-b18f-4da749d0a574 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:01.698328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:01.735211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:01.736408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:01.737128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:01.737844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:01.738165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:01.738859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:01.739192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:01.740776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:01.748841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 509/995] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:41:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:01.753856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:01.766917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:01.825863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ba6b-1463-4020-b104-e9ea78206360 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:01.870948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:01.872445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:01.872978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:01.873479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] PATH_INFO: `/v3/services/image` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:01.874264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:01.875821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:01.886574 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:01.887946 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:01.889095 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 487/996] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Sep 27 09:41:01 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:01.897766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:01.900079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d51af-0597-4453-a111-0349150a928c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:01.958562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:01.959597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:01.960091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:01.960543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:01.961681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:01.962210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:01.964215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:01.972016 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 510/997] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Sep 27 09:41:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:41:01.978875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:02.040058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:02.044047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011f3be-1006-406e-8132-20c3f08c7aba None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:02.080206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:02.084857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49987e7-b2ed-4885-b38b-82edf77da4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:02.085520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49987e7-b2ed-4885-b38b-82edf77da4de None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:02.085797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49987e7-b2ed-4885-b38b-82edf77da4de None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:02.086069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49987e7-b2ed-4885-b38b-82edf77da4de None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:02.086709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49987e7-b2ed-4885-b38b-82edf77da4de None admin] RBAC: Adding query filter params (name=image) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:02.087011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49987e7-b2ed-4885-b38b-82edf77da4de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:02.088459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49987e7-b2ed-4885-b38b-82edf77da4de None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:02.093731 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 488/998] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:02.095077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f41f-eb18-4904-a21d-8201dec7abe7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:02.095330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f41f-eb18-4904-a21d-8201dec7abe7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:02.095605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f41f-eb18-4904-a21d-8201dec7abe7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:02.097832 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 489/999] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:02.101466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36f41f-eb18-4904-a21d-8201dec7abe7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:02.110965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619a65-b401-4b4e-b8eb-5b6d34319716 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:02.148859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272c45c-bbeb-4c21-8b28-c52bd8fd655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:02.149488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c45c-bbeb-4c21-8b28-c52bd8fd655b None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:02.149760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c45c-bbeb-4c21-8b28-c52bd8fd655b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:02.150029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c45c-bbeb-4c21-8b28-c52bd8fd655b None admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:02.150708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272c45c-bbeb-4c21-8b28-c52bd8fd655b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:02.152101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272c45c-bbeb-4c21-8b28-c52bd8fd655b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:02.182433 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 511/1000] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:41:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:41:02.183913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03111cb3-73a7-4399-b09f-3ac176e45dcf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:02.184275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03111cb3-73a7-4399-b09f-3ac176e45dcf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:02.184614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03111cb3-73a7-4399-b09f-3ac176e45dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:02.205051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-03111cb3-73a7-4399-b09f-3ac176e45dcf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:02.206384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-03111cb3-73a7-4399-b09f-3ac176e45dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:02.249769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36f41f-eb18-4904-a21d-8201dec7abe7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:02.300758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 512/1001] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:02.314665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:02.315584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:02.316050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:02.316516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:02.317550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] RBAC: Adding query filter params (type=image) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:02.317986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:02.320049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:02.325598 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 490/1002] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:41:02.654128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03111cb3-73a7-4399-b09f-3ac176e45dcf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:02.746305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:02.795326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03111cb3-73a7-4399-b09f-3ac176e45dcf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:02.836475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:02.837166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:02.837418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:02.837674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] PATH_INFO: `/v3/services/compute` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:02.838294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:02.839703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:02.850430 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:02.851552 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:41:02.852496 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:02.855529 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 513/1003] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:41:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:02.857953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:02.891863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:02.980955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c577ce-7695-40bb-ad03-9e117de2ce55 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:03.027749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:03.028787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:03.029261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:03.029721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:03.030834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] RBAC: Adding query filter params (name=compute) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:03.031348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:03.033867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:03.048082 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 491/1004] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:41:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:03.049726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:03.160172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91dec9-ee40-40ba-a7d2-c5dd33ee4e12 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:03.187936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:03.197321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:03.198024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:03.198368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:03.198702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:03.199414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] RBAC: Adding query filter params (type=compute) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:03.199779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:03.201269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:03.206766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 514/1005] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:41:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:03.212970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:03.313782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:03.331741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f78f0-dddd-432b-bf00-4e92b53eaa33 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:03.375596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9bc4-09d8-415b-a397-acd8f0d92fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:03.376372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9bc4-09d8-415b-a397-acd8f0d92fb8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:03.376773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9bc4-09d8-415b-a397-acd8f0d92fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:03.377140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9bc4-09d8-415b-a397-acd8f0d92fb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:03.378084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a9bc4-09d8-415b-a397-acd8f0d92fb8 None admin] RBAC: Adding query filter params (interface=public, service_id=f7ae032b920e4f3eb690910d3f325a13, region_id=RegionOne) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:03.378512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a9bc4-09d8-415b-a397-acd8f0d92fb8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:03.380150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a9bc4-09d8-415b-a397-acd8f0d92fb8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:03.391425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 492/1006] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:41:03 2022] GET /identity/v3/endpoints?service_id=f7ae032b920e4f3eb690910d3f325a13&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:03.392467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:03.501082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6eff6-7ca7-45de-a029-6f349fcf861a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:03.548386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f663e36-5cf6-4286-9d3f-984e411c6dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:03.549176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663e36-5cf6-4286-9d3f-984e411c6dbf None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:03.549558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663e36-5cf6-4286-9d3f-984e411c6dbf None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:03.549919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663e36-5cf6-4286-9d3f-984e411c6dbf None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:03.550718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f663e36-5cf6-4286-9d3f-984e411c6dbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:03.552204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f663e36-5cf6-4286-9d3f-984e411c6dbf None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:03.563228 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 515/1007] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:41:03 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:41:04.715479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89747bd7-da0b-4b30-9a47-c832dbc874be None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:04.716044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89747bd7-da0b-4b30-9a47-c832dbc874be None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:04.716044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89747bd7-da0b-4b30-9a47-c832dbc874be None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:04.717743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 493/1008] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:04.724311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b274a-0b35-4cb5-88bd-384d932f5707 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:04.724311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b274a-0b35-4cb5-88bd-384d932f5707 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:04.725040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b274a-0b35-4cb5-88bd-384d932f5707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:04.744869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b38b274a-0b35-4cb5-88bd-384d932f5707 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:04.746085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b38b274a-0b35-4cb5-88bd-384d932f5707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:04.811037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:04.831531 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 516/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:04.831834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1e68-9ffd-41b2-b4b1-9aeb5acb9b00 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:04.832135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1e68-9ffd-41b2-b4b1-9aeb5acb9b00 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:04.832422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1e68-9ffd-41b2-b4b1-9aeb5acb9b00 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:04.834112 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 494/1010] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:04.839650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986281fa-bc84-4684-8d3f-6ea3182e5eb8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:04.839935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986281fa-bc84-4684-8d3f-6ea3182e5eb8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:04.840214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986281fa-bc84-4684-8d3f-6ea3182e5eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:04.859452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-986281fa-bc84-4684-8d3f-6ea3182e5eb8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:04.860758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-986281fa-bc84-4684-8d3f-6ea3182e5eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:04.936393 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 517/1011] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:41:05.156837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1e68-9ffd-41b2-b4b1-9aeb5acb9b00 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:05.264074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986281fa-bc84-4684-8d3f-6ea3182e5eb8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:05.346318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1e68-9ffd-41b2-b4b1-9aeb5acb9b00 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:05.399955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:05.401084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:05.401430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:05.401753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:05.402952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:05.404411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:05.410147 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 495/1012] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:05.413827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:05.416440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986281fa-bc84-4684-8d3f-6ea3182e5eb8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:05.439232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:05.444886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:05.466599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba6681-b146-486a-be6b-14481e968897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:05.467255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba6681-b146-486a-be6b-14481e968897 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:05.467498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba6681-b146-486a-be6b-14481e968897 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:05.467778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba6681-b146-486a-be6b-14481e968897 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:05.468610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba6681-b146-486a-be6b-14481e968897 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:05.469978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba6681-b146-486a-be6b-14481e968897 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:05.496704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 518/1013] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:41:05 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:41:05.562536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0451c-bdb2-4fbb-9549-63a0a7087eb0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:05.627788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528c14a-6c75-4464-8ab0-7a573ab0caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:05.628513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c14a-6c75-4464-8ab0-7a573ab0caa2 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:05.628854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c14a-6c75-4464-8ab0-7a573ab0caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:05.629323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c14a-6c75-4464-8ab0-7a573ab0caa2 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:05.630198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528c14a-6c75-4464-8ab0-7a573ab0caa2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:05.631873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528c14a-6c75-4464-8ab0-7a573ab0caa2 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:05.667202 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 496/1014] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:41:05 2022] POST /identity/v3/projects => generated 311 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:41:06.125112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716da00-0504-43a3-b31b-1e41b3c4e75d None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:06.125839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716da00-0504-43a3-b31b-1e41b3c4e75d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:06.125839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716da00-0504-43a3-b31b-1e41b3c4e75d None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:06.127074 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 519/1015] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:41:06.133638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d518a-5e83-49e8-a8f1-bf16c9851cab None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:06.134604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d518a-5e83-49e8-a8f1-bf16c9851cab None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:06.134604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d518a-5e83-49e8-a8f1-bf16c9851cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:06.155211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-896d518a-5e83-49e8-a8f1-bf16c9851cab None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:06.156543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-896d518a-5e83-49e8-a8f1-bf16c9851cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:06.247181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:06.258087 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 497/1016] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:06 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:06.608835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716da00-0504-43a3-b31b-1e41b3c4e75d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:06.736807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716da00-0504-43a3-b31b-1e41b3c4e75d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:06.782886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:06.783536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:06.783811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:06.784082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] PATH_INFO: `/v3/services/compute` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:06.785063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:06.786881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:06.796419 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:06.798520 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:06.800255 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 520/1017] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:41:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:41:06.808100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d518a-5e83-49e8-a8f1-bf16c9851cab None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:06.934853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d518a-5e83-49e8-a8f1-bf16c9851cab None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:06.976525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:06.977536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:06.977963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:06.978375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:06.979173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] RBAC: Adding query filter params (name=compute) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:06.979637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:06.981244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:06.987483 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 498/1018] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:41:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:06.993679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:07.041756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:07.108924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348dbb5-5458-4389-b60c-20097735ac11 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:07.147467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604bdf8-4ae2-4b84-a00e-fb1c844154a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:07.148516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604bdf8-4ae2-4b84-a00e-fb1c844154a6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:07.148516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604bdf8-4ae2-4b84-a00e-fb1c844154a6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:07.148839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604bdf8-4ae2-4b84-a00e-fb1c844154a6 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:07.149252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604bdf8-4ae2-4b84-a00e-fb1c844154a6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:07.149559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604bdf8-4ae2-4b84-a00e-fb1c844154a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:07.150942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604bdf8-4ae2-4b84-a00e-fb1c844154a6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:07.156734 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 521/1019] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Sep 27 09:41:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:07.167421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:07.247016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:07.299905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b028b1-4640-4bef-a8e9-e4909694dddb None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:07.346353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1be0cd-3bc3-4bde-a94a-277ab2369a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:07.347043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1be0cd-3bc3-4bde-a94a-277ab2369a8b None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:07.348091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1be0cd-3bc3-4bde-a94a-277ab2369a8b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:07.348397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1be0cd-3bc3-4bde-a94a-277ab2369a8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:07.349121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1be0cd-3bc3-4bde-a94a-277ab2369a8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:07.350660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1be0cd-3bc3-4bde-a94a-277ab2369a8b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:07.378428 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 499/1020] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:41:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:41:08.257920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b29b4-a3ad-4f3a-94a3-861c0d16a535 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:08.258483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b29b4-a3ad-4f3a-94a3-861c0d16a535 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:08.258483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b29b4-a3ad-4f3a-94a3-861c0d16a535 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:08.260700 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 522/1021] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:08.268155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad99d11-8aa8-4e30-a2fd-d9161b314605 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:08.268889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad99d11-8aa8-4e30-a2fd-d9161b314605 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:08.268889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad99d11-8aa8-4e30-a2fd-d9161b314605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:08.297877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4ad99d11-8aa8-4e30-a2fd-d9161b314605 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:08.299356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ad99d11-8aa8-4e30-a2fd-d9161b314605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:08.342789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c422a8-2cbe-472f-925a-aef56754edc3 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:08.343689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c422a8-2cbe-472f-925a-aef56754edc3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:08.344126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c422a8-2cbe-472f-925a-aef56754edc3 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:08.346338 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 523/1022] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:08.352742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b5f45-2334-45d9-9201-ee96249fe542 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:08.353104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b5f45-2334-45d9-9201-ee96249fe542 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:08.353324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b5f45-2334-45d9-9201-ee96249fe542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:08.374715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8f5b5f45-2334-45d9-9201-ee96249fe542 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:08.376196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8f5b5f45-2334-45d9-9201-ee96249fe542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:08.395712 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 500/1023] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:08.460317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 524/1024] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:08.725303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad99d11-8aa8-4e30-a2fd-d9161b314605 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:08.779650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:08.796004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b5f45-2334-45d9-9201-ee96249fe542 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:08.830927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:08.850500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad99d11-8aa8-4e30-a2fd-d9161b314605 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:08.890650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:08.891218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:08.891496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:08.891764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] PATH_INFO: `/v3/services/image` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:08.892360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:08.893775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:08.903548 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:08.904704 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:08.905593 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 501/1025] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Sep 27 09:41:08 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:08.910655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:08.910975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b5f45-2334-45d9-9201-ee96249fe542 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:08.947958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:08.948596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:08.948908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:08.949177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:08.950288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:08.951817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:08.953633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 525/1026] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:41:08.959860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:09.019094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c92b9-6a9a-4856-a9ab-d1ba910ddc38 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:09.028633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:09.057421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:09.058046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:09.058319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:09.058588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:09.059220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] RBAC: Adding query filter params (name=image) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:09.059519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:09.060918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:09.066293 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 502/1027] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:41:09.071878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:09.075097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb638fee-bd0a-49a3-a961-a87f5d739fc4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:09.077722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:09.116842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:09.117460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:09.117801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:09.118361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:09.118714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:09.120287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:09.146647 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 526/1028] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:41:08 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:41:09.191776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017b82a-2634-4b74-acff-05377e9e379e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:09.233809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:09.234534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:09.234878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:09.235216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:09.235930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] RBAC: Adding query filter params (type=image) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:09.236298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:09.237871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:09.244089 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 503/1029] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:09.250243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:09.388933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b1d37-6aba-47a8-b3f7-bc7ee3b22430 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:09.425221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf0a4b-919e-45ca-aba8-09adfb4e2f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:09.426142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf0a4b-919e-45ca-aba8-09adfb4e2f62 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:09.426142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf0a4b-919e-45ca-aba8-09adfb4e2f62 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:09.426416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf0a4b-919e-45ca-aba8-09adfb4e2f62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:09.427137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf0a4b-919e-45ca-aba8-09adfb4e2f62 None admin] RBAC: Adding query filter params (interface=public, service_id=1776c39490d5471bb9a87d3ac4ba525f, region_id=RegionOne) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:09.427451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf0a4b-919e-45ca-aba8-09adfb4e2f62 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:09.428899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf0a4b-919e-45ca-aba8-09adfb4e2f62 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:09.434267 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 527/1030] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:41:09 2022] GET /identity/v3/endpoints?service_id=1776c39490d5471bb9a87d3ac4ba525f&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:09.440107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:09.483169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:09.551573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dbc6d-b37a-4571-a205-e4cabd3c3df5 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:09.588912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4be35d-80d7-4346-86b1-c82908360968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:09.589555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4be35d-80d7-4346-86b1-c82908360968 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:09.589829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4be35d-80d7-4346-86b1-c82908360968 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:09.590096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4be35d-80d7-4346-86b1-c82908360968 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:09.590720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4be35d-80d7-4346-86b1-c82908360968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:09.592146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4be35d-80d7-4346-86b1-c82908360968 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:09.600296 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 504/1031] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:41:09 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:41:10.038115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903ce1-181c-4e66-a3e0-76c1ccadae5a None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:10.038511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903ce1-181c-4e66-a3e0-76c1ccadae5a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:10.038674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903ce1-181c-4e66-a3e0-76c1ccadae5a None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:10.039996 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 528/1032] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:10.045906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef806a-d758-4bc9-8cf1-3cf1e109f988 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:10.046323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef806a-d758-4bc9-8cf1-3cf1e109f988 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:10.046701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef806a-d758-4bc9-8cf1-3cf1e109f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:10.066159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-26ef806a-d758-4bc9-8cf1-3cf1e109f988 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:10.067495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-26ef806a-d758-4bc9-8cf1-3cf1e109f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:10.168798 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 505/1033] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:10.489938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7903ce1-181c-4e66-a3e0-76c1ccadae5a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:10.516322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:10.605128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7903ce1-181c-4e66-a3e0-76c1ccadae5a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:10.644547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:10.645421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:10.645693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:10.645960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:10.656825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:10.662255 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:10.663231 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:10.664086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 529/1034] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 27 09:41:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:41:10.671069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef806a-d758-4bc9-8cf1-3cf1e109f988 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:10.707401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:10.781415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef806a-d758-4bc9-8cf1-3cf1e109f988 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:10.817318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:10.817926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:10.818201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:10.818469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:10.819124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:10.819416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:10.820811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:10.829415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 506/1035] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 27 09:41:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:10.835893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:10.923664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:10.944540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25669-6dc9-4a62-8f3c-df451e0642b2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:10.980384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:10.980958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:10.981233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:10.981502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:10.992564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:10.994053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:11.008896 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:11.009923 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:41:11.015944 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:11.015944 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:11.015944 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:11.015944 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 530/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:11.017937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:11.127754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:11.142566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e489cd-7315-4b03-b0e5-fff0f21a677b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:11.189899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:11.190524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:11.190796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:11.191210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:11.192080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:11.192317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:11.193742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:11.206231 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 507/1037] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:11.212100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:11.321671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85abf08-adec-4baa-97c9-792a8a91971c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:11.361905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:11.362601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:11.362910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:11.363184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:11.365765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:11.373425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:11.374875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:41:11.386897 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:11.387871 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:41:11.389389 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:11.390296 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 531/1038] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:41:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:11.393834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:11.503713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d343a-a064-4e4b-8daa-296703a86008 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:11.539867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3ac39-1049-4a34-91d9-9d6ed50b87ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:11.540486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3ac39-1049-4a34-91d9-9d6ed50b87ae None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:11.540783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3ac39-1049-4a34-91d9-9d6ed50b87ae None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:11.541052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3ac39-1049-4a34-91d9-9d6ed50b87ae None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:11.541782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3ac39-1049-4a34-91d9-9d6ed50b87ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:11.542086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3ac39-1049-4a34-91d9-9d6ed50b87ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:11.543513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3ac39-1049-4a34-91d9-9d6ed50b87ae None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:11.571329 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 508/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:41:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:11.572608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:11.695885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5520748-af9b-41a9-9772-7dfa970a3a9e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:11.699679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d24617-c029-40cd-b4aa-0ed195744546 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:11.700152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d24617-c029-40cd-b4aa-0ed195744546 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:11.700230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d24617-c029-40cd-b4aa-0ed195744546 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:11.701599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 509/1040] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:11.706922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261509f-ad38-4e49-8ce3-63319b5d1005 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:11.707209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261509f-ad38-4e49-8ce3-63319b5d1005 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:11.707495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261509f-ad38-4e49-8ce3-63319b5d1005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:11.725922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8261509f-ad38-4e49-8ce3-63319b5d1005 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:11.727124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8261509f-ad38-4e49-8ce3-63319b5d1005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:11.730123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:11.737686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67a24d-fd70-44d3-a655-4d0195499409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:11.738384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67a24d-fd70-44d3-a655-4d0195499409 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:11.738661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67a24d-fd70-44d3-a655-4d0195499409 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:11.738932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67a24d-fd70-44d3-a655-4d0195499409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:11.803318 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 510/1041] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:11.809575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:11.820965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67a24d-fd70-44d3-a655-4d0195499409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:11.822492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67a24d-fd70-44d3-a655-4d0195499409 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:11.943598 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 532/1042] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 27 09:41:11.949580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261509f-ad38-4e49-8ce3-63319b5d1005 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.054812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261509f-ad38-4e49-8ce3-63319b5d1005 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.074609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e0a9a-68bb-4837-996a-730349ee11de None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.074891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e0a9a-68bb-4837-996a-730349ee11de None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.075163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e0a9a-68bb-4837-996a-730349ee11de None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.076525 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 533/1043] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:12.083532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547d02-de32-4915-8bf7-4d260cf12c73 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.083808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547d02-de32-4915-8bf7-4d260cf12c73 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.084241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547d02-de32-4915-8bf7-4d260cf12c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.098482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.099810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.100090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.100358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.106003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ec547d02-de32-4915-8bf7-4d260cf12c73 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:12.107378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ec547d02-de32-4915-8bf7-4d260cf12c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:12.111655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:12.115967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.118530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:12.141573 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:12.142942 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:41:12.144510 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:12.144510 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:12.144510 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:12.144510 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 511/1044] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:12.144881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.206228 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 534/1045] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:41:12.207987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547d02-de32-4915-8bf7-4d260cf12c73 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.236171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:12.274873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b695b0-5829-4211-9ae9-13f90af11a6d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.315436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.316165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.316451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.316727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.324589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547d02-de32-4915-8bf7-4d260cf12c73 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.329104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:12.334697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:12.335879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:12.336784 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 512/1046] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:12.342133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.372808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.373716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.373986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.374670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.375421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:12.375638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.377106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.392481 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 535/1047] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:12.398695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.416101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:12.476805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74bf06-fa6b-4e9d-9409-0d73ce52e19f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.496997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:12.514071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.514693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.514973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.515204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed46493-6ddd-4bc8-aa53-68acf5c06bb8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.515407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.516011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:12.516320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.517733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.526622 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 513/1048] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:12.532719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.550827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.551474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.551742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.552009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.559320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.560809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:41:12.572994 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:12.574440 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 536/1049] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:12.575349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.645245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040f2ed-c0a0-449a-9c06-965476e14e86 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.658861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:12.680269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.680975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.681241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.681508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] PATH_INFO: `/v3/users/46625e52c4dd4a0780113a07233b36b0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.684902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88545c21-3406-482a-a956-cf6615937d7d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.695714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] RBAC: Authorizing `identity:get_user(user_id=46625e52c4dd4a0780113a07233b36b0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.697206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.703590 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 514/1050] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/users/46625e52c4dd4a0780113a07233b36b0 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:12.703590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.718847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.719617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.719795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.720035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] PATH_INFO: `/v3/services/image` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.720633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.722036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:12.732006 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:12.733281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:12.734332 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 537/1051] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:12.735178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.746485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:12.805911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79c4ca-9174-4f3a-9112-29873895d98d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.839611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.839942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e5e0b-beb2-4a00-a7e7-ae29916dbf9b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.840218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.840558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.840864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.841557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:12.841855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.843278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.854829 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 515/1052] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:12.855665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:12.877640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:12.878251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:12.878627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:12.879720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:12.892597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:12.956163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:12.957993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:12.966182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6090e-958f-4700-9302-c573a12aa510 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:12.968083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:13.001871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:13.002400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:13.002673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:13.002940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:13.003654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] RBAC: Adding query filter params (name=image) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:13.004011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:13.005415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:13.010479 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 516/1053] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:13.011535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:13.064111 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 538/1054] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:13.065082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:13.114167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:13.122279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f691503-ff33-4343-861d-09d152a238b4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:13.168931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:13.169646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:13.169955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:13.170364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:13.171145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] RBAC: Adding query filter params (role.id=e851bb42f132413f967ccd21e7bc8f37, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=3c8ed3cf7a5742808029d336ebc784fa) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:13.171458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:13.172344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7be086-b85d-43b4-92f4-fc4112fd2f45 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:13.173208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:13.196687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 517/1055] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:13 2022] GET /identity/v3/role_assignments?user.id=3c8ed3cf7a5742808029d336ebc784fa&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=e851bb42f132413f967ccd21e7bc8f37 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:13.196687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:13.198369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:13.217053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:13.217791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:13.218131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:13.218467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:13.219218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:13.219586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:13.221101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:13.228732 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 539/1056] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:13.234690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:13.301012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bb44a-bfde-46c7-a2cd-8fdd72d4fc42 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:13.337802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:13.338465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:13.338749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:13.340730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:13.341254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] RBAC: Authorizing `identity:get_project(project_id=4b76c9f802df469db5132e3840e298ac)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:13.342727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:13.350045 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 518/1057] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:13 2022] GET /identity/v3/projects/4b76c9f802df469db5132e3840e298ac => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:13.351051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:13.356353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abd407-8891-468b-92d8-b666aaef27c5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:13.384971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:13.398317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bac88-4b17-470b-be83-434418214cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:13.398951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bac88-4b17-470b-be83-434418214cff None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:13.399230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bac88-4b17-470b-be83-434418214cff None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:13.399500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bac88-4b17-470b-be83-434418214cff None admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:13.400183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bac88-4b17-470b-be83-434418214cff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:13.402292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bac88-4b17-470b-be83-434418214cff None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:13.428591 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 540/1058] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:41:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:41:13.482341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4cb9b-c61c-4b7c-ad9c-52b0e3edfa08 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:13.527280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819014d-9e77-467a-9d7b-6dd09eed21d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:13.527913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819014d-9e77-467a-9d7b-6dd09eed21d7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:13.528188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819014d-9e77-467a-9d7b-6dd09eed21d7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:13.528456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819014d-9e77-467a-9d7b-6dd09eed21d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:13.529248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819014d-9e77-467a-9d7b-6dd09eed21d7 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=4b76c9f802df469db5132e3840e298ac, user.id=46625e52c4dd4a0780113a07233b36b0) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:13.529563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819014d-9e77-467a-9d7b-6dd09eed21d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:13.531029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819014d-9e77-467a-9d7b-6dd09eed21d7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:13.544707 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 519/1059] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:13 2022] GET /identity/v3/role_assignments?user.id=46625e52c4dd4a0780113a07233b36b0&scope.project.id=4b76c9f802df469db5132e3840e298ac&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:15.706365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51cd62-fbd6-488d-ad26-742a31e76a73 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:15.707099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51cd62-fbd6-488d-ad26-742a31e76a73 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:15.707544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51cd62-fbd6-488d-ad26-742a31e76a73 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:15.709022 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 541/1060] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:15.715366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852f71-77aa-419e-bee0-d451844aa134 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:15.715948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852f71-77aa-419e-bee0-d451844aa134 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:15.715948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852f71-77aa-419e-bee0-d451844aa134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:15.737268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-99852f71-77aa-419e-bee0-d451844aa134 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:15.739246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99852f71-77aa-419e-bee0-d451844aa134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:15.822715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 520/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:15.911029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b2bf9-c1ee-4435-bb93-d21ce1a4e234 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:15.911235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b2bf9-c1ee-4435-bb93-d21ce1a4e234 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:15.911996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b2bf9-c1ee-4435-bb93-d21ce1a4e234 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:15.913526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 542/1062] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:15.919810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6818a-5385-4ef3-b5af-8e4f3b5818e0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:15.920080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6818a-5385-4ef3-b5af-8e4f3b5818e0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:15.920307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6818a-5385-4ef3-b5af-8e4f3b5818e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:15.940285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-87e6818a-5385-4ef3-b5af-8e4f3b5818e0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:15.941654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-87e6818a-5385-4ef3-b5af-8e4f3b5818e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:15.959284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:15.965353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989532ff-a6db-4774-9671-0322d8e3b779 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:15.965929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989532ff-a6db-4774-9671-0322d8e3b779 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:15.966204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989532ff-a6db-4774-9671-0322d8e3b779 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:15.967550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 543/1063] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:41:15.973393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41848c6a-f853-4526-a472-1b67e2761b01 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:15.973633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41848c6a-f853-4526-a472-1b67e2761b01 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:15.974201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41848c6a-f853-4526-a472-1b67e2761b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:15.993825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-41848c6a-f853-4526-a472-1b67e2761b01 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:15.995037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-41848c6a-f853-4526-a472-1b67e2761b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:16.021695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:16.025663 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 521/1064] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:16.080705 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 544/1065] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:41:16.161551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6818a-5385-4ef3-b5af-8e4f3b5818e0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.266937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6818a-5385-4ef3-b5af-8e4f3b5818e0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.307827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:16.308479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:16.308818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:16.309068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:16.320027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:16.324879 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:16.325818 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:41:16.326656 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:16.329031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 522/1066] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:16.332593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41848c6a-f853-4526-a472-1b67e2761b01 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.370837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.383038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:16.447662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41848c6a-f853-4526-a472-1b67e2761b01 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.482866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d70c56-5a53-4459-81d3-5e0b1849f2b9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.483102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:16.483486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:16.483758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:16.484028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:16.484695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:16.485030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:16.486410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:16.495416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 545/1067] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:16.496238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.518598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:16.525314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:16.526188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:16.526463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:16.526729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:16.596988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:16.598619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:16.608448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bc845-bcbd-4b85-be7a-927c1ea1a060 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.614400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:16.647100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:16.647732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:16.648002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:16.648266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:16.659338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:16.664595 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:16.665894 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 546/1068] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:16.666763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.715934 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 523/1069] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 27 09:41:16.717413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.748571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:16.768183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961038b1-9f2d-4534-843e-c45e5aa18b3c None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.803750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:16.804430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:16.804715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:16.804997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:16.815697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:16.817247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:16.819623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92081b-a4fc-4683-a46b-610a0f19c4b4 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.829047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:16.831345 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:16.833186 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:41:16.834176 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:16.834176 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:16.834176 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:16.834176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 547/1070] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:16.834176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.858365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:16.858979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:16.859256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:16.859528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:16.860197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:16.860497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:16.861892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:16.870608 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 524/1071] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:16.871269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:16.935757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3a3db-abe7-4cce-a331-6a4808faf322 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.965257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:16.971302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:16.971877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:16.972167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:16.972448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:16.975036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9302-bbc8-4ecb-97c8-310b84d2b142 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:16.983443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:16.984939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:16.999221 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:17.000246 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:41:17.006807 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:17.006807 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:17.006807 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:17.006807 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 548/1072] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:17.006807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.008259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.008906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.009176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.009441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.010089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:17.010387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.011784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.023976 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 525/1073] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:17.025047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.056941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:17.104507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e0a94-edcb-4fc6-b28c-8390e2e1655e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.128302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77426c50-fa62-4792-9ddf-dfdabf18b006 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.139662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.140298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.140573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.140900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] PATH_INFO: `/v3/users/46625e52c4dd4a0780113a07233b36b0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.142038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] RBAC: Authorizing `identity:get_user(user_id=46625e52c4dd4a0780113a07233b36b0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.143482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.145304 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 549/1074] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/users/46625e52c4dd4a0780113a07233b36b0 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:17.145905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.170055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.170662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.170944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.171214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.171860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:17.172160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.173568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.184906 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 526/1075] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:17.185767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.201263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:17.256560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907fd60-f2c2-4fb1-a960-5e03f4cdbe3d None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.303506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.304164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.304440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.304745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.310767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:17.314410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.316274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.323880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42040c2-62b7-4e64-88ed-f328a3e08dd3 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:41:17.331658 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:17.332764 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:41:17.333754 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:17.335000 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 550/1076] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:17.340389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.369198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.369929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.370142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.370417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.371658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] RBAC: Authorizing `identity:get_project(project_id=4b76c9f802df469db5132e3840e298ac)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.373204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.380599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 527/1077] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/projects/4b76c9f802df469db5132e3840e298ac => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:17.382998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.468122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8c86c-1e4e-4b63-be4f-941d1a5be918 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.487321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:17.506952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a87573-c1f3-4639-b8c0-1675695e55bf None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.514172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.514785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.515053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.515317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.516048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:17.516343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.517851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.538814 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 551/1078] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:17.545610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.550043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.550355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.550661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.551389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac/users/46625e52c4dd4a0780113a07233b36b0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.553291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b76c9f802df469db5132e3840e298ac, user_id=46625e52c4dd4a0780113a07233b36b0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.554406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.572588 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 528/1079] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:41:17 2022] PUT /identity/v3/projects/4b76c9f802df469db5132e3840e298ac/users/46625e52c4dd4a0780113a07233b36b0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:41:17.674389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9a4a5-a5cc-4d0d-a3f2-27d55dcb5c50 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.713152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.713813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.714077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.714346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.725815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.727329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:17.743788 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:17.745010 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:41:17.745969 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:17.745969 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:17.745969 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:17.745969 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 552/1080] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:41:17.751371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.775177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:17.861850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79488c-48e0-48cc-9738-b1d1ac9c8c03 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:17.897999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:17.898616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:17.898891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:17.899163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:17.899804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:17.900107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:17.901511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:17.914032 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 529/1081] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:17.919919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:17.945618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:18.024238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1eee4-5c56-4de3-b32c-0379131ecb12 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:18.059245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:18.059880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:18.060154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:18.060422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:18.067313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:18.068784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:41:18.080029 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:18.081061 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:18.082093 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 553/1082] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:41:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:18.087569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:18.166726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:18.188943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae957f2-ee83-492f-8b1d-c2fc8384ede9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:18.224671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d10e93-a4b2-4cd8-8f04-d1b72574cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:18.225311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d10e93-a4b2-4cd8-8f04-d1b72574cf19 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:18.225712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d10e93-a4b2-4cd8-8f04-d1b72574cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:18.225832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d10e93-a4b2-4cd8-8f04-d1b72574cf19 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:18.226535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d10e93-a4b2-4cd8-8f04-d1b72574cf19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:18.226897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d10e93-a4b2-4cd8-8f04-d1b72574cf19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:18.228324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d10e93-a4b2-4cd8-8f04-d1b72574cf19 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:18.240289 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 530/1083] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:18.246821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:18.340541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:18.360430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b19475-c9af-425f-b516-f02a62f6958b None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:18.394044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181c6cd-0978-4f64-8eb0-77ab2ff8d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:18.394892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181c6cd-0978-4f64-8eb0-77ab2ff8d4f2 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:18.395182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181c6cd-0978-4f64-8eb0-77ab2ff8d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:18.395451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181c6cd-0978-4f64-8eb0-77ab2ff8d4f2 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/3c8ed3cf7a5742808029d336ebc784fa/roles/e851bb42f132413f967ccd21e7bc8f37` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:18.397724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181c6cd-0978-4f64-8eb0-77ab2ff8d4f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=3c8ed3cf7a5742808029d336ebc784fa, role_id=e851bb42f132413f967ccd21e7bc8f37)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:18.399165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181c6cd-0978-4f64-8eb0-77ab2ff8d4f2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:18.414217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 554/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:41:18 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/3c8ed3cf7a5742808029d336ebc784fa/roles/e851bb42f132413f967ccd21e7bc8f37 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:41:20.008028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f040-2c16-45f4-83c2-4d0f1b4b6944 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:20.008562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f040-2c16-45f4-83c2-4d0f1b4b6944 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:20.008562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f040-2c16-45f4-83c2-4d0f1b4b6944 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:20.009828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 531/1085] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:20.015514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbdaa9-ad79-4de6-9b3a-a33945f92ee0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:20.015923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbdaa9-ad79-4de6-9b3a-a33945f92ee0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:20.016093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbdaa9-ad79-4de6-9b3a-a33945f92ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:20.036155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-84fbdaa9-ad79-4de6-9b3a-a33945f92ee0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:20.037740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84fbdaa9-ad79-4de6-9b3a-a33945f92ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:20.110270 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 555/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:20.432341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f040-2c16-45f4-83c2-4d0f1b4b6944 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:20.547965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f040-2c16-45f4-83c2-4d0f1b4b6944 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:20.586346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:20.587003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:20.587277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:20.587546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:20.594904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:20.602453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:20.605043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:20.605991 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:20.606868 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 532/1087] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:20.612965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbdaa9-ad79-4de6-9b3a-a33945f92ee0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:20.682864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:20.714874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbdaa9-ad79-4de6-9b3a-a33945f92ee0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:20.750468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf81ad-28c8-480c-99c1-1f783eeedeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:20.751071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf81ad-28c8-480c-99c1-1f783eeedeee None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:20.751343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf81ad-28c8-480c-99c1-1f783eeedeee None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:20.751584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf81ad-28c8-480c-99c1-1f783eeedeee None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:20.752279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf81ad-28c8-480c-99c1-1f783eeedeee None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:20.752571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf81ad-28c8-480c-99c1-1f783eeedeee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:20.753990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf81ad-28c8-480c-99c1-1f783eeedeee None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:20.762429 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 556/1088] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:20.768539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:20.837224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57ac0-79c7-4df3-b97c-56fdfacb9832 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:20.837564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57ac0-79c7-4df3-b97c-56fdfacb9832 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:20.837746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57ac0-79c7-4df3-b97c-56fdfacb9832 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:20.839220 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 557/1089] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:41:20.844201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7d7e-ce32-4607-906a-5b628bf56852 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:20.844478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7d7e-ce32-4607-906a-5b628bf56852 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:20.844777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7d7e-ce32-4607-906a-5b628bf56852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:20.864420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0ffb7d7e-ce32-4607-906a-5b628bf56852 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:20.865671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0ffb7d7e-ce32-4607-906a-5b628bf56852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:20.872590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efd537-92d6-4b6c-914d-04a4c827b706 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:20.912353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:20.913023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:20.913298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:20.913571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] PATH_INFO: `/v3/users/46625e52c4dd4a0780113a07233b36b0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:20.914724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] RBAC: Authorizing `identity:get_user(user_id=46625e52c4dd4a0780113a07233b36b0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:20.916162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:20.917782 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 533/1090] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:20 2022] GET /identity/v3/users/46625e52c4dd4a0780113a07233b36b0 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:20.923886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:20.936257 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 558/1091] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:41:20.991146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:21.029423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b618f-ffa2-4f18-85ff-459ad34941fe None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:21.064407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:21.065149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:21.065425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:21.065682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:21.136895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:21.138366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:21.205554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:21.242385 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 534/1092] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:21.248489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb7d7e-ce32-4607-906a-5b628bf56852 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:21.269352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:21.311065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:21.350665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb7d7e-ce32-4607-906a-5b628bf56852 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:21.377220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b45f0d-b0f8-4500-b828-d6c622b129ec None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:21.385479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:21.386123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:21.386404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:21.386673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:21.387803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] RBAC: Authorizing `identity:get_project(project_id=4b76c9f802df469db5132e3840e298ac)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:21.389282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:21.391094 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 559/1093] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:21 2022] GET /identity/v3/projects/4b76c9f802df469db5132e3840e298ac => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:21.397412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:21.415227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:21.415866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:21.416140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:21.416406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:21.426538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:21.431333 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:21.436701 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:21.437784 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 535/1094] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Sep 27 09:41:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:21.438684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:21.472161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:21.512370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576ad5-ebaa-4170-b8bb-03e6ef05c3ee None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:21.544138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:21.547640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ef2ea-dd40-4af6-b559-0678249a988d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:21.550449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:21.551088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:21.551381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:21.551646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:21.552364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=4b76c9f802df469db5132e3840e298ac, user.id=46625e52c4dd4a0780113a07233b36b0) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:21.552676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:21.554105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:21.561097 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 560/1095] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:21 2022] GET /identity/v3/role_assignments?user.id=46625e52c4dd4a0780113a07233b36b0&scope.project.id=4b76c9f802df469db5132e3840e298ac&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:21.586598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:21.587228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:21.587502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:21.587843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:21.588538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:21.588866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:21.590342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:21.606453 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 536/1096] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 27 09:41:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:21.610833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:21.721137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d39d2a-3b7a-4681-b564-59f0caeadc9f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:21.758875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:21.759419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:21.759690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:21.760062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:21.772128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:21.773659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:21.789837 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:21.791038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:41:21.793393 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:21.793393 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:21.793393 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:21.793393 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 561/1097] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:21.797350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:21.902734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:21.916863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf6504-8108-4428-a882-0aaae7ac75cf None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:21.951550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:21.952160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:21.952439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:21.952719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:21.953393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:21.953691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:21.955066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:21.968112 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 537/1098] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:21.974855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:22.030099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:22.083358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b22f-379f-44c1-82af-61c31bd8c5c3 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:22.117324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:22.117946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:22.118215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:22.118473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] PATH_INFO: `/v3/users/nova` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:22.126460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:22.127890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:41:22.139425 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:22.140431 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:22.141422 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 562/1099] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Sep 27 09:41:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:22.147539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:22.276508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac118c-5cfc-4cae-9c66-b5ca1674d140 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:22.326042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:22.326619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:22.326905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:22.327207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:22.328118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:22.328442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:22.329976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:22.349425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 538/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Sep 27 09:41:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:22.355268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:22.463259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc94b33-29e3-4305-b346-6cbe8ba7853f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:22.472758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:22.499476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:22.500089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:22.500363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:22.500644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:22.571869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:22.573392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:22.701882 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 563/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:22.708730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:22.718594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:22.816128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c73dd-3efe-44a4-809e-7677293b3293 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:22.855055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:22.855624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:22.855899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:22.856173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:22.868741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:22.870236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:22.887882 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:22.889178 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:41:22.890584 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:22.890584 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:41:22.890584 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:22.890584 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 539/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:22.895423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:22.907729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:23.012755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93571ee-2de3-4a7b-8ff1-42ed756dd00a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:23.053523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb681-823f-4382-8130-02b268493a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:23.054141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb681-823f-4382-8130-02b268493a20 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:23.054419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb681-823f-4382-8130-02b268493a20 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:23.054689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb681-823f-4382-8130-02b268493a20 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:23.055343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb681-823f-4382-8130-02b268493a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:23.055647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb681-823f-4382-8130-02b268493a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:23.057060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb681-823f-4382-8130-02b268493a20 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:23.070251 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 564/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:23.076171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:23.145936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:23.190922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9c691-c21c-430c-ae98-60d070d9dcbe None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:23.226788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:23.227429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:23.227703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:23.227971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:23.234730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:23.236180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:41:23.247489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:23.248437 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:23.249419 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 540/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:41:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:23.254995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb681-823f-4382-8130-02b268493a20 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:23.324547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:23.362541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb681-823f-4382-8130-02b268493a20 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:23.403529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca93d5e-d338-451d-a5e0-42231f07baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:23.404046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93d5e-d338-451d-a5e0-42231f07baec None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:23.404319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93d5e-d338-451d-a5e0-42231f07baec None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:23.404583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93d5e-d338-451d-a5e0-42231f07baec None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:23.405288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca93d5e-d338-451d-a5e0-42231f07baec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:23.405584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca93d5e-d338-451d-a5e0-42231f07baec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:23.407007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca93d5e-d338-451d-a5e0-42231f07baec None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:23.420221 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 565/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:41:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:23.426889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:23.541722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efe711-1a50-472f-b02b-8527881472f0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:23.561941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:23.584909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4f49e-03a6-429c-b90d-28100c9835c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:23.585532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4f49e-03a6-429c-b90d-28100c9835c8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:23.585817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4f49e-03a6-429c-b90d-28100c9835c8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:23.586087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4f49e-03a6-429c-b90d-28100c9835c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:23.586834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4f49e-03a6-429c-b90d-28100c9835c8 None admin] RBAC: Adding query filter params (role.id=e851bb42f132413f967ccd21e7bc8f37, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=3c8ed3cf7a5742808029d336ebc784fa) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:23.587130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4f49e-03a6-429c-b90d-28100c9835c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:23.588569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4f49e-03a6-429c-b90d-28100c9835c8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:23.602660 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 541/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:23 2022] GET /identity/v3/role_assignments?user.id=3c8ed3cf7a5742808029d336ebc784fa&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=e851bb42f132413f967ccd21e7bc8f37 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:24.085568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192055-f91c-4d89-84b4-0c3555319570 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:24.085847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192055-f91c-4d89-84b4-0c3555319570 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:24.086118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192055-f91c-4d89-84b4-0c3555319570 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:24.087504 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 566/1107] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:24.093309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c1bd7-07d2-4f69-b2c0-11ac8e40d950 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:24.093586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c1bd7-07d2-4f69-b2c0-11ac8e40d950 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:24.093856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c1bd7-07d2-4f69-b2c0-11ac8e40d950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:24.113551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-323c1bd7-07d2-4f69-b2c0-11ac8e40d950 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:24.114739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-323c1bd7-07d2-4f69-b2c0-11ac8e40d950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:24.184064 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 542/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:24.508461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192055-f91c-4d89-84b4-0c3555319570 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:24.615661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192055-f91c-4d89-84b4-0c3555319570 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:24.643086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:24.652354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ad1f3-7c03-46ec-be82-55c6bfcec1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:24.653060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ad1f3-7c03-46ec-be82-55c6bfcec1df None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:24.653330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ad1f3-7c03-46ec-be82-55c6bfcec1df None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:24.653593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ad1f3-7c03-46ec-be82-55c6bfcec1df None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:24.654753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ad1f3-7c03-46ec-be82-55c6bfcec1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:24.656187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ad1f3-7c03-46ec-be82-55c6bfcec1df None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:24.657819 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 567/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:24.664500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c1bd7-07d2-4f69-b2c0-11ac8e40d950 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:24.767059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c1bd7-07d2-4f69-b2c0-11ac8e40d950 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:24.781089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:24.804618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a043a-6566-47b4-8636-7e8cbc58f525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:24.805256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a043a-6566-47b4-8636-7e8cbc58f525 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:24.805529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a043a-6566-47b4-8636-7e8cbc58f525 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:24.805812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a043a-6566-47b4-8636-7e8cbc58f525 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:24.806455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a043a-6566-47b4-8636-7e8cbc58f525 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:24.807878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a043a-6566-47b4-8636-7e8cbc58f525 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:24.831301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 543/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:41:24 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:41:27.307838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0423cd-9435-46b3-a530-0763d05ba190 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:27.308436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0423cd-9435-46b3-a530-0763d05ba190 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:27.308436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0423cd-9435-46b3-a530-0763d05ba190 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:27.309758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 568/1111] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:27.315163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28976db7-abbf-46a1-bf99-946f0b241bc6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:27.315596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28976db7-abbf-46a1-bf99-946f0b241bc6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:27.315712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28976db7-abbf-46a1-bf99-946f0b241bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:27.334777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-28976db7-abbf-46a1-bf99-946f0b241bc6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:27.335993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-28976db7-abbf-46a1-bf99-946f0b241bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:27.406264 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 544/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:27.729367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0423cd-9435-46b3-a530-0763d05ba190 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:27.844067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0423cd-9435-46b3-a530-0763d05ba190 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:27.888006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:27.888560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:27.888865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:27.889135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] PATH_INFO: `/v3/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:27.890546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] RBAC: Authorizing `identity:get_role(role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:27.891930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:27.893305 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 569/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:27 2022] GET /identity/v3/roles/6f0bec1f01514029a1162fe631d80648 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:27.900082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28976db7-abbf-46a1-bf99-946f0b241bc6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:28.001371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:28.005738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28976db7-abbf-46a1-bf99-946f0b241bc6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:28.043660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:28.044117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:28.044385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:28.044649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] PATH_INFO: `/v3/users/da2acb6504db4ef4b7c69be747df5ac0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:28.058962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] RBAC: Authorizing `identity:get_user(user_id=da2acb6504db4ef4b7c69be747df5ac0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:28.060407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:28.062064 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 545/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:27 2022] GET /identity/v3/users/da2acb6504db4ef4b7c69be747df5ac0 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:28.068298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:28.135525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:28.177525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e1696-1373-4230-a460-95fe1ff53f56 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:28.214254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:28.214856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:28.215131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:28.215403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:28.288961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:28.290483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:28.360783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:28.404450 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 570/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:28.413698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:28.520004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c18071-a400-4b37-9ee9-99873387730e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:28.555943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998063b-bb5b-46a2-9e59-ccd3e2664668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:28.556581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998063b-bb5b-46a2-9e59-ccd3e2664668 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:28.556887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998063b-bb5b-46a2-9e59-ccd3e2664668 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:28.557158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998063b-bb5b-46a2-9e59-ccd3e2664668 None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:28.558294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998063b-bb5b-46a2-9e59-ccd3e2664668 None admin] RBAC: Authorizing `identity:get_project(project_id=4b76c9f802df469db5132e3840e298ac)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:28.559725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998063b-bb5b-46a2-9e59-ccd3e2664668 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:28.561394 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 546/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:28 2022] GET /identity/v3/projects/4b76c9f802df469db5132e3840e298ac => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:41:28.568245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:28.672684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79594fc-2c79-40f6-b333-a0c100903e24 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:28.707804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb80b2e-85d1-4102-aa87-2a9a92854e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:28.708393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb80b2e-85d1-4102-aa87-2a9a92854e27 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:28.708697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb80b2e-85d1-4102-aa87-2a9a92854e27 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:28.708959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb80b2e-85d1-4102-aa87-2a9a92854e27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:28.709667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb80b2e-85d1-4102-aa87-2a9a92854e27 None admin] RBAC: Adding query filter params (role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=4b76c9f802df469db5132e3840e298ac, user.id=da2acb6504db4ef4b7c69be747df5ac0) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:28.709972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb80b2e-85d1-4102-aa87-2a9a92854e27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:28.711388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb80b2e-85d1-4102-aa87-2a9a92854e27 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:28.716868 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 571/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:28 2022] GET /identity/v3/role_assignments?user.id=da2acb6504db4ef4b7c69be747df5ac0&scope.project.id=4b76c9f802df469db5132e3840e298ac&role.id=6f0bec1f01514029a1162fe631d80648 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:31.124680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cec1b-c351-4ee2-a7c5-f98826fd0260 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:31.125309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cec1b-c351-4ee2-a7c5-f98826fd0260 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:31.125309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cec1b-c351-4ee2-a7c5-f98826fd0260 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:31.126571 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 547/1118] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:31.131954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22ac9f-e3de-4e31-9e75-729edee71a52 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:31.132411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22ac9f-e3de-4e31-9e75-729edee71a52 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:31.132515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22ac9f-e3de-4e31-9e75-729edee71a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:31.152183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6a22ac9f-e3de-4e31-9e75-729edee71a52 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:31.153431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6a22ac9f-e3de-4e31-9e75-729edee71a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:31.178765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:31.225373 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 572/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:31.554636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cec1b-c351-4ee2-a7c5-f98826fd0260 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:31.586716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:31.677467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cec1b-c351-4ee2-a7c5-f98826fd0260 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:31.715928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:31.716572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:31.716883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:31.717159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] PATH_INFO: `/v3/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:31.718665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] RBAC: Authorizing `identity:get_role(role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:31.720053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:31.721458 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 548/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:31 2022] GET /identity/v3/roles/6f0bec1f01514029a1162fe631d80648 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:31.727759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22ac9f-e3de-4e31-9e75-729edee71a52 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:31.836957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22ac9f-e3de-4e31-9e75-729edee71a52 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:31.875248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:31.875908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:31.876187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:31.876457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] PATH_INFO: `/v3/users/da2acb6504db4ef4b7c69be747df5ac0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:31.877830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] RBAC: Authorizing `identity:get_user(user_id=da2acb6504db4ef4b7c69be747df5ac0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:31.879992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:31.881661 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 573/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:31 2022] GET /identity/v3/users/da2acb6504db4ef4b7c69be747df5ac0 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:31.888497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:31.992535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:32.002306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcaf3c-c9d3-4ec7-8851-9219510daf8d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:32.037365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69f8c6-7723-4d72-b01a-2b4be6c10dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:32.038007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69f8c6-7723-4d72-b01a-2b4be6c10dcf None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:32.038280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69f8c6-7723-4d72-b01a-2b4be6c10dcf None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:32.038545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69f8c6-7723-4d72-b01a-2b4be6c10dcf None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:32.039669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69f8c6-7723-4d72-b01a-2b4be6c10dcf None admin] RBAC: Authorizing `identity:get_project(project_id=4b76c9f802df469db5132e3840e298ac)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:32.041119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69f8c6-7723-4d72-b01a-2b4be6c10dcf None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:32.042738 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 549/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:31 2022] GET /identity/v3/projects/4b76c9f802df469db5132e3840e298ac => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:32.049064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:32.080220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:32.156325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901153b3-57c9-4f0b-a59e-0da09ac3b2f7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:32.193607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f6b2d-2042-499a-b540-b91eb6bfa23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:32.194301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f6b2d-2042-499a-b540-b91eb6bfa23e None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:32.194573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f6b2d-2042-499a-b540-b91eb6bfa23e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:32.194839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f6b2d-2042-499a-b540-b91eb6bfa23e None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac/users/da2acb6504db4ef4b7c69be747df5ac0/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:32.197172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f6b2d-2042-499a-b540-b91eb6bfa23e None admin] RBAC: Authorizing `identity:create_grant(project_id=4b76c9f802df469db5132e3840e298ac, user_id=da2acb6504db4ef4b7c69be747df5ac0, role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:32.198602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f6b2d-2042-499a-b540-b91eb6bfa23e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:32.213985 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 574/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:41:32 2022] PUT /identity/v3/projects/4b76c9f802df469db5132e3840e298ac/users/da2acb6504db4ef4b7c69be747df5ac0/roles/6f0bec1f01514029a1162fe631d80648 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:41:34.621505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323ac5-c468-44e1-9db1-e783bc2f6b6a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:34.622093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323ac5-c468-44e1-9db1-e783bc2f6b6a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:34.622267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323ac5-c468-44e1-9db1-e783bc2f6b6a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:34.623594 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 550/1124] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:34.629133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443952f-9add-46e8-a2e6-3918d8bd4052 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:34.629608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443952f-9add-46e8-a2e6-3918d8bd4052 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:34.629714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443952f-9add-46e8-a2e6-3918d8bd4052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:34.648883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f443952f-9add-46e8-a2e6-3918d8bd4052 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:34.650081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f443952f-9add-46e8-a2e6-3918d8bd4052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:34.721056 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 575/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:35.048128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323ac5-c468-44e1-9db1-e783bc2f6b6a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:35.155956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323ac5-c468-44e1-9db1-e783bc2f6b6a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:35.192071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:35.192578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:35.192887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:35.193162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] PATH_INFO: `/v3/roles/6f0bec1f01514029a1162fe631d80648` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:35.194615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] RBAC: Authorizing `identity:get_role(role_id=6f0bec1f01514029a1162fe631d80648)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:35.195989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:35.197348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 551/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:35 2022] GET /identity/v3/roles/6f0bec1f01514029a1162fe631d80648 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:35.203549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443952f-9add-46e8-a2e6-3918d8bd4052 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:35.222620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:35.308929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443952f-9add-46e8-a2e6-3918d8bd4052 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:35.344815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:35.345443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:35.345714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:35.345986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] PATH_INFO: `/v3/users/da2acb6504db4ef4b7c69be747df5ac0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:35.347155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] RBAC: Authorizing `identity:get_user(user_id=da2acb6504db4ef4b7c69be747df5ac0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:35.348681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:35.350333 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 576/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:35 2022] GET /identity/v3/users/da2acb6504db4ef4b7c69be747df5ac0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:35.356626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:35.387305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:35.463796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609a295-010a-4bdd-b5ec-792b34466d05 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:35.501395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:35.502032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:35.502288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:35.502563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:35.576046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:35.577448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:35.608352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:35.691353 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 552/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:35.697669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:35.785546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:35.803645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26da55-3bb5-421d-865e-87b4d7a5fde1 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:35.842897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200e3fe-4703-4370-81a1-9eaa6863d036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:35.843434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200e3fe-4703-4370-81a1-9eaa6863d036 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:35.843705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200e3fe-4703-4370-81a1-9eaa6863d036 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:35.843967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200e3fe-4703-4370-81a1-9eaa6863d036 None admin] PATH_INFO: `/v3/projects/4b76c9f802df469db5132e3840e298ac` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:35.845134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200e3fe-4703-4370-81a1-9eaa6863d036 None admin] RBAC: Authorizing `identity:get_project(project_id=4b76c9f802df469db5132e3840e298ac)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:35.846616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200e3fe-4703-4370-81a1-9eaa6863d036 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:35.848593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 577/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:35 2022] GET /identity/v3/projects/4b76c9f802df469db5132e3840e298ac => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:35.855403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:35.960445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70272e47-ce3a-44da-b68c-e3fb6d465687 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:35.989911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:35.998703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ca3ad-5aa9-4b7d-beb9-f217c2777c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:35.999325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca3ad-5aa9-4b7d-beb9-f217c2777c5c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:35.999597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca3ad-5aa9-4b7d-beb9-f217c2777c5c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:35.999864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca3ad-5aa9-4b7d-beb9-f217c2777c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:36.000583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca3ad-5aa9-4b7d-beb9-f217c2777c5c None admin] RBAC: Adding query filter params (role.id=6f0bec1f01514029a1162fe631d80648, scope.project.id=4b76c9f802df469db5132e3840e298ac, user.id=da2acb6504db4ef4b7c69be747df5ac0) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:36.000920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca3ad-5aa9-4b7d-beb9-f217c2777c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:36.002347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca3ad-5aa9-4b7d-beb9-f217c2777c5c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:36.008602 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 553/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:35 2022] GET /identity/v3/role_assignments?user.id=da2acb6504db4ef4b7c69be747df5ac0&scope.project.id=4b76c9f802df469db5132e3840e298ac&role.id=6f0bec1f01514029a1162fe631d80648 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:38.440078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240548cd-63b8-486f-8f1b-4cd1b68c0052 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:38.440625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240548cd-63b8-486f-8f1b-4cd1b68c0052 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:38.440771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240548cd-63b8-486f-8f1b-4cd1b68c0052 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:38.442147 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 578/1131] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:38.449130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05626da7-4fc8-447b-87d9-668eeeb31b57 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:38.449651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05626da7-4fc8-447b-87d9-668eeeb31b57 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:38.449651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05626da7-4fc8-447b-87d9-668eeeb31b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:38.470073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-05626da7-4fc8-447b-87d9-668eeeb31b57 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:38.471279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-05626da7-4fc8-447b-87d9-668eeeb31b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:38.550379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 554/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:38.872532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240548cd-63b8-486f-8f1b-4cd1b68c0052 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:38.991342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240548cd-63b8-486f-8f1b-4cd1b68c0052 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:39.027736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd92e8-2703-4d0d-afbf-1a334769895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:39.028367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd92e8-2703-4d0d-afbf-1a334769895f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:39.028644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd92e8-2703-4d0d-afbf-1a334769895f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:39.028939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd92e8-2703-4d0d-afbf-1a334769895f None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:39.030068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd92e8-2703-4d0d-afbf-1a334769895f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:39.031509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd92e8-2703-4d0d-afbf-1a334769895f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:39.033133 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 579/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:39.040009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05626da7-4fc8-447b-87d9-668eeeb31b57 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:39.149404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05626da7-4fc8-447b-87d9-668eeeb31b57 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:39.176198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:39.188673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762afc3-b1e0-47c0-9273-262c1b20895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:39.189304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762afc3-b1e0-47c0-9273-262c1b20895e None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:39.189586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762afc3-b1e0-47c0-9273-262c1b20895e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:39.189857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762afc3-b1e0-47c0-9273-262c1b20895e None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:39.190517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762afc3-b1e0-47c0-9273-262c1b20895e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:39.192617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762afc3-b1e0-47c0-9273-262c1b20895e None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:39.227613 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 555/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:41:39 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:41:41.698808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86bb3-f835-4afa-8cfb-fa8e8fc4fa6b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:41.699408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86bb3-f835-4afa-8cfb-fa8e8fc4fa6b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:41.699408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86bb3-f835-4afa-8cfb-fa8e8fc4fa6b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:41.700703 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 580/1135] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:41.706342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c92e16-1a9c-4e3b-8ce4-68181b4c09b5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:41.706893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c92e16-1a9c-4e3b-8ce4-68181b4c09b5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:41.706893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c92e16-1a9c-4e3b-8ce4-68181b4c09b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:41.725938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c4c92e16-1a9c-4e3b-8ce4-68181b4c09b5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:41.727126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c4c92e16-1a9c-4e3b-8ce4-68181b4c09b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:41.797028 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 556/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:41:42.123507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d86bb3-f835-4afa-8cfb-fa8e8fc4fa6b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:42.146318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:42.228299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d86bb3-f835-4afa-8cfb-fa8e8fc4fa6b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:42.262206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d026148-b642-4346-b7d6-7e5f9c1af676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:42.262830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d026148-b642-4346-b7d6-7e5f9c1af676 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:42.263102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d026148-b642-4346-b7d6-7e5f9c1af676 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:42.263370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d026148-b642-4346-b7d6-7e5f9c1af676 None admin] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:42.264493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d026148-b642-4346-b7d6-7e5f9c1af676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:42.265959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d026148-b642-4346-b7d6-7e5f9c1af676 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:42.267541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 581/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:41:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:42.273718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c92e16-1a9c-4e3b-8ce4-68181b4c09b5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:42.385441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c92e16-1a9c-4e3b-8ce4-68181b4c09b5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:42.387737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:42.424128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c48d7-418a-4100-93b8-d67757c44f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:42.424761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c48d7-418a-4100-93b8-d67757c44f68 None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:42.425036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c48d7-418a-4100-93b8-d67757c44f68 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:42.425378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c48d7-418a-4100-93b8-d67757c44f68 None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:42.426248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c48d7-418a-4100-93b8-d67757c44f68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:42.427689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c48d7-418a-4100-93b8-d67757c44f68 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:42.450327 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 557/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:41:42 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:41:44.990315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a79d-3234-4e98-9622-591b4726611c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:44.990870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a79d-3234-4e98-9622-591b4726611c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:44.990870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a79d-3234-4e98-9622-591b4726611c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:44.992101 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 582/1139] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:44.997635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4e46e-4746-44a1-933b-69ed3d984114 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:44.998099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4e46e-4746-44a1-933b-69ed3d984114 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:44.998213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4e46e-4746-44a1-933b-69ed3d984114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:45.017753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bdb4e46e-4746-44a1-933b-69ed3d984114 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:45.018944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bdb4e46e-4746-44a1-933b-69ed3d984114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:45.091038 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 558/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:45.421652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6a79d-3234-4e98-9622-591b4726611c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:45.517071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:45.531901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6a79d-3234-4e98-9622-591b4726611c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:45.568532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:45.569204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:45.569475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:45.569745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:45.580413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:45.586004 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:45.587184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:45.588385 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:45.592247 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 583/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:41:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:45.593486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4e46e-4746-44a1-933b-69ed3d984114 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:45.690920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:45.703019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4e46e-4746-44a1-933b-69ed3d984114 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:45.740746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:45.741370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:45.741647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:45.741917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:45.742581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:45.742881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:45.744255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:45.753090 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 559/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:41:45.759749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:45.865819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba559b12-0801-4914-8266-f07e2d0648d8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:45.900556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:45.901215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:45.901463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:45.901749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] PATH_INFO: `/v3/users/b884ec20cfbf48cca39e6b8b06cecf45` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:45.916174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] RBAC: Authorizing `identity:get_user(user_id=b884ec20cfbf48cca39e6b8b06cecf45)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:45.917621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:45.919232 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 584/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:45 2022] GET /identity/v3/users/b884ec20cfbf48cca39e6b8b06cecf45 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:45.925423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:46.030282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab644b-2c82-4f8d-81db-19a2ebc1299a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:46.068522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:46.069174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:46.069455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:46.069725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:46.094461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:46.147614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:46.149129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:46.267443 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 560/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:46.273813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:46.285019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:46.379987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706dcf1-8b32-4d81-aaf1-9e13d39e2d51 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:46.415617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1051ea-b44a-4dc3-bd82-b57d79a0b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:46.416242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1051ea-b44a-4dc3-bd82-b57d79a0b314 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:46.416510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1051ea-b44a-4dc3-bd82-b57d79a0b314 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:46.416809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1051ea-b44a-4dc3-bd82-b57d79a0b314 None admin] PATH_INFO: `/v3/projects/9680e5731fd047cd86aa25837bad677b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:46.417955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1051ea-b44a-4dc3-bd82-b57d79a0b314 None admin] RBAC: Authorizing `identity:get_project(project_id=9680e5731fd047cd86aa25837bad677b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:46.419375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1051ea-b44a-4dc3-bd82-b57d79a0b314 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:46.421027 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 585/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:46 2022] GET /identity/v3/projects/9680e5731fd047cd86aa25837bad677b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:46.428095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:46.485789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:46.535390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed8d2d-bfe1-4879-bb5c-287a8199c3fb None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:46.572045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1356e13-bf97-4151-8b4a-d819405ea75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:46.572648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1356e13-bf97-4151-8b4a-d819405ea75d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:46.572953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1356e13-bf97-4151-8b4a-d819405ea75d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:46.573210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1356e13-bf97-4151-8b4a-d819405ea75d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:46.573931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1356e13-bf97-4151-8b4a-d819405ea75d None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=9680e5731fd047cd86aa25837bad677b, user.id=b884ec20cfbf48cca39e6b8b06cecf45) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:46.574230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1356e13-bf97-4151-8b4a-d819405ea75d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:46.575637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1356e13-bf97-4151-8b4a-d819405ea75d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:46.581184 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 561/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:46 2022] GET /identity/v3/role_assignments?user.id=b884ec20cfbf48cca39e6b8b06cecf45&scope.project.id=9680e5731fd047cd86aa25837bad677b&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:49.041924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea38513-a8bd-4325-a900-360c775f002e None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:49.042515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea38513-a8bd-4325-a900-360c775f002e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:49.042515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea38513-a8bd-4325-a900-360c775f002e None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:49.043786 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 586/1147] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:49.049259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa756ff-f2a8-4c55-b9c0-bfaf312e50f5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:49.049702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa756ff-f2a8-4c55-b9c0-bfaf312e50f5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:49.049790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa756ff-f2a8-4c55-b9c0-bfaf312e50f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:49.069562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-efa756ff-f2a8-4c55-b9c0-bfaf312e50f5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:49.070782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-efa756ff-f2a8-4c55-b9c0-bfaf312e50f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:49.139475 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 562/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:49.471873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea38513-a8bd-4325-a900-360c775f002e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:49.555331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:49.586969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea38513-a8bd-4325-a900-360c775f002e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:49.625068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:49.625699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:49.625978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:49.626244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:49.637424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:49.642319 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:49.643479 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:49.644694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 587/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:41:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:49.649759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa756ff-f2a8-4c55-b9c0-bfaf312e50f5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:49.699934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:49.760626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa756ff-f2a8-4c55-b9c0-bfaf312e50f5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:49.798096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:49.798711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:49.798982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:49.799253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:49.799912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:49.800199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:49.801785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:49.810472 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 563/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:41:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:49.816629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:49.921825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133aca0a-4737-4710-89b0-dae566b7d5f7 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:49.957006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:49.957593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:49.957863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:49.958122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] PATH_INFO: `/v3/users/b884ec20cfbf48cca39e6b8b06cecf45` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:49.959274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] RBAC: Authorizing `identity:get_user(user_id=b884ec20cfbf48cca39e6b8b06cecf45)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:49.960770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:49.962365 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 588/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:49 2022] GET /identity/v3/users/b884ec20cfbf48cca39e6b8b06cecf45 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:49.968617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:50.085117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef3fbf-6d32-46e3-9b58-d04c0ff8772c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:50.095604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:50.126297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175ff7f-2dad-4881-b216-642b08ccade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:50.126965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175ff7f-2dad-4881-b216-642b08ccade6 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:50.127246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175ff7f-2dad-4881-b216-642b08ccade6 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:50.127519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175ff7f-2dad-4881-b216-642b08ccade6 None admin] PATH_INFO: `/v3/projects/9680e5731fd047cd86aa25837bad677b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:50.128909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175ff7f-2dad-4881-b216-642b08ccade6 None admin] RBAC: Authorizing `identity:get_project(project_id=9680e5731fd047cd86aa25837bad677b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:50.130389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175ff7f-2dad-4881-b216-642b08ccade6 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:50.132107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 564/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:49 2022] GET /identity/v3/projects/9680e5731fd047cd86aa25837bad677b => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:50.138438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:50.141510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:50.247693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd570c-104b-4a1b-8cb8-9594aa1e2f12 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:50.284575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250da06e-6bb7-4adb-86e5-e3fc5ffa4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:50.285327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250da06e-6bb7-4adb-86e5-e3fc5ffa4794 None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:50.285599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250da06e-6bb7-4adb-86e5-e3fc5ffa4794 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:50.285864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250da06e-6bb7-4adb-86e5-e3fc5ffa4794 None admin] PATH_INFO: `/v3/projects/9680e5731fd047cd86aa25837bad677b/users/b884ec20cfbf48cca39e6b8b06cecf45/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:50.288197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250da06e-6bb7-4adb-86e5-e3fc5ffa4794 None admin] RBAC: Authorizing `identity:create_grant(project_id=9680e5731fd047cd86aa25837bad677b, user_id=b884ec20cfbf48cca39e6b8b06cecf45, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:50.289677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250da06e-6bb7-4adb-86e5-e3fc5ffa4794 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:50.304902 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 589/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:41:50 2022] PUT /identity/v3/projects/9680e5731fd047cd86aa25837bad677b/users/b884ec20cfbf48cca39e6b8b06cecf45/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:41:52.740714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc0657-f554-4109-97f3-98dce3c254b7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:52.741604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc0657-f554-4109-97f3-98dce3c254b7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:52.741604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc0657-f554-4109-97f3-98dce3c254b7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:52.743683 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 565/1154] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:52.752462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1f04-6aaf-4d16-9d57-2a04a60535dc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:52.752915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1f04-6aaf-4d16-9d57-2a04a60535dc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:52.753121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1f04-6aaf-4d16-9d57-2a04a60535dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:52.778488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f59c1f04-6aaf-4d16-9d57-2a04a60535dc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:52.779693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f59c1f04-6aaf-4d16-9d57-2a04a60535dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:52.841496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:52.853505 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 590/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:53.172275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc0657-f554-4109-97f3-98dce3c254b7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:53.291853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc0657-f554-4109-97f3-98dce3c254b7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:53.330092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:53.330727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:53.331003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:53.331272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:53.343162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:41:53.348104 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:53.349303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:41:53.350170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 566/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:41:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:41:53.356472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1f04-6aaf-4d16-9d57-2a04a60535dc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:53.471381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1f04-6aaf-4d16-9d57-2a04a60535dc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:53.516306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:53.517314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:53.517772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:53.518202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:53.519411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:53.519713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:53.521105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:53.529913 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 591/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:41:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:53.536939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:53.581534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:53.651043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab39c71-d5c9-4a64-b009-3b36e2f7321e None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:53.686458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:53.687054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:53.687344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:53.687593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] PATH_INFO: `/v3/users/b884ec20cfbf48cca39e6b8b06cecf45` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:53.688772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] RBAC: Authorizing `identity:get_user(user_id=b884ec20cfbf48cca39e6b8b06cecf45)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:53.690199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:53.691786 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 567/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:41:53 2022] GET /identity/v3/users/b884ec20cfbf48cca39e6b8b06cecf45 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:53.698049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:53.756142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:53.807288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c9d8d-7e9e-45b4-8428-0dfa444dee88 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:53.843179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:53.843784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:53.844061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:53.844328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:53.922952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:53.924803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:53.988873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:54.038705 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 592/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:54.045120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:54.152346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb2072-a062-4250-9b2a-2e1aedf714b0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:54.154793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:54.188626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e05b2-2c1c-4378-aa63-c117a36103de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:54.189297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e05b2-2c1c-4378-aa63-c117a36103de None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:54.189572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e05b2-2c1c-4378-aa63-c117a36103de None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:54.189844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e05b2-2c1c-4378-aa63-c117a36103de None admin] PATH_INFO: `/v3/projects/9680e5731fd047cd86aa25837bad677b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:54.191154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e05b2-2c1c-4378-aa63-c117a36103de None admin] RBAC: Authorizing `identity:get_project(project_id=9680e5731fd047cd86aa25837bad677b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:54.192747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e05b2-2c1c-4378-aa63-c117a36103de None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:54.194392 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 568/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:41:54 2022] GET /identity/v3/projects/9680e5731fd047cd86aa25837bad677b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:41:54.200907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:54.312634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780333-f5bb-4964-9a57-8da26c5362f9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:54.349461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5b8fc-1c94-4ea9-998e-7988b4190e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:54.350165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5b8fc-1c94-4ea9-998e-7988b4190e98 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:54.350541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5b8fc-1c94-4ea9-998e-7988b4190e98 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:54.350899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5b8fc-1c94-4ea9-998e-7988b4190e98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:54.351718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5b8fc-1c94-4ea9-998e-7988b4190e98 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=9680e5731fd047cd86aa25837bad677b, user.id=b884ec20cfbf48cca39e6b8b06cecf45) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:54.352114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5b8fc-1c94-4ea9-998e-7988b4190e98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:54.353693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5b8fc-1c94-4ea9-998e-7988b4190e98 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:54.360464 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 593/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:41:54 2022] GET /identity/v3/role_assignments?user.id=b884ec20cfbf48cca39e6b8b06cecf45&scope.project.id=9680e5731fd047cd86aa25837bad677b&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:41:56.979083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e17454-f7db-4a16-9da4-088f64e5aab8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:56.979617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e17454-f7db-4a16-9da4-088f64e5aab8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:56.979617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e17454-f7db-4a16-9da4-088f64e5aab8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:56.980930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 569/1162] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:41:56.986604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1dac-3814-4ea0-8d33-be411e52f920 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:56.987071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1dac-3814-4ea0-8d33-be411e52f920 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:56.987158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d1dac-3814-4ea0-8d33-be411e52f920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:57.006580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-660d1dac-3814-4ea0-8d33-be411e52f920 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:41:57.007976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-660d1dac-3814-4ea0-8d33-be411e52f920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:41:57.024721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:57.086699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 594/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:41:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:41:57.409114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e17454-f7db-4a16-9da4-088f64e5aab8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:57.526983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e17454-f7db-4a16-9da4-088f64e5aab8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:57.565073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:57.565684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:57.565974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:57.566369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:57.615242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:57.649954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:57.651452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:57.774430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 570/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:41:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:41:57.781582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1dac-3814-4ea0-8d33-be411e52f920 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:57.898118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d1dac-3814-4ea0-8d33-be411e52f920 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:57.937387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0d844-d6c7-4d23-857d-865e921a9f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:57.938150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0d844-d6c7-4d23-857d-865e921a9f7b None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:57.938503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0d844-d6c7-4d23-857d-865e921a9f7b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:57.938864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0d844-d6c7-4d23-857d-865e921a9f7b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:57.946791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0d844-d6c7-4d23-857d-865e921a9f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:57.948250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0d844-d6c7-4d23-857d-865e921a9f7b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-8ac0d844-d6c7-4d23-857d-865e921a9f7b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:41:57.960990 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:41:57.962404 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:41:57.963348 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 595/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:41:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:41:57.968883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:41:58.049878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:41:58.093663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe16714-ed13-4b9b-b0e7-86c630ae140d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:41:58.146214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649bfa8-58d1-4cda-ad34-e62f45d1464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:41:58.147267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649bfa8-58d1-4cda-ad34-e62f45d1464a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:41:58.147745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649bfa8-58d1-4cda-ad34-e62f45d1464a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:41:58.148066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649bfa8-58d1-4cda-ad34-e62f45d1464a None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:41:58.149043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649bfa8-58d1-4cda-ad34-e62f45d1464a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:41:58.149366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649bfa8-58d1-4cda-ad34-e62f45d1464a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:41:58.150788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649bfa8-58d1-4cda-ad34-e62f45d1464a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:41:58.159422 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 571/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Sep 27 09:41:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:00.627266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52175db5-ec7f-468a-992c-8f87d848adab None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:00.627884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52175db5-ec7f-468a-992c-8f87d848adab None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:00.627884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52175db5-ec7f-468a-992c-8f87d848adab None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:00.629545 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 596/1167] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:42:00.635613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116830a-9422-41f3-8d5a-ac23a0087bd8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:00.636149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116830a-9422-41f3-8d5a-ac23a0087bd8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:00.636149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116830a-9422-41f3-8d5a-ac23a0087bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:00.672888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b116830a-9422-41f3-8d5a-ac23a0087bd8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:42:00.674132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b116830a-9422-41f3-8d5a-ac23a0087bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:42:00.757700 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 572/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:42:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:42:01.127194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52175db5-ec7f-468a-992c-8f87d848adab None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:01.140028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:01.243930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52175db5-ec7f-468a-992c-8f87d848adab None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:01.283734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7d460-b36f-44d6-ad55-f45acb06ac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:01.284361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d460-b36f-44d6-ad55-f45acb06ac4b None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:01.284635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d460-b36f-44d6-ad55-f45acb06ac4b None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:01.284938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d460-b36f-44d6-ad55-f45acb06ac4b None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:01.285647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7d460-b36f-44d6-ad55-f45acb06ac4b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:01.287030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7d460-b36f-44d6-ad55-f45acb06ac4b None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:01.306185 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 597/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Tue Sep 27 09:42:01 2022] POST /identity/v3/domains => generated 267 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:42:03.834776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d1d5-fc56-47b2-8eb7-2eb5281474da None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:03.835446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d1d5-fc56-47b2-8eb7-2eb5281474da None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:03.835446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d1d5-fc56-47b2-8eb7-2eb5281474da None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:03.836681 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 573/1170] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:42:03.842971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcacb4-0add-4331-a3af-30026f80c231 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:03.843474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcacb4-0add-4331-a3af-30026f80c231 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:03.843565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcacb4-0add-4331-a3af-30026f80c231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:03.864555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-48dcacb4-0add-4331-a3af-30026f80c231 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:42:03.865729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-48dcacb4-0add-4331-a3af-30026f80c231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:42:03.938152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:03.946055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 598/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:42:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:42:04.285291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28d1d5-fc56-47b2-8eb7-2eb5281474da None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:04.362107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:04.407007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28d1d5-fc56-47b2-8eb7-2eb5281474da None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:04.448819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db339889-a19b-4593-9ee5-8301ba746b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:04.449604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339889-a19b-4593-9ee5-8301ba746b4f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:04.449947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339889-a19b-4593-9ee5-8301ba746b4f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:04.450266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339889-a19b-4593-9ee5-8301ba746b4f None admin] PATH_INFO: `/v3/domains/971ddb23675d43cdb27c481a42a3e7cf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:04.462773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db339889-a19b-4593-9ee5-8301ba746b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=971ddb23675d43cdb27c481a42a3e7cf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:04.464684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db339889-a19b-4593-9ee5-8301ba746b4f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:04.466989 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 574/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Sep 27 09:42:04 2022] GET /identity/v3/domains/971ddb23675d43cdb27c481a42a3e7cf => generated 267 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:04.473661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcacb4-0add-4331-a3af-30026f80c231 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:04.592860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcacb4-0add-4331-a3af-30026f80c231 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:04.631656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91eaf8-c954-45e0-85ee-ee54adef9bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:04.632295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91eaf8-c954-45e0-85ee-ee54adef9bf5 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:04.632600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91eaf8-c954-45e0-85ee-ee54adef9bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:04.632919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91eaf8-c954-45e0-85ee-ee54adef9bf5 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:04.633592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91eaf8-c954-45e0-85ee-ee54adef9bf5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:04.635195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91eaf8-c954-45e0-85ee-ee54adef9bf5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:04.668894 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 599/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:42:04 2022] POST /identity/v3/projects => generated 361 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:42:07.333969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c392f-cce9-45ef-b0ed-3345106ab77a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:07.334528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c392f-cce9-45ef-b0ed-3345106ab77a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:07.334651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c392f-cce9-45ef-b0ed-3345106ab77a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:07.335972 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 575/1174] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:42:07.342244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9dbc-33c8-49df-8eff-d4ab9b543350 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:07.342751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9dbc-33c8-49df-8eff-d4ab9b543350 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:07.342854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9dbc-33c8-49df-8eff-d4ab9b543350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:07.372006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-99cc9dbc-33c8-49df-8eff-d4ab9b543350 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:42:07.373256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-99cc9dbc-33c8-49df-8eff-d4ab9b543350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:42:07.424122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:07.446665 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 600/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:42:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:42:07.784399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c392f-cce9-45ef-b0ed-3345106ab77a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:07.917005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c392f-cce9-45ef-b0ed-3345106ab77a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:07.952199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:07.968640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457b0ce-b56c-4974-b9d4-9d3c65e412e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:07.969703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b0ce-b56c-4974-b9d4-9d3c65e412e3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:07.970092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b0ce-b56c-4974-b9d4-9d3c65e412e3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:07.970504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b0ce-b56c-4974-b9d4-9d3c65e412e3 None admin] PATH_INFO: `/v3/domains/971ddb23675d43cdb27c481a42a3e7cf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:07.972626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457b0ce-b56c-4974-b9d4-9d3c65e412e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=971ddb23675d43cdb27c481a42a3e7cf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:07.974363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457b0ce-b56c-4974-b9d4-9d3c65e412e3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:07.976628 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 576/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Sep 27 09:42:07 2022] GET /identity/v3/domains/971ddb23675d43cdb27c481a42a3e7cf => generated 267 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:07.982890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9dbc-33c8-49df-8eff-d4ab9b543350 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:08.097593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9dbc-33c8-49df-8eff-d4ab9b543350 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:08.140450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b178bcd-9a22-41a5-b6c9-dfaa82194b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:08.141189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b178bcd-9a22-41a5-b6c9-dfaa82194b5f None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:08.141466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b178bcd-9a22-41a5-b6c9-dfaa82194b5f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:08.141737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b178bcd-9a22-41a5-b6c9-dfaa82194b5f None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:08.142391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b178bcd-9a22-41a5-b6c9-dfaa82194b5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:08.143901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b178bcd-9a22-41a5-b6c9-dfaa82194b5f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:08.171306 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 601/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:42:07 2022] POST /identity/v3/users => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:42:10.813694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ceb36-5edd-4d87-bf28-f5e42413f46a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:10.814355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ceb36-5edd-4d87-bf28-f5e42413f46a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:10.814355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ceb36-5edd-4d87-bf28-f5e42413f46a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:10.815732 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 577/1178] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:42:10.822801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b4fdf-2b41-4382-aefe-bca8b7e52ec4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:10.823327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b4fdf-2b41-4382-aefe-bca8b7e52ec4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:10.823427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b4fdf-2b41-4382-aefe-bca8b7e52ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:10.845428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-491b4fdf-2b41-4382-aefe-bca8b7e52ec4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:42:10.846700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-491b4fdf-2b41-4382-aefe-bca8b7e52ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:42:10.881705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:10.927373 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 602/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:42:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:42:11.253566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ceb36-5edd-4d87-bf28-f5e42413f46a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:11.380928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ceb36-5edd-4d87-bf28-f5e42413f46a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:11.418755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:11.419395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:11.419691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:11.419937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:11.431250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:42:11.436102 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:42:11.437737 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:42:11.438842 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 578/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:42:11.443734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b4fdf-2b41-4382-aefe-bca8b7e52ec4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:11.555056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b4fdf-2b41-4382-aefe-bca8b7e52ec4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:11.596778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:11.597377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:11.597635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:11.597902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:11.598574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:42:11.598875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:11.600276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:11.609873 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 603/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:11.616984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:11.684257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:11.736037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acefaf4-1144-4d5a-8c60-77ef0f8be09d None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:11.775443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:11.776181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:11.776458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:11.776758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] PATH_INFO: `/v3/users/1edbe0a0aa5643158eb975a35b53aefb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:11.793372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] RBAC: Authorizing `identity:get_user(user_id=1edbe0a0aa5643158eb975a35b53aefb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:11.795037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:11.796954 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 579/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:42:11 2022] GET /identity/v3/users/1edbe0a0aa5643158eb975a35b53aefb => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:11.803557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:11.843825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:11.924103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf0a69-c9f2-43ba-8719-de026a29cf38 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:11.966893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:11.967565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:11.967825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:11.968139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:12.043574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:12.045046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:12.074528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:12.162076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 604/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:42:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:42:12.171024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:12.280688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529d5a7-7e96-4a20-b968-9b99c450ded9 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:12.296863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:12.320907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c3bd5-259e-4359-a65d-b5380c94b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:12.321554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c3bd5-259e-4359-a65d-b5380c94b7ea None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:12.321830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c3bd5-259e-4359-a65d-b5380c94b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:12.322101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c3bd5-259e-4359-a65d-b5380c94b7ea None admin] PATH_INFO: `/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:12.323232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c3bd5-259e-4359-a65d-b5380c94b7ea None admin] RBAC: Authorizing `identity:get_project(project_id=f1dd7a758ff943c2b1e6c8936c8e6216)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:12.324715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c3bd5-259e-4359-a65d-b5380c94b7ea None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:12.326361 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 580/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:42:12 2022] GET /identity/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:12.333164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:12.449054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869e4f-bb89-4236-91e0-2b2e9fc997b2 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:12.482553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:12.491951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84917931-f0fc-42e2-b7f0-86836d4c5000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:12.492638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84917931-f0fc-42e2-b7f0-86836d4c5000 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:12.492945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84917931-f0fc-42e2-b7f0-86836d4c5000 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:12.493218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84917931-f0fc-42e2-b7f0-86836d4c5000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:12.493952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84917931-f0fc-42e2-b7f0-86836d4c5000 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=f1dd7a758ff943c2b1e6c8936c8e6216, user.id=1edbe0a0aa5643158eb975a35b53aefb) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:42:12.494259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84917931-f0fc-42e2-b7f0-86836d4c5000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:12.495741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84917931-f0fc-42e2-b7f0-86836d4c5000 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:12.504074 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 605/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:42:12 2022] GET /identity/v3/role_assignments?user.id=1edbe0a0aa5643158eb975a35b53aefb&scope.project.id=f1dd7a758ff943c2b1e6c8936c8e6216&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:15.071495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622bd2-d25a-4ca1-9d83-5b335ee617b3 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:15.072105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622bd2-d25a-4ca1-9d83-5b335ee617b3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:15.072105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622bd2-d25a-4ca1-9d83-5b335ee617b3 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:15.073378 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 581/1186] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:42:15.080688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bcdad-faf3-49f7-b51e-6b83e092156c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:15.081308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bcdad-faf3-49f7-b51e-6b83e092156c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:15.081308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bcdad-faf3-49f7-b51e-6b83e092156c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:15.101951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6f6bcdad-faf3-49f7-b51e-6b83e092156c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:42:15.103170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6f6bcdad-faf3-49f7-b51e-6b83e092156c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:42:15.188267 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 606/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:42:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:42:15.529260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83622bd2-d25a-4ca1-9d83-5b335ee617b3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:15.649270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83622bd2-d25a-4ca1-9d83-5b335ee617b3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:15.687308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:15.687949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:15.688232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:15.688506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:15.699015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:42:15.703996 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:42:15.705132 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:42:15.707121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 582/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:42:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:42:15.711833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bcdad-faf3-49f7-b51e-6b83e092156c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:15.824013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bcdad-faf3-49f7-b51e-6b83e092156c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:15.848425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:15.861061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:15.861679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:15.861951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:15.862219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:15.863088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:42:15.863387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:15.864804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:15.873633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 607/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:42:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:42:15.879945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:15.927247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:15.997388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426288ab-dd28-4a99-9679-0ae27d77bafb None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:16.037475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:16.038558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:16.038838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:16.039113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] PATH_INFO: `/v3/users/1edbe0a0aa5643158eb975a35b53aefb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:16.040265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] RBAC: Authorizing `identity:get_user(user_id=1edbe0a0aa5643158eb975a35b53aefb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:16.041732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:16.043334 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 583/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:42:15 2022] GET /identity/v3/users/1edbe0a0aa5643158eb975a35b53aefb => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:16.051385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:16.170820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ca18-453d-4cc4-b9c9-be3cee9fff72 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:16.210589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4f653-b148-4107-aef6-2dc1b1889a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:16.211237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4f653-b148-4107-aef6-2dc1b1889a45 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:16.211513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4f653-b148-4107-aef6-2dc1b1889a45 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:16.211779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4f653-b148-4107-aef6-2dc1b1889a45 None admin] PATH_INFO: `/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:16.212943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4f653-b148-4107-aef6-2dc1b1889a45 None admin] RBAC: Authorizing `identity:get_project(project_id=f1dd7a758ff943c2b1e6c8936c8e6216)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:16.214391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4f653-b148-4107-aef6-2dc1b1889a45 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:16.216066 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 608/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:42:16 2022] GET /identity/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216 => generated 361 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:16.223678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:16.335947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0cd4f-9b60-413c-8615-17553daf25ff None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:16.347292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:16.388615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667265e-b481-4419-b3eb-ce7d3f71d9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:16.389745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667265e-b481-4419-b3eb-ce7d3f71d9f4 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:16.390359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667265e-b481-4419-b3eb-ce7d3f71d9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:16.390814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667265e-b481-4419-b3eb-ce7d3f71d9f4 None admin] PATH_INFO: `/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216/users/1edbe0a0aa5643158eb975a35b53aefb/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:16.394363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667265e-b481-4419-b3eb-ce7d3f71d9f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1dd7a758ff943c2b1e6c8936c8e6216, user_id=1edbe0a0aa5643158eb975a35b53aefb, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:16.396534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667265e-b481-4419-b3eb-ce7d3f71d9f4 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:16.416338 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 584/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:42:16 2022] PUT /identity/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216/users/1edbe0a0aa5643158eb975a35b53aefb/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:42:18.993231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9a17c-4107-496a-9225-57bf89aa972c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:18.993801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9a17c-4107-496a-9225-57bf89aa972c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:18.993801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9a17c-4107-496a-9225-57bf89aa972c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:18.995014 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 609/1193] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:42:19.002052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931510b-2c8d-40bb-834d-2a9e6d026727 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:19.002598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931510b-2c8d-40bb-834d-2a9e6d026727 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:19.002848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931510b-2c8d-40bb-834d-2a9e6d026727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:19.029695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9931510b-2c8d-40bb-834d-2a9e6d026727 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:42:19.031612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9931510b-2c8d-40bb-834d-2a9e6d026727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:42:19.117659 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 585/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:42:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:42:19.470734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9a17c-4107-496a-9225-57bf89aa972c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:19.535399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:19.589695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9a17c-4107-496a-9225-57bf89aa972c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:19.629714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:19.630367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:19.630638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:19.630911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:19.642347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:42:19.647746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:42:19.648917 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:42:19.650235 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 610/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:42:19.655441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931510b-2c8d-40bb-834d-2a9e6d026727 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:19.777206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:19.782661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931510b-2c8d-40bb-834d-2a9e6d026727 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:19.821829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6305-989a-4bca-b666-170569475a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:19.822484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6305-989a-4bca-b666-170569475a10 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:19.822803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6305-989a-4bca-b666-170569475a10 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:19.823076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6305-989a-4bca-b666-170569475a10 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:19.823815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c6305-989a-4bca-b666-170569475a10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:42:19.824150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c6305-989a-4bca-b666-170569475a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:19.825670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c6305-989a-4bca-b666-170569475a10 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:19.835701 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 586/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:19.842440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:19.957689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569e238-e5f1-4851-9c68-8d9ff73ed6be None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:19.970623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:20.000601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:20.001378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:20.001700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:20.001973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] PATH_INFO: `/v3/users/1edbe0a0aa5643158eb975a35b53aefb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:20.003375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] RBAC: Authorizing `identity:get_user(user_id=1edbe0a0aa5643158eb975a35b53aefb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:20.005108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:20.006895 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 611/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Sep 27 09:42:19 2022] GET /identity/v3/users/1edbe0a0aa5643158eb975a35b53aefb => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:42:20.013064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6305-989a-4bca-b666-170569475a10 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:20.134220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6305-989a-4bca-b666-170569475a10 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:20.179291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:20.179916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:20.180202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:20.180456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:20.211842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:20.260990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:20.262458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:20.371006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 587/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:42:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:42:20.377410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:20.495701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bee8-472e-4279-9a9f-d3da09eca9fd None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:20.533696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594906-009d-4b1e-8d78-806e2e2a9c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:20.534517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594906-009d-4b1e-8d78-806e2e2a9c11 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:20.534793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594906-009d-4b1e-8d78-806e2e2a9c11 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:20.535168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594906-009d-4b1e-8d78-806e2e2a9c11 None admin] PATH_INFO: `/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:20.536333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594906-009d-4b1e-8d78-806e2e2a9c11 None admin] RBAC: Authorizing `identity:get_project(project_id=f1dd7a758ff943c2b1e6c8936c8e6216)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:20.538254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594906-009d-4b1e-8d78-806e2e2a9c11 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:20.540092 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 612/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Sep 27 09:42:20 2022] GET /identity/v3/projects/f1dd7a758ff943c2b1e6c8936c8e6216 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:42:20.548006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:42:20.623430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:20.669684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43893f-552d-4b05-be4b-e1853076752a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:42:20.709301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0b315-c7d5-4dde-99b5-53f8228f9850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:42:20.709916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b315-c7d5-4dde-99b5-53f8228f9850 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:42:20.710186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b315-c7d5-4dde-99b5-53f8228f9850 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:42:20.710453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b315-c7d5-4dde-99b5-53f8228f9850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:42:20.711182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0b315-c7d5-4dde-99b5-53f8228f9850 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=f1dd7a758ff943c2b1e6c8936c8e6216, user.id=1edbe0a0aa5643158eb975a35b53aefb) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:42:20.711486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0b315-c7d5-4dde-99b5-53f8228f9850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:42:20.712984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0b315-c7d5-4dde-99b5-53f8228f9850 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:42:20.719524 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 588/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:42:20 2022] GET /identity/v3/role_assignments?user.id=1edbe0a0aa5643158eb975a35b53aefb&scope.project.id=f1dd7a758ff943c2b1e6c8936c8e6216&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:42:30.528574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=76 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:42:30.717136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=38 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:26.643460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3b9fd-fcc7-4cf2-b7ec-b5ce2c6e7665 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:26.644054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3b9fd-fcc7-4cf2-b7ec-b5ce2c6e7665 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:26.644054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3b9fd-fcc7-4cf2-b7ec-b5ce2c6e7665 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:26.647122 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 613/1201] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:43:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:43:26.652211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdf6a6-8373-4848-a591-bc0570a8f7cb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:26.652785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdf6a6-8373-4848-a591-bc0570a8f7cb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:26.652785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdf6a6-8373-4848-a591-bc0570a8f7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:26.678544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b3fdf6a6-8373-4848-a591-bc0570a8f7cb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:43:26.679966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b3fdf6a6-8373-4848-a591-bc0570a8f7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:43:26.811171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 589/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:43:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:43:27.156426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3b9fd-fcc7-4cf2-b7ec-b5ce2c6e7665 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:27.322571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3b9fd-fcc7-4cf2-b7ec-b5ce2c6e7665 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:27.386017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:27.387230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:27.387708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:27.388285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:27.410465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:27.412538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:27.438138 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:27.441233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:27.443469 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:27.443469 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:27.443469 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:27.443469 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 614/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:27.449078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdf6a6-8373-4848-a591-bc0570a8f7cb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:27.625187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdf6a6-8373-4848-a591-bc0570a8f7cb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:27.662477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:27.669968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a677-b1ff-4eef-8e63-b5d9cb0bdaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:27.670614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a677-b1ff-4eef-8e63-b5d9cb0bdaa3 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:27.670933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a677-b1ff-4eef-8e63-b5d9cb0bdaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:27.671160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a677-b1ff-4eef-8e63-b5d9cb0bdaa3 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:27.675019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a677-b1ff-4eef-8e63-b5d9cb0bdaa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:27.675019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a677-b1ff-4eef-8e63-b5d9cb0bdaa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:27.675019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a677-b1ff-4eef-8e63-b5d9cb0bdaa3 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:27.688225 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 590/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:27.695092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:27.767357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:27.847705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a507254-c98f-4bea-901f-55888182bbf6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:27.903100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b38c41-e4e7-4c98-b81f-8ad517185438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:27.904289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b38c41-e4e7-4c98-b81f-8ad517185438 None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:27.904840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b38c41-e4e7-4c98-b81f-8ad517185438 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:27.905350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b38c41-e4e7-4c98-b81f-8ad517185438 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:27.906502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b38c41-e4e7-4c98-b81f-8ad517185438 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:27.909048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b38c41-e4e7-4c98-b81f-8ad517185438 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:27.945076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 615/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Sep 27 09:43:27 2022] POST /identity/v3/users => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:43:30.695978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ced1f0-45d4-48ce-87b0-6e5e28b45b08 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:30.696613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ced1f0-45d4-48ce-87b0-6e5e28b45b08 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:30.696613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ced1f0-45d4-48ce-87b0-6e5e28b45b08 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:30.698004 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 591/1206] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:43:30.704298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80eb7-cdbf-4428-9e1f-2e21e1928403 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:30.704898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80eb7-cdbf-4428-9e1f-2e21e1928403 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:30.704898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80eb7-cdbf-4428-9e1f-2e21e1928403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:30.725698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fef80eb7-cdbf-4428-9e1f-2e21e1928403 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:43:30.726905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fef80eb7-cdbf-4428-9e1f-2e21e1928403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:43:30.811054 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 616/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:43:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:43:31.153246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ced1f0-45d4-48ce-87b0-6e5e28b45b08 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:31.285851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ced1f0-45d4-48ce-87b0-6e5e28b45b08 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:31.330468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:31.331112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:31.331391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:31.331728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:31.344961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:43:31.350253 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:31.351493 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:43:31.352371 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:31.356532 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 592/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:43:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:31.357719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef80eb7-cdbf-4428-9e1f-2e21e1928403 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:31.405184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:31.478350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef80eb7-cdbf-4428-9e1f-2e21e1928403 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:31.521664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:31.522353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:31.522627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:31.522922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:31.523642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:31.523940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:31.525435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:31.535640 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 617/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:43:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:43:31.542183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:31.610154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:31.665701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c160e9c-20a8-452e-974b-271ae26bbc98 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:31.707633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:31.708402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:31.708690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:31.708873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:31.720522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:31.722011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:31.738303 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:31.739581 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:31.740489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:31.740489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:31.740489 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:31.740489 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 593/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:31.745754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:31.870945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:31.875061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c00dc-bfd9-45cd-adb2-db643a37d7bf None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:31.920974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:31.921699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:31.921986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:31.922258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:31.923072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:31.923383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:31.925120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:31.940194 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 618/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:31.946026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:32.057979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d766-78d7-4921-a43d-02d9d799d40c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:32.059939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:32.097829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:32.098480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:32.098752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:32.099073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:32.108504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:32.110288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:43:32.125225 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:32.126538 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:32.127791 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:32.132788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 594/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:32.133553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:32.254378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211bb4-53e1-4a1d-9af7-78831e55f2e4 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:32.302380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:32.302992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:32.303263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:32.303530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:32.304252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:32.304549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:32.306017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:32.327678 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 619/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 27 09:43:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:32.334831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:32.477414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90233f1-7dbc-4862-99aa-3f894fc2cf2f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:32.515507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:32.516180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:32.516538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:32.516891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:32.555619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:32.602259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:32.605339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:32.721527 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 595/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:43:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:43:32.728196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:32.749550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:32.840724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade767e6-cf15-420f-a0d6-7a384954d355 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:32.881542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:32.882597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:32.882976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:32.883344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:32.895876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:32.897667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:32.921341 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:32.922916 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:32.923910 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:32.923910 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:32.923910 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:32.923910 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 620/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:32.928865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:32.996266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:33.043313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cae73-464a-431b-a359-c32c4ab60882 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:33.085642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:33.086269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:33.086542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:33.086834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:33.087467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:33.087766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:33.090533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:33.104927 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 596/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:33.110803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:33.203033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:33.242387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188d1e9-6cca-47f9-90a5-e59337a684f0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:33.277263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:33.277962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:33.278242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:33.278515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:33.288861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:33.290958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:43:33.305813 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:33.307038 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:33.308085 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:33.311143 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 621/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:33.317341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:33.461053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e307132-b121-4ba0-8fcf-dabad902aca7 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:33.489567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:33.506352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbf159-c178-4660-aa2c-a86208095f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:33.507029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbf159-c178-4660-aa2c-a86208095f43 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:33.507301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbf159-c178-4660-aa2c-a86208095f43 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:33.507640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbf159-c178-4660-aa2c-a86208095f43 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:33.508396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbf159-c178-4660-aa2c-a86208095f43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:33.508792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbf159-c178-4660-aa2c-a86208095f43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:33.510235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbf159-c178-4660-aa2c-a86208095f43 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:33.525469 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 597/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:43:33.531796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:33.646429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f9dc2-4f5e-41b5-b4ca-3d72eaef93ad None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:33.679712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:33.689377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7dc64-2360-40ed-8137-f5f083ff012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:33.690112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7dc64-2360-40ed-8137-f5f083ff012c None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:33.690425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7dc64-2360-40ed-8137-f5f083ff012c None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:33.690740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7dc64-2360-40ed-8137-f5f083ff012c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:33.691630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7dc64-2360-40ed-8137-f5f083ff012c None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=19f8118e581143ffb079e2b1aea3870d) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:33.692023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7dc64-2360-40ed-8137-f5f083ff012c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:33.693552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7dc64-2360-40ed-8137-f5f083ff012c None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:33.700610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 622/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:43:33 2022] GET /identity/v3/role_assignments?user.id=19f8118e581143ffb079e2b1aea3870d&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:36.383749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82834e0-eb9b-48a5-a33b-4421728123cd None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:36.384552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82834e0-eb9b-48a5-a33b-4421728123cd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:36.384552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82834e0-eb9b-48a5-a33b-4421728123cd None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:36.386154 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 598/1220] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:43:36.393310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a26b7f-4ce9-4396-9434-93af636785a5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:36.393837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a26b7f-4ce9-4396-9434-93af636785a5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:36.394275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a26b7f-4ce9-4396-9434-93af636785a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:36.415251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a1a26b7f-4ce9-4396-9434-93af636785a5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:43:36.416582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a1a26b7f-4ce9-4396-9434-93af636785a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:43:36.510383 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 623/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:43:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:43:36.843365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82834e0-eb9b-48a5-a33b-4421728123cd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:36.977311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82834e0-eb9b-48a5-a33b-4421728123cd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:37.016281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:37.016961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:37.017240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:37.017536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:37.029752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:43:37.034800 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:37.036017 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:37.037266 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 599/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:37.043476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a26b7f-4ce9-4396-9434-93af636785a5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:37.163776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a26b7f-4ce9-4396-9434-93af636785a5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:37.193177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:37.205276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:37.205889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:37.206204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:37.206429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:37.207181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] RBAC: Adding query filter params (name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:37.207663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:37.209069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:37.217690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 624/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:43:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:37.224347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:37.277583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:37.353982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f4a64-048c-404d-9fe8-be2257b36df2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:37.401158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:37.402155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:37.402155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:37.402322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:37.415737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:37.417208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:37.437506 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:37.438770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:37.439867 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:37.439867 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:37.439867 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:37.439867 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 600/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:43:37.445628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:37.585483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e59bba-18aa-464d-ad83-d129179ed66e None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:37.626626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:37.627497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:37.627948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:37.628391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:37.629727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:37.630112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:37.631738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:37.646279 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 625/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:37.652121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:37.753079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:37.760593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ca6b2-1f31-4792-8cc6-abdc21ae85d0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:37.805799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:37.807097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:37.807652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:37.808161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] PATH_INFO: `/v3/users/placement` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:37.820595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:37.822818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:43:37.839264 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:37.840367 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:37.846248 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 601/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:37.847676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:37.907850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:37.989818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378309ab-3d90-42b8-9e60-581c72a2a13f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:38.047332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:38.048232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:38.048629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:38.049170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:38.050161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:38.050573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:38.052072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:38.073513 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 626/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 27 09:43:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:43:38.079648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:38.190420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb544f8-8db1-4ad1-abe7-62f8beb1f77b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:38.229855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:38.230488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:38.230760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:38.231028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:38.242141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:38.243604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:38.258110 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:38.259958 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:38.261713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:38.261713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:38.261713 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:38.261713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 602/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:38.266502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:38.385398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a615-742d-4f98-ad39-594038ee23a8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:38.385398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:38.421113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:38.421732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:38.422030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:38.422267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:38.422924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:38.423217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:38.424595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:38.439796 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 627/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:38.448295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:38.462119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:38.600957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabe90d-0cca-42fd-9297-f1794ceac057 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:38.644883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:38.645728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:38.646261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:38.646723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:38.655220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:38.656729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:43:38.670039 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:38.671205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:38.672230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 603/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:43:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:38.678735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:38.796826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cce033-8e46-4779-84bc-b03b06e011ab None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:38.838034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fd380-7d22-4fd1-97f1-594fa045bc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:38.838712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fd380-7d22-4fd1-97f1-594fa045bc75 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:38.839002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fd380-7d22-4fd1-97f1-594fa045bc75 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:38.839306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fd380-7d22-4fd1-97f1-594fa045bc75 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:38.840301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fd380-7d22-4fd1-97f1-594fa045bc75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:38.840703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fd380-7d22-4fd1-97f1-594fa045bc75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:38.842209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fd380-7d22-4fd1-97f1-594fa045bc75 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:38.858192 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 628/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:43:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:38.865544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:38.946549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:38.990500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4221c8-ed46-4b64-a582-1d15aaa5075b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:39.032328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167b00b-7723-4ad9-85fc-bdf14bc8adc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:39.033026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167b00b-7723-4ad9-85fc-bdf14bc8adc9 None admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:39.033304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167b00b-7723-4ad9-85fc-bdf14bc8adc9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:39.033576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167b00b-7723-4ad9-85fc-bdf14bc8adc9 None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/19f8118e581143ffb079e2b1aea3870d/roles/d2b916599fc849669bff95d9ac632b79` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:39.051007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167b00b-7723-4ad9-85fc-bdf14bc8adc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=19f8118e581143ffb079e2b1aea3870d, role_id=d2b916599fc849669bff95d9ac632b79)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:39.052727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167b00b-7723-4ad9-85fc-bdf14bc8adc9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:39.066842 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 604/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:43:38 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/19f8118e581143ffb079e2b1aea3870d/roles/d2b916599fc849669bff95d9ac632b79 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:43:41.572801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57914dc-5b07-4a9b-8483-ea8ed7bf4970 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:41.573380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57914dc-5b07-4a9b-8483-ea8ed7bf4970 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:41.573495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57914dc-5b07-4a9b-8483-ea8ed7bf4970 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:41.574876 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 629/1233] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:43:41.581030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ad5e1-0551-41bf-940e-5d665ad30e4a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:41.584990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ad5e1-0551-41bf-940e-5d665ad30e4a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:41.584990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ad5e1-0551-41bf-940e-5d665ad30e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:41.605765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7e3ad5e1-0551-41bf-940e-5d665ad30e4a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:43:41.606879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7e3ad5e1-0551-41bf-940e-5d665ad30e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:43:41.708714 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 605/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:43:42.042469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57914dc-5b07-4a9b-8483-ea8ed7bf4970 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:42.074032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:42.193965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57914dc-5b07-4a9b-8483-ea8ed7bf4970 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:42.238574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:42.239229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:42.239503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:42.239770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] PATH_INFO: `/v3/roles/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:42.255799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:43:42.260800 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:42.261882 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:42.262804 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 630/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Sep 27 09:43:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:42.268638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ad5e1-0551-41bf-940e-5d665ad30e4a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:42.320858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:42.404835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ad5e1-0551-41bf-940e-5d665ad30e4a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:42.458087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:42.458720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:42.458998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:42.459258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:42.459929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] RBAC: Adding query filter params (name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:42.460228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:42.461664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:42.471030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 606/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Sep 27 09:43:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:43:42.480004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:42.581216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:42.611860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289a596-790b-4949-832d-1bad6be40686 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:42.655567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:42.656353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:42.657769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:42.658204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:42.673541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:42.675631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:42.693315 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:42.694507 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:42.695777 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:42.695777 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:42.695777 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:42.695777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 631/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:42.701147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:42.816557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58b372-14e3-4f74-bd0b-3ae4662773ac None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:42.831469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:42.868310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:42.868969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:42.869324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:42.869601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:42.870325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:42.870654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:42.872024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:42.892639 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 607/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:42.894590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:43.018536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c85b77-d09d-4c5e-9544-c887b25a1fad None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:43.071277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:43.072076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:43.072465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:43.072982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:43.077034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:43.085296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:43.087601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:43:43.102683 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:43.104223 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:43.105364 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 632/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:43.111625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:43.227500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34837c51-d83e-4dbb-9cea-c3fb079cdba2 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:43.263213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:43.263812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:43.264090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:43.264351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:43.265112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:43.265416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:43.266841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:43.286936 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 608/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 27 09:43:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:43.293597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:43.416627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48db51-0783-459b-a526-68cec10a24ff None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:43.462013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:43.462752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:43.463090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:43.463322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:43.545973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:43.554665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:43.556161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:43.687720 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 633/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:43:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:43:43.694486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:43.731826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:43.826302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6a362-329b-4f6d-8e9f-0ef75e97a7ff None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:43.871849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:43.872530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:43.872805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:43.873073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:43.885308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:43.886798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:43.908053 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:43.909174 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:43.910903 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:43.910903 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:43.910903 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:43.910903 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 609/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:43.919050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:44.055105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:44.064046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9438a-07eb-47f8-8f10-c7a10dd4d1a9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:44.104402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:44.105381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:44.105744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:44.106026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:44.106772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:44.107153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:44.109016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:44.125780 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 634/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:44.133224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:44.225600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:44.252561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fd0c7-30a9-4a43-b9b0-5ec83163fad8 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:44.293287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:44.293990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:44.294288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:44.294535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:44.302290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:44.303761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:43:44.315974 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:44.322740 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:44.323988 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 610/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:43:44.325907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:44.466203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73f35-eeec-4e9c-83f7-3b093698fed5 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:44.513442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6913d-b22f-4488-8ad5-decfcca54fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:44.514208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6913d-b22f-4488-8ad5-decfcca54fa3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:44.514497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6913d-b22f-4488-8ad5-decfcca54fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:44.514820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6913d-b22f-4488-8ad5-decfcca54fa3 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:44.515691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6913d-b22f-4488-8ad5-decfcca54fa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:44.516052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6913d-b22f-4488-8ad5-decfcca54fa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:44.517653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6913d-b22f-4488-8ad5-decfcca54fa3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:44.531833 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 635/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:44.538791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:44.664199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a48887-d417-484d-b39c-6d609877802b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:44.713468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b5c6e-599d-469f-9a19-83a842e8f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:44.714486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b5c6e-599d-469f-9a19-83a842e8f577 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:44.714958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b5c6e-599d-469f-9a19-83a842e8f577 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:44.715419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b5c6e-599d-469f-9a19-83a842e8f577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:44.716731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b5c6e-599d-469f-9a19-83a842e8f577 None admin] RBAC: Adding query filter params (role.id=d2b916599fc849669bff95d9ac632b79, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=19f8118e581143ffb079e2b1aea3870d) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:44.717269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b5c6e-599d-469f-9a19-83a842e8f577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:44.720035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b5c6e-599d-469f-9a19-83a842e8f577 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:44.731395 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 611/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:43:44 2022] GET /identity/v3/role_assignments?user.id=19f8118e581143ffb079e2b1aea3870d&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=d2b916599fc849669bff95d9ac632b79 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:47.512555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b649c-4eed-476a-a51d-9c36a416b8c2 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:47.513637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b649c-4eed-476a-a51d-9c36a416b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:47.513637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b649c-4eed-476a-a51d-9c36a416b8c2 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:47.516711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 636/1247] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:43:47.523580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0454b1-34e4-414a-9a93-47b1da15925a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:47.524389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0454b1-34e4-414a-9a93-47b1da15925a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:47.524389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0454b1-34e4-414a-9a93-47b1da15925a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:47.530975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:47.546713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2d0454b1-34e4-414a-9a93-47b1da15925a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:43:47.548581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2d0454b1-34e4-414a-9a93-47b1da15925a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:43:47.653319 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 612/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:43:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:43:47.980637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b649c-4eed-476a-a51d-9c36a416b8c2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:48.002907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:48.129236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b649c-4eed-476a-a51d-9c36a416b8c2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:48.201457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:48.202875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:48.203480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:48.203972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:48.223494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:43:48.233309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:48.235245 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:43:48.237686 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:48.243308 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 637/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 09:43:48.245088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0454b1-34e4-414a-9a93-47b1da15925a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:48.378182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0454b1-34e4-414a-9a93-47b1da15925a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:48.425017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:48.432025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:48.432858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:48.433220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:48.433589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:48.434616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:48.435079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:48.437167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:48.450987 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 613/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:43:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:48.461129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:48.577093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:48.611424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9f71b-1154-4ba5-9b1d-e2feb3a799f9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:48.653785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:48.654578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:48.654854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:48.655120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:48.670958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:48.672496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:48.691434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:48.693128 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:48.694512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:48.694512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:48.694512 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:48.694512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 638/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:48.699067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:48.831901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b7734-39d9-44d1-bcf4-70321d401447 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:48.883504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:48.884265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:48.884536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:48.885016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:48.885687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:48.886011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:48.887513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:48.904196 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 614/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:48.915186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:49.066595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deb615-467e-4009-9529-3de6ae5263dd None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:49.100213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:49.112288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:49.113434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:49.113901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:49.114354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] PATH_INFO: `/v3/users/placement` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:49.128895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:49.131576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:43:49.153233 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:49.154695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:49.155692 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 639/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:49.162052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:49.253092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:49.322787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f672d63-fb7f-4d96-8fb6-ef7c3bb1f81f None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:49.374448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:49.375323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:49.375709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:49.376068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:49.377044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:49.377443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:49.379226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:49.401755 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 615/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 27 09:43:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:49.409227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:49.530436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5726c0c-974d-42d3-966d-19639d6bd73f None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:49.574178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:49.575231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:49.575714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:49.576192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:49.661426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:49.677524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:49.679037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:49.814948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 640/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:43:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:43:49.816361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:49.951306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e0fad-ee91-49d0-9f6f-0e96b23a0fde None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:49.968321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:49.997245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:49.998134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:49.998476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:49.999017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:50.011208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:50.013229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:50.030015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:50.031232 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:50.032300 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:50.032300 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:50.032300 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:50.032300 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 616/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:50.037539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:50.153708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:50.174102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31325acb-e493-4657-a7b4-cad865c5f038 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:50.216031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:50.216545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:50.216859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:50.217117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:50.217761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:50.218058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:50.219447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:50.232253 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 641/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:43:50.238411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:50.346289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c8246-ad04-440a-8a11-9177706dec2a None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:50.386819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:50.387542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:50.387867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:50.388177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:50.395800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:50.397300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:43:50.409960 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:50.411752 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:50.414025 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 617/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:43:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:50.419732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:50.533498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4dbf0-186f-4d8c-aec6-5e38ffca5bba None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:50.572082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60bab7-6984-4fb6-bd17-b494b14be004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:50.572712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60bab7-6984-4fb6-bd17-b494b14be004 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:50.572995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60bab7-6984-4fb6-bd17-b494b14be004 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:50.573261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60bab7-6984-4fb6-bd17-b494b14be004 None admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:50.573958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60bab7-6984-4fb6-bd17-b494b14be004 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:50.574256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60bab7-6984-4fb6-bd17-b494b14be004 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:50.575677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60bab7-6984-4fb6-bd17-b494b14be004 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:50.589533 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 642/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:50.596175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:50.625595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:50.719761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39915650-2fd4-4958-8547-9d4bcb5aa137 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:50.758441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ade0b-31f7-442b-b8a8-7e23edcd9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:50.759061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ade0b-31f7-442b-b8a8-7e23edcd9250 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:50.759338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ade0b-31f7-442b-b8a8-7e23edcd9250 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:50.759609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ade0b-31f7-442b-b8a8-7e23edcd9250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:50.760343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ade0b-31f7-442b-b8a8-7e23edcd9250 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=19f8118e581143ffb079e2b1aea3870d) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:50.760645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ade0b-31f7-442b-b8a8-7e23edcd9250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:50.762131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ade0b-31f7-442b-b8a8-7e23edcd9250 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:50.768242 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 618/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:43:50 2022] GET /identity/v3/role_assignments?user.id=19f8118e581143ffb079e2b1aea3870d&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:53.228646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267ee87-60b8-40f2-881c-98810f44404e None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:53.229269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267ee87-60b8-40f2-881c-98810f44404e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:53.229269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267ee87-60b8-40f2-881c-98810f44404e None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:53.230564 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 643/1261] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:43:53.237204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548971-2042-4b3e-861e-7d328ca1c472 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:53.237893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548971-2042-4b3e-861e-7d328ca1c472 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:53.238335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548971-2042-4b3e-861e-7d328ca1c472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:53.259667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fb548971-2042-4b3e-861e-7d328ca1c472 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:43:53.260899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fb548971-2042-4b3e-861e-7d328ca1c472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:43:53.333770 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 619/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:43:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:43:53.649979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267ee87-60b8-40f2-881c-98810f44404e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:53.661427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:53.760197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267ee87-60b8-40f2-881c-98810f44404e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:53.797102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:53.797738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:53.798006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:53.798269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:53.809485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:43:53.814281 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:53.815427 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:53.816621 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 644/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:43:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:53.822054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb548971-2042-4b3e-861e-7d328ca1c472 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:53.842865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:53.934762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb548971-2042-4b3e-861e-7d328ca1c472 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:53.973297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:53.973920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:53.974208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:53.974477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:53.975153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:53.975456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:53.976882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:53.985668 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 620/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:43:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:53.992244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:54.073910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:54.109394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee851d0f-edaa-4141-8869-b6045389d8b8 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:54.154041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:54.154682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:54.154959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:54.155312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:54.167825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:54.169330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:54.185261 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:54.186443 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:54.187661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:54.187661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:54.187661 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:54.191930 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 645/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:54.193326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:54.278238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:54.299394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01717f22-f32c-499c-aa52-b06f1e065464 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:54.337531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749af812-6251-4288-8156-530a282f2c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:54.338251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749af812-6251-4288-8156-530a282f2c76 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:54.338567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749af812-6251-4288-8156-530a282f2c76 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:54.338898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749af812-6251-4288-8156-530a282f2c76 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:54.339609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749af812-6251-4288-8156-530a282f2c76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:54.339952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749af812-6251-4288-8156-530a282f2c76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:54.341452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749af812-6251-4288-8156-530a282f2c76 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:54.356701 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 621/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:43:54.364968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:54.482306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c2b53-4e68-4034-871c-0894aaea6e14 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:54.505249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:54.525430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:54.526124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:54.526403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:54.526682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:54.537814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:54.539700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:43:54.553756 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:54.555003 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:54.556208 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 646/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:54.564445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749af812-6251-4288-8156-530a282f2c76 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:54.679798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749af812-6251-4288-8156-530a282f2c76 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:54.715541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:54.716166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:54.716440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:54.716734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:54.717508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:54.717813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:54.719250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:54.729436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:54.739706 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 622/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 27 09:43:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:54.745608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:54.853398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdb18d-39dc-4506-a092-819253ef91e9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:54.888432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:54.889092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:54.889339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:54.889609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:54.901299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:54.902750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:54.918497 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:54.919731 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:54.920960 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:54.920960 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:54.920960 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:54.920960 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 647/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:54.927004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:55.047348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc0d78-3c07-48ae-a09a-c990422fb99b None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:55.099839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:55.100466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:55.100765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:55.101042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:55.101714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:55.102013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:55.103429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:55.119826 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 623/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:55.127882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:55.174608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:55.254637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fcc68-c21a-4b03-876c-8577f61e2a99 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:55.299451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:55.299990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:55.300315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:55.300585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:55.310080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:55.311555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:43:55.324557 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:55.325746 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:43:55.327667 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 648/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:55.334245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:55.404143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:55.454186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed8779-bcef-44fc-853e-a0a1d163bbd0 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:55.497695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bf736-83e2-468e-a137-647b5220d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:55.498398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bf736-83e2-468e-a137-647b5220d1d9 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:55.498774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bf736-83e2-468e-a137-647b5220d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:55.499126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bf736-83e2-468e-a137-647b5220d1d9 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:55.500003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bf736-83e2-468e-a137-647b5220d1d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:55.500387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bf736-83e2-468e-a137-647b5220d1d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:55.501993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bf736-83e2-468e-a137-647b5220d1d9 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:55.515895 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 624/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:55.526585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:55.646587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:55.664581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055a3c3-14a2-4e19-ac59-3ff2fc4a4cb6 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:55.711220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c2f5f-4b41-45a6-9e63-731306c3251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:55.711220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2f5f-4b41-45a6-9e63-731306c3251d None admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:55.711220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2f5f-4b41-45a6-9e63-731306c3251d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:55.711220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2f5f-4b41-45a6-9e63-731306c3251d None admin] PATH_INFO: `/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/19f8118e581143ffb079e2b1aea3870d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:55.713049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c2f5f-4b41-45a6-9e63-731306c3251d None admin] RBAC: Authorizing `identity:create_grant(project_id=cd30607a8a404dd2a608b2ec915dc81d, user_id=19f8118e581143ffb079e2b1aea3870d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:55.714533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c2f5f-4b41-45a6-9e63-731306c3251d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:55.731953 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 649/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Sep 27 09:43:55 2022] PUT /identity/v3/projects/cd30607a8a404dd2a608b2ec915dc81d/users/19f8118e581143ffb079e2b1aea3870d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:43:58.342203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf1897-63f6-4dda-9996-dfb7a655788d None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:58.342835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf1897-63f6-4dda-9996-dfb7a655788d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:58.342835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf1897-63f6-4dda-9996-dfb7a655788d None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:58.344107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 625/1274] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:43:58.350369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174f9b-d600-4e05-b2ae-6d4cb1dce474 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:58.350952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174f9b-d600-4e05-b2ae-6d4cb1dce474 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:58.350952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174f9b-d600-4e05-b2ae-6d4cb1dce474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:58.371704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0d174f9b-d600-4e05-b2ae-6d4cb1dce474 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:43:58.372976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0d174f9b-d600-4e05-b2ae-6d4cb1dce474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:43:58.448034 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 650/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:43:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:43:58.771821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf1897-63f6-4dda-9996-dfb7a655788d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:58.880648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf1897-63f6-4dda-9996-dfb7a655788d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:58.899321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:58.919947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:58.920586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:58.920909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:58.921181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:58.932777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 09:43:58.937617 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:58.938715 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:58.939577 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 626/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Sep 27 09:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:58.945625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174f9b-d600-4e05-b2ae-6d4cb1dce474 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:59.039923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:59.058993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174f9b-d600-4e05-b2ae-6d4cb1dce474 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:59.096176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:59.096814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:59.097496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:59.097765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:59.098424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] RBAC: Adding query filter params (name=admin) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:59.098722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:59.100092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:59.110777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 651/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Sep 27 09:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:59.119788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:59.249675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfb62b-fc31-4f69-99b3-ea7dc749d0bb None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:59.287496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:59.288358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:59.289021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:59.289300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:59.301008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:59.302588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:43:59.318370 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:59.319640 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:43:59.320851 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:43:59.320851 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:43:59.320851 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:59.320851 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 627/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:59.325546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:59.438378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95926692-22dc-4653-8e03-e6391907d515 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:59.478852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:59.479523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:59.479840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:59.480155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:59.480918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:59.481260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:59.482818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:59.496646 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 652/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:59.502957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:59.541014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:59.621890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0e28-d25e-4ffc-acd2-7ae9a5465967 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:59.659041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:59.659673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:59.660106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:59.660266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:59.669553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:59.671000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 09:43:59.683541 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:43:59.684653 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:43:59.685838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 628/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:43:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:43:59.691059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:59.695279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:59.797149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9068-bc90-4f0e-9f3e-47228dc3d73a None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:43:59.832592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:43:59.833236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:43:59.833513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:43:59.833784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:43:59.834515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:43:59.834814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:43:59.836249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:43:59.853926 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 653/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Sep 27 09:43:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:43:59.860045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:43:59.957185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:43:59.967707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8c738-fb9b-47df-9698-7821f7b43a77 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:00.002136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:00.002742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:00.003016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:00.003460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:00.074417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:00.075784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:00.170573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:00.181908 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 629/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Sep 27 09:43:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 09:44:00.188020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:00.268582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:00.309031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525bbf0-a0a6-4c36-a65d-ac7449c4b4e9 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:00.343942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:00.344489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:00.344790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:00.345055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:00.355866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:00.357354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 09:44:00.371849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:44:00.372999 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 09:44:00.373921 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 09:44:00.373921 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 09:44:00.373921 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:44:00.373921 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 654/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:44:00.379211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:00.480254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade97d2-a1f7-4f25-86dd-26533eb20134 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:00.517972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:00.518625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:00.518853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:00.519712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:00.520365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:00.520703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:00.522059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:00.535454 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 630/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Sep 27 09:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:44:00.542110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:00.648575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d933b-da3b-44b3-812e-3962743b1409 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:00.662522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:00.685001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:00.685631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:00.685913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:00.686185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] PATH_INFO: `/v3/projects/service` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:00.693534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:00.695033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 09:44:00.707395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:44:00.708478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 09:44:00.709717 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 655/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Sep 27 09:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:44:00.714686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:00.771107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:00.829229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4c1ac-0ad6-465c-9564-20364f744cf5 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:00.865506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1f0b2-e703-462f-85c5-beee10f5dca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:00.866072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1f0b2-e703-462f-85c5-beee10f5dca0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:00.866348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1f0b2-e703-462f-85c5-beee10f5dca0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:00.866614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1f0b2-e703-462f-85c5-beee10f5dca0 None admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:00.867315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1f0b2-e703-462f-85c5-beee10f5dca0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:00.867615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1f0b2-e703-462f-85c5-beee10f5dca0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:00.869091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1f0b2-e703-462f-85c5-beee10f5dca0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:00.882104 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 631/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Sep 27 09:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:44:00.888434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:00.991483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f28dc-621c-42e3-b609-af8805cf7765 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:01.026551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2f12-94e2-4f03-a518-435464d724d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:01.027152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2f12-94e2-4f03-a518-435464d724d6 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:01.027432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2f12-94e2-4f03-a518-435464d724d6 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:01.027702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2f12-94e2-4f03-a518-435464d724d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:01.028417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2f12-94e2-4f03-a518-435464d724d6 None admin] RBAC: Adding query filter params (role.id=53aff494b6b44d5da69d62199ff0d237, scope.project.id=cd30607a8a404dd2a608b2ec915dc81d, user.id=19f8118e581143ffb079e2b1aea3870d) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:01.028749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2f12-94e2-4f03-a518-435464d724d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:01.030191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2f12-94e2-4f03-a518-435464d724d6 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:01.036114 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 656/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Sep 27 09:44:00 2022] GET /identity/v3/role_assignments?user.id=19f8118e581143ffb079e2b1aea3870d&scope.project.id=cd30607a8a404dd2a608b2ec915dc81d&role.id=53aff494b6b44d5da69d62199ff0d237 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:44:03.481963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fec72-1e83-441d-b82d-6c78d57aa813 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:03.482646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fec72-1e83-441d-b82d-6c78d57aa813 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:03.482646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fec72-1e83-441d-b82d-6c78d57aa813 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:03.483807 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 632/1288] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:44:03.490058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9663b50-6b03-400a-aefd-c4a427f49826 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:03.490600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9663b50-6b03-400a-aefd-c4a427f49826 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:03.490600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9663b50-6b03-400a-aefd-c4a427f49826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:03.510916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b9663b50-6b03-400a-aefd-c4a427f49826 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:44:03.512124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b9663b50-6b03-400a-aefd-c4a427f49826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:44:03.545987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:03.586080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 657/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:44:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:44:03.908689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fec72-1e83-441d-b82d-6c78d57aa813 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:04.017242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fec72-1e83-441d-b82d-6c78d57aa813 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:04.029933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:04.058877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:04.059536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:04.059816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:04.060085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] PATH_INFO: `/v3/services/placement` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:04.060705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:04.062153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:44:04.071929 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:44:04.073342 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:04.074326 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 633/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:44:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:44:04.079373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9663b50-6b03-400a-aefd-c4a427f49826 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:04.204364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9663b50-6b03-400a-aefd-c4a427f49826 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:04.242809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30d9d7-61e2-482c-9e5f-f49c641e5ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:04.243249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d9d7-61e2-482c-9e5f-f49c641e5ee3 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:04.243423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d9d7-61e2-482c-9e5f-f49c641e5ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:04.243776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d9d7-61e2-482c-9e5f-f49c641e5ee3 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:04.244311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30d9d7-61e2-482c-9e5f-f49c641e5ee3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:04.244610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30d9d7-61e2-482c-9e5f-f49c641e5ee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:04.246018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30d9d7-61e2-482c-9e5f-f49c641e5ee3 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:04.251439 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 658/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 27 09:44:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:44:04.257802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:04.369267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349ccb0-2a44-4555-9a51-a3ea60b7035c None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:04.406303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89f266-5a06-4692-bdcb-beb45026a577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:04.406918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89f266-5a06-4692-bdcb-beb45026a577 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:04.407201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89f266-5a06-4692-bdcb-beb45026a577 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:04.407472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89f266-5a06-4692-bdcb-beb45026a577 None admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:04.408106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89f266-5a06-4692-bdcb-beb45026a577 None admin] RBAC: Adding query filter params (type=placement) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:04.408410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89f266-5a06-4692-bdcb-beb45026a577 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:04.409867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89f266-5a06-4692-bdcb-beb45026a577 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:04.415431 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 634/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 27 09:44:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:44:06.857961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162b107-5e1e-41e5-9d28-ae1cf412f5a0 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:06.858516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162b107-5e1e-41e5-9d28-ae1cf412f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:06.858516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162b107-5e1e-41e5-9d28-ae1cf412f5a0 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:06.859740 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 659/1293] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:44:06.865906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2801f8b-3722-4153-9641-c1813fbe181c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:06.866404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2801f8b-3722-4153-9641-c1813fbe181c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:06.866509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2801f8b-3722-4153-9641-c1813fbe181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:06.886217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d2801f8b-3722-4153-9641-c1813fbe181c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:44:06.887416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d2801f8b-3722-4153-9641-c1813fbe181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:44:06.920346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:06.959947 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 635/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:44:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:44:07.280206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162b107-5e1e-41e5-9d28-ae1cf412f5a0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:07.317935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:07.395223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162b107-5e1e-41e5-9d28-ae1cf412f5a0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:07.431239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c7405-c7f8-4ed2-ba42-0fb77abdf21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:07.431843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c7405-c7f8-4ed2-ba42-0fb77abdf21d None admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:07.432128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c7405-c7f8-4ed2-ba42-0fb77abdf21d None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:07.432393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c7405-c7f8-4ed2-ba42-0fb77abdf21d None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:07.433076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c7405-c7f8-4ed2-ba42-0fb77abdf21d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:07.434446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c7405-c7f8-4ed2-ba42-0fb77abdf21d None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:07.450612 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 660/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Sep 27 09:44:07 2022] POST /identity/v3/services => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:44:09.920639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b7a0-228e-4549-83e7-406d231f2e3e None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:09.921270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b7a0-228e-4549-83e7-406d231f2e3e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:09.921270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b7a0-228e-4549-83e7-406d231f2e3e None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:09.922479 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 636/1296] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:44:09.928871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d3e7e-924b-4b30-b271-e15c19200246 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:09.929491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d3e7e-924b-4b30-b271-e15c19200246 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:09.929491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d3e7e-924b-4b30-b271-e15c19200246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:09.949696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-353d3e7e-924b-4b30-b271-e15c19200246 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:44:09.950944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-353d3e7e-924b-4b30-b271-e15c19200246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:44:10.034055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 661/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:44:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 09:44:10.356425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b7a0-228e-4549-83e7-406d231f2e3e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:10.468123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b7a0-228e-4549-83e7-406d231f2e3e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:10.506981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:10.507625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:10.507898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:10.508188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] PATH_INFO: `/v3/services/placement` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:10.508811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:10.510190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:44:10.519535 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:44:10.520742 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:10.521802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 637/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:44:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:44:10.527171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d3e7e-924b-4b30-b271-e15c19200246 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:10.548105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:10.632046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d3e7e-924b-4b30-b271-e15c19200246 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:10.669694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:10.670300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:10.670570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:10.670835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:10.671461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:10.671758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:10.673157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:10.678458 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 662/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 27 09:44:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:44:10.685018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:10.718943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:10.793430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4377d11-f773-4a4d-83df-8e3d31e99662 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:10.829240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dad70-5abc-48b7-92b7-fb51d3ac61e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:10.829860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dad70-5abc-48b7-92b7-fb51d3ac61e0 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:10.830138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dad70-5abc-48b7-92b7-fb51d3ac61e0 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:10.830408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dad70-5abc-48b7-92b7-fb51d3ac61e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:10.831118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dad70-5abc-48b7-92b7-fb51d3ac61e0 None admin] RBAC: Adding query filter params (interface=public, service_id=349ef0fca40d430dbf74b51550c8eefa, region_id=RegionOne) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:10.831466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dad70-5abc-48b7-92b7-fb51d3ac61e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:10.832887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dad70-5abc-48b7-92b7-fb51d3ac61e0 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:10.838280 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 638/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Sep 27 09:44:10 2022] GET /identity/v3/endpoints?service_id=349ef0fca40d430dbf74b51550c8eefa&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:44:10.844599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:10.935416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:10.954131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd68e3-1235-4b38-b1e8-061052a4f7c0 None admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:10.989861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d28a05-463b-4188-ae6d-2bfbe92fa2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:10.990479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28a05-463b-4188-ae6d-2bfbe92fa2fe None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:10.990756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28a05-463b-4188-ae6d-2bfbe92fa2fe None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:10.991021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d28a05-463b-4188-ae6d-2bfbe92fa2fe None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:10.991610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d28a05-463b-4188-ae6d-2bfbe92fa2fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:10.993009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d28a05-463b-4188-ae6d-2bfbe92fa2fe None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:11.001018 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 663/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Sep 27 09:44:10 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:44:13.448182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d584563-269c-4a96-915c-7c914a9d08e3 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:13.448812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d584563-269c-4a96-915c-7c914a9d08e3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:13.448812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d584563-269c-4a96-915c-7c914a9d08e3 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:13.450071 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 639/1302] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:44:13.455768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3a6e7-6fea-4065-96dc-eee8943f4ffa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:13.456296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3a6e7-6fea-4065-96dc-eee8943f4ffa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:13.456823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3a6e7-6fea-4065-96dc-eee8943f4ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:13.475772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f7f3a6e7-6fea-4065-96dc-eee8943f4ffa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:44:13.477002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f7f3a6e7-6fea-4065-96dc-eee8943f4ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:44:13.548193 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 664/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:44:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 09:44:13.868931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d584563-269c-4a96-915c-7c914a9d08e3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:13.972076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:13.979884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d584563-269c-4a96-915c-7c914a9d08e3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:14.015720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:14.016362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:14.016641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:14.016955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] PATH_INFO: `/v3/services/placement` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:14.017539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:14.018958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 09:44:14.028026 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 09:44:14.029250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 09:44:14.030250 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 640/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Sep 27 09:44:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 09:44:14.035506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3a6e7-6fea-4065-96dc-eee8943f4ffa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:14.117738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:14.144514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3a6e7-6fea-4065-96dc-eee8943f4ffa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:14.184567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30e3bf-acb7-4bce-b5e0-800ab1baede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:14.185229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e3bf-acb7-4bce-b5e0-800ab1baede9 None admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:14.185505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e3bf-acb7-4bce-b5e0-800ab1baede9 None admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:14.185773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30e3bf-acb7-4bce-b5e0-800ab1baede9 None admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:14.186439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30e3bf-acb7-4bce-b5e0-800ab1baede9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:44:14.186746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30e3bf-acb7-4bce-b5e0-800ab1baede9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:14.188127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30e3bf-acb7-4bce-b5e0-800ab1baede9 None admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:14.194251 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 665/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Sep 27 09:44:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:44:14.200290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:44:14.314016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef739c90-0016-43f0-b21a-e74c3a0abe84 None admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:44:14.351573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4713d0-4d6f-4878-963d-82e96f633c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:44:14.352187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4713d0-4d6f-4878-963d-82e96f633c8f None admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:44:14.352469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4713d0-4d6f-4878-963d-82e96f633c8f None admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:44:14.352763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4713d0-4d6f-4878-963d-82e96f633c8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:44:14.353420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4713d0-4d6f-4878-963d-82e96f633c8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:44:14.354814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4713d0-4d6f-4878-963d-82e96f633c8f None admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:44:14.378904 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 641/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:44:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:44:24.191317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=30 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:44:24.373974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:33.220877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdd03d-67e4-4a0e-918e-bcd6c8c2de45 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:33.221432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdd03d-67e4-4a0e-918e-bcd6c8c2de45 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:33.221515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdd03d-67e4-4a0e-918e-bcd6c8c2de45 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:33.227040 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 666/1307] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:45:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:33.230837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7eb5a-c141-4e56-924e-c3e48dbbd9fc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:33.230837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7eb5a-c141-4e56-924e-c3e48dbbd9fc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:33.230837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7eb5a-c141-4e56-924e-c3e48dbbd9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:33.324684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-eed7eb5a-c141-4e56-924e-c3e48dbbd9fc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:33.326035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eed7eb5a-c141-4e56-924e-c3e48dbbd9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:33.448687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 642/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:45:33.802727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b283fd-6523-4f74-b46e-550612d85554 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:33.803037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b283fd-6523-4f74-b46e-550612d85554 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:33.803742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b283fd-6523-4f74-b46e-550612d85554 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:33.805991 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 667/1309] 10.222.0.26 () {54 vars in 964 bytes} [Tue Sep 27 09:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:33.812889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95611773-8c8e-41bf-8cc1-f38949e98c98 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:33.813241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95611773-8c8e-41bf-8cc1-f38949e98c98 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:33.813416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95611773-8c8e-41bf-8cc1-f38949e98c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:33.912849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-95611773-8c8e-41bf-8cc1-f38949e98c98 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:33.914060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-95611773-8c8e-41bf-8cc1-f38949e98c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:33.941959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:34.021917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 643/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Sep 27 09:45:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:45:34.031247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b283fd-6523-4f74-b46e-550612d85554 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:45:34.033067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44b283fd-6523-4f74-b46e-550612d85554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:34.187574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b283fd-6523-4f74-b46e-550612d85554 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:45:34.281987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a989-f485-4830-b155-50705911bc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:45:34.282703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a989-f485-4830-b155-50705911bc12 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:34.282826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a989-f485-4830-b155-50705911bc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:34.283194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a989-f485-4830-b155-50705911bc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:34.285441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-70f9a989-f485-4830-b155-50705911bc12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:34.356119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:34.429495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a989-f485-4830-b155-50705911bc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:45:34.430995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a989-f485-4830-b155-50705911bc12 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:45:34.454124 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 668/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:45:37.165804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498dc42-51fa-4fec-93ae-a11b382d97b1 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:37.166411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498dc42-51fa-4fec-93ae-a11b382d97b1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:37.166788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498dc42-51fa-4fec-93ae-a11b382d97b1 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:37.168243 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 644/1312] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:37.174275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95837b-bd4a-4fd4-a5f7-a710d5800cee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:37.174970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95837b-bd4a-4fd4-a5f7-a710d5800cee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:37.174970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95837b-bd4a-4fd4-a5f7-a710d5800cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:37.265086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4f95837b-bd4a-4fd4-a5f7-a710d5800cee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:37.266259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4f95837b-bd4a-4fd4-a5f7-a710d5800cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:37.388696 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 669/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:45:37.744296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29297a67-6ead-4799-a32f-978d76205865 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:37.744681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29297a67-6ead-4799-a32f-978d76205865 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:37.744863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29297a67-6ead-4799-a32f-978d76205865 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:37.746372 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 645/1314] 10.222.0.26 () {54 vars in 964 bytes} [Tue Sep 27 09:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:37.752463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb6a84-c930-4df8-ae2c-0d96447a6012 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:37.753024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb6a84-c930-4df8-ae2c-0d96447a6012 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:37.753698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb6a84-c930-4df8-ae2c-0d96447a6012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:37.766956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:37.846961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-83cb6a84-c930-4df8-ae2c-0d96447a6012 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:37.848260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-83cb6a84-c930-4df8-ae2c-0d96447a6012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:37.944021 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 670/1315] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Sep 27 09:45:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:45:37.953575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29297a67-6ead-4799-a32f-978d76205865 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:45:37.967083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-29297a67-6ead-4799-a32f-978d76205865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:38.109231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29297a67-6ead-4799-a32f-978d76205865 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:45:38.206679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:38.210064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32046b4-437e-4c93-a77e-a48f52bc8168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:45:38.210810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32046b4-437e-4c93-a77e-a48f52bc8168 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:38.211226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32046b4-437e-4c93-a77e-a48f52bc8168 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:38.211509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32046b4-437e-4c93-a77e-a48f52bc8168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:38.214450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b32046b4-437e-4c93-a77e-a48f52bc8168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:38.362665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32046b4-437e-4c93-a77e-a48f52bc8168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:45:38.364153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32046b4-437e-4c93-a77e-a48f52bc8168 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:45:38.386838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 646/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:45:41.381172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b71c1-413c-4c90-9a21-de5a639bcd1b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:41.381801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b71c1-413c-4c90-9a21-de5a639bcd1b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:41.381801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b71c1-413c-4c90-9a21-de5a639bcd1b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:41.383159 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 671/1317] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:41.389430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55312398-c24a-4b0f-8ac0-87636d84cb03 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:41.390061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55312398-c24a-4b0f-8ac0-87636d84cb03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:41.390061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55312398-c24a-4b0f-8ac0-87636d84cb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:41.467047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-55312398-c24a-4b0f-8ac0-87636d84cb03 None None] MFA Rules not processed for user `cc2b0cd654dd4bf8a616c60ea6b268d3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:41.468280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55312398-c24a-4b0f-8ac0-87636d84cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:41.528424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:41.582609 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 647/1318] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:45:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:45:41.929540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b71c1-413c-4c90-9a21-de5a639bcd1b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:45:41.931204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-015b71c1-413c-4c90-9a21-de5a639bcd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:42.043485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b71c1-413c-4c90-9a21-de5a639bcd1b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:45:42.046302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:42.125116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebf916-16ce-457c-80d7-9e909e654da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:45:42.125731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebf916-16ce-457c-80d7-9e909e654da8 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:42.126004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebf916-16ce-457c-80d7-9e909e654da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:42.126318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebf916-16ce-457c-80d7-9e909e654da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:42.128332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-33ebf916-16ce-457c-80d7-9e909e654da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:42.243644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebf916-16ce-457c-80d7-9e909e654da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:45:42.245008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebf916-16ce-457c-80d7-9e909e654da8 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:45:42.268486 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 672/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:45:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:45:45.247296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43db2a-7f6f-416b-9c48-1eb94852f8ac None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:45.248132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43db2a-7f6f-416b-9c48-1eb94852f8ac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:45.248132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43db2a-7f6f-416b-9c48-1eb94852f8ac None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:45.250043 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 648/1320] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:45:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:45.256528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feeebb-41ca-4eae-941a-5591bfd5840c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:45.257022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feeebb-41ca-4eae-941a-5591bfd5840c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:45.257140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feeebb-41ca-4eae-941a-5591bfd5840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:45.299691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:45.327962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-79feeebb-41ca-4eae-941a-5591bfd5840c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:45.329229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79feeebb-41ca-4eae-941a-5591bfd5840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:45.434050 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 673/1321] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:45:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:45:45.770419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43db2a-7f6f-416b-9c48-1eb94852f8ac None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:45:45.772455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f43db2a-7f6f-416b-9c48-1eb94852f8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:45.888959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43db2a-7f6f-416b-9c48-1eb94852f8ac None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:45:45.975360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37987713-b919-435d-8ca6-a65658641e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:45:45.975979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37987713-b919-435d-8ca6-a65658641e56 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:45.976249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37987713-b919-435d-8ca6-a65658641e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:45.976513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37987713-b919-435d-8ca6-a65658641e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:45.978171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-37987713-b919-435d-8ca6-a65658641e56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:45.985477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:46.097250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37987713-b919-435d-8ca6-a65658641e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:45:46.098696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37987713-b919-435d-8ca6-a65658641e56 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:45:46.119082 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 649/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:45:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:45:49.472167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890cedc-c0c5-439c-81f0-8bcace5cb053 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:49.472768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890cedc-c0c5-439c-81f0-8bcace5cb053 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:49.472895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890cedc-c0c5-439c-81f0-8bcace5cb053 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:49.474317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 674/1323] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:49.480581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27d8e-6086-447d-9836-acbdef05a2e0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:49.481268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27d8e-6086-447d-9836-acbdef05a2e0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:49.481268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27d8e-6086-447d-9836-acbdef05a2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:49.558446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3da27d8e-6086-447d-9836-acbdef05a2e0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:49.559914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3da27d8e-6086-447d-9836-acbdef05a2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:49.633078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:49.675230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 650/1324] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:45:50.014190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890cedc-c0c5-439c-81f0-8bcace5cb053 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:45:50.015774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f890cedc-c0c5-439c-81f0-8bcace5cb053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:50.133570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890cedc-c0c5-439c-81f0-8bcace5cb053 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:45:50.150062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:50.215609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d359e39-f71d-418a-82b3-81ddbf317cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:45:50.216220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359e39-f71d-418a-82b3-81ddbf317cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:50.216557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359e39-f71d-418a-82b3-81ddbf317cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:50.216776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359e39-f71d-418a-82b3-81ddbf317cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:50.218423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6d359e39-f71d-418a-82b3-81ddbf317cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:50.335581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d359e39-f71d-418a-82b3-81ddbf317cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:45:50.337069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d359e39-f71d-418a-82b3-81ddbf317cf3 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:45:50.356338 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 675/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:45:52.905267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975a3bf-d934-4b18-984f-7f43de71e575 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:52.905820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975a3bf-d934-4b18-984f-7f43de71e575 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:52.905820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975a3bf-d934-4b18-984f-7f43de71e575 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:52.907248 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 651/1326] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:52.913087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951b834-6116-4958-a816-1c1c3fdbf45f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:52.913087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951b834-6116-4958-a816-1c1c3fdbf45f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:52.913731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951b834-6116-4958-a816-1c1c3fdbf45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:52.964679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:52.984113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2951b834-6116-4958-a816-1c1c3fdbf45f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:52.985365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2951b834-6116-4958-a816-1c1c3fdbf45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:53.085909 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 676/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:45:53.430443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975a3bf-d934-4b18-984f-7f43de71e575 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:45:53.432001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1975a3bf-d934-4b18-984f-7f43de71e575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:53.549324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975a3bf-d934-4b18-984f-7f43de71e575 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:45:53.638095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0cabf-7bfa-4820-b1ad-9ce0f78b8392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:45:53.638708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0cabf-7bfa-4820-b1ad-9ce0f78b8392 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:53.638981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0cabf-7bfa-4820-b1ad-9ce0f78b8392 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:53.639247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0cabf-7bfa-4820-b1ad-9ce0f78b8392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:53.640880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-13e0cabf-7bfa-4820-b1ad-9ce0f78b8392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:53.661001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:53.790536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0cabf-7bfa-4820-b1ad-9ce0f78b8392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:45:53.792078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0cabf-7bfa-4820-b1ad-9ce0f78b8392 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:45:53.817152 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 652/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:45:56.350164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d375-1446-4853-8376-383f36b23f54 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:56.350745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d375-1446-4853-8376-383f36b23f54 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:56.350745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d375-1446-4853-8376-383f36b23f54 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:56.352217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 677/1329] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:45:56.358176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894adf52-9f51-4cd0-8688-c613d76cc3d9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:56.358651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894adf52-9f51-4cd0-8688-c613d76cc3d9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:56.358761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894adf52-9f51-4cd0-8688-c613d76cc3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:56.427494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-894adf52-9f51-4cd0-8688-c613d76cc3d9 None None] MFA Rules not processed for user `cc2b0cd654dd4bf8a616c60ea6b268d3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:45:56.428724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-894adf52-9f51-4cd0-8688-c613d76cc3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:56.496278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:56.529312 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 653/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:45:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:45:56.866373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61d375-1446-4853-8376-383f36b23f54 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:45:56.867940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1d61d375-1446-4853-8376-383f36b23f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:56.993759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61d375-1446-4853-8376-383f36b23f54 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:45:57.019538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:45:57.082581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29d075-f9d4-44c8-8893-5c9fdea9030a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:45:57.083203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d075-f9d4-44c8-8893-5c9fdea9030a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:45:57.083475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d075-f9d4-44c8-8893-5c9fdea9030a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:45:57.083743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d075-f9d4-44c8-8893-5c9fdea9030a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:45:57.085463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dc29d075-f9d4-44c8-8893-5c9fdea9030a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:45:57.204064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29d075-f9d4-44c8-8893-5c9fdea9030a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:45:57.205519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29d075-f9d4-44c8-8893-5c9fdea9030a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:45:57.223710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 678/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:45:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:00.039182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff260-2b1d-4070-b454-70d3bd52ab0b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:00.039769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff260-2b1d-4070-b454-70d3bd52ab0b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:00.039769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff260-2b1d-4070-b454-70d3bd52ab0b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:00.041491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 654/1332] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:00.047404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eaf28-e003-4055-9f8e-ccfc46e7cdd7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:00.047977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eaf28-e003-4055-9f8e-ccfc46e7cdd7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:00.047977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eaf28-e003-4055-9f8e-ccfc46e7cdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:00.124597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:00.128685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3f3eaf28-e003-4055-9f8e-ccfc46e7cdd7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:00.130712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f3eaf28-e003-4055-9f8e-ccfc46e7cdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:00.247876 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 679/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:00.585999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff260-2b1d-4070-b454-70d3bd52ab0b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:00.587546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e5eff260-2b1d-4070-b454-70d3bd52ab0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:00.715469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff260-2b1d-4070-b454-70d3bd52ab0b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:00.804902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f451ad-8816-4601-b7b6-6a6fd8efec9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:00.805531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f451ad-8816-4601-b7b6-6a6fd8efec9d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:00.805803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f451ad-8816-4601-b7b6-6a6fd8efec9d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:00.806061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f451ad-8816-4601-b7b6-6a6fd8efec9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:00.807716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66f451ad-8816-4601-b7b6-6a6fd8efec9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:00.838533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:00.940358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f451ad-8816-4601-b7b6-6a6fd8efec9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:00.941988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f451ad-8816-4601-b7b6-6a6fd8efec9d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:00.964733 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 655/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:03.547650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0721b2-a776-4c9f-a868-864b3a0deb44 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:03.548251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0721b2-a776-4c9f-a868-864b3a0deb44 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:03.548251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0721b2-a776-4c9f-a868-864b3a0deb44 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:03.550456 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 680/1335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:03.556324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb220d-3815-4f60-bd73-1ac1109f5542 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:03.556888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb220d-3815-4f60-bd73-1ac1109f5542 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:03.556888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb220d-3815-4f60-bd73-1ac1109f5542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:03.632608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2abb220d-3815-4f60-bd73-1ac1109f5542 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:03.635056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2abb220d-3815-4f60-bd73-1ac1109f5542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:03.719147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:03.744027 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 656/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:46:04.091291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0721b2-a776-4c9f-a868-864b3a0deb44 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:04.092896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f0721b2-a776-4c9f-a868-864b3a0deb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:04.204978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0721b2-a776-4c9f-a868-864b3a0deb44 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:04.237099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:04.290669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc4d5f-655f-4df3-b858-dcac63948ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:04.291288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc4d5f-655f-4df3-b858-dcac63948ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:04.291568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc4d5f-655f-4df3-b858-dcac63948ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:04.291839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc4d5f-655f-4df3-b858-dcac63948ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:04.293490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bbcc4d5f-655f-4df3-b858-dcac63948ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:04.408142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc4d5f-655f-4df3-b858-dcac63948ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:04.409731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc4d5f-655f-4df3-b858-dcac63948ca8 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:04.429325 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 681/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:07.298187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb7f82-5eaa-4c7e-986e-991476e738f8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:07.298844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb7f82-5eaa-4c7e-986e-991476e738f8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:07.298844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb7f82-5eaa-4c7e-986e-991476e738f8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:07.300248 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 657/1338] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:07.305925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ab2f-689a-4467-a9f4-060fb0db0eb8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:07.306459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ab2f-689a-4467-a9f4-060fb0db0eb8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:07.306459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ab2f-689a-4467-a9f4-060fb0db0eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:07.377061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c213ab2f-689a-4467-a9f4-060fb0db0eb8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:07.378889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c213ab2f-689a-4467-a9f4-060fb0db0eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:07.387850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:07.485147 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 682/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:07.816688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb7f82-5eaa-4c7e-986e-991476e738f8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:07.818259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aabb7f82-5eaa-4c7e-986e-991476e738f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:07.925128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb7f82-5eaa-4c7e-986e-991476e738f8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:08.003983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb6204-c0ed-44d6-853d-f42f10ea4be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:08.004625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb6204-c0ed-44d6-853d-f42f10ea4be8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:08.004929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb6204-c0ed-44d6-853d-f42f10ea4be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:08.005240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb6204-c0ed-44d6-853d-f42f10ea4be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:08.006793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5ddb6204-c0ed-44d6-853d-f42f10ea4be8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:08.048257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:08.134268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb6204-c0ed-44d6-853d-f42f10ea4be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:08.135846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb6204-c0ed-44d6-853d-f42f10ea4be8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:08.164138 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 658/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:46:12.620072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639de7c5-2c11-4a68-a17b-221bd0048ae4 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:12.620603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639de7c5-2c11-4a68-a17b-221bd0048ae4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:12.620721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639de7c5-2c11-4a68-a17b-221bd0048ae4 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:12.622132 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 683/1341] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:12.628302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb672da-669d-4f85-b923-4d05fd055720 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:12.628821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb672da-669d-4f85-b923-4d05fd055720 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:12.629054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb672da-669d-4f85-b923-4d05fd055720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:12.700345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4fb672da-669d-4f85-b923-4d05fd055720 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:12.701612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4fb672da-669d-4f85-b923-4d05fd055720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:12.797251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:12.809064 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 659/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:46:13.147752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639de7c5-2c11-4a68-a17b-221bd0048ae4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:13.149604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-639de7c5-2c11-4a68-a17b-221bd0048ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:13.265186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639de7c5-2c11-4a68-a17b-221bd0048ae4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:13.307176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:13.346106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f2768-6e39-45fd-b2e7-2fd0e1a8018c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:13.346762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f2768-6e39-45fd-b2e7-2fd0e1a8018c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:13.347036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f2768-6e39-45fd-b2e7-2fd0e1a8018c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:13.347300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f2768-6e39-45fd-b2e7-2fd0e1a8018c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:13.348921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-db7f2768-6e39-45fd-b2e7-2fd0e1a8018c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:13.467187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f2768-6e39-45fd-b2e7-2fd0e1a8018c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:13.468709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f2768-6e39-45fd-b2e7-2fd0e1a8018c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:13.489552 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 684/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:16.622302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f75bb-15b7-43b4-bab3-03386213a781 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:16.622936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f75bb-15b7-43b4-bab3-03386213a781 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:16.622936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f75bb-15b7-43b4-bab3-03386213a781 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:16.624138 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 660/1344] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:16.630002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ee134-c199-4850-83c4-33bf6d70348a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:16.630516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ee134-c199-4850-83c4-33bf6d70348a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:16.630516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ee134-c199-4850-83c4-33bf6d70348a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:16.709912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-525ee134-c199-4850-83c4-33bf6d70348a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:16.711162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-525ee134-c199-4850-83c4-33bf6d70348a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:16.730858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:16.818272 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 685/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:17.164357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f75bb-15b7-43b4-bab3-03386213a781 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:17.166725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af2f75bb-15b7-43b4-bab3-03386213a781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:17.285962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f75bb-15b7-43b4-bab3-03386213a781 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:17.368068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40704a-e154-4a3d-8829-3ddc3be92af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:17.368684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40704a-e154-4a3d-8829-3ddc3be92af1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:17.368991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40704a-e154-4a3d-8829-3ddc3be92af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:17.369261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40704a-e154-4a3d-8829-3ddc3be92af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:17.370881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6c40704a-e154-4a3d-8829-3ddc3be92af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:17.421259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:17.493840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40704a-e154-4a3d-8829-3ddc3be92af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:17.495296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40704a-e154-4a3d-8829-3ddc3be92af1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:17.514555 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 661/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:22.375927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e13f-53f4-494d-b341-7f9904e7b35a None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:22.376457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e13f-53f4-494d-b341-7f9904e7b35a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:22.376457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e13f-53f4-494d-b341-7f9904e7b35a None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:22.379514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 686/1347] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:22.385229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32a117-fec5-41f3-b068-985f3b0beb66 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:22.385748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32a117-fec5-41f3-b068-985f3b0beb66 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:22.385748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32a117-fec5-41f3-b068-985f3b0beb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:22.466332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4f32a117-fec5-41f3-b068-985f3b0beb66 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:22.467472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4f32a117-fec5-41f3-b068-985f3b0beb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:22.584297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:22.586814 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 662/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:46:22.925919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e13f-53f4-494d-b341-7f9904e7b35a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:22.927481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-59f2e13f-53f4-494d-b341-7f9904e7b35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:23.042713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e13f-53f4-494d-b341-7f9904e7b35a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:23.103984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:23.136236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c6adf-f634-4083-81f4-46fefaff23fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:23.136907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6adf-f634-4083-81f4-46fefaff23fc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:23.137228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6adf-f634-4083-81f4-46fefaff23fc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:23.137453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6adf-f634-4083-81f4-46fefaff23fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:23.139240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a47c6adf-f634-4083-81f4-46fefaff23fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:23.273089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c6adf-f634-4083-81f4-46fefaff23fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:23.274545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c6adf-f634-4083-81f4-46fefaff23fc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:23.294875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 687/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:28.992773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aabece-9e4f-4e3e-80a9-44f9412e8cd7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:28.992773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aabece-9e4f-4e3e-80a9-44f9412e8cd7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:28.992773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aabece-9e4f-4e3e-80a9-44f9412e8cd7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:28.992773 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 663/1350] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:29.002579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8003b-d4c3-4499-912b-55d26137231d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:29.003176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8003b-d4c3-4499-912b-55d26137231d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:29.003288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8003b-d4c3-4499-912b-55d26137231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:29.106371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-19d8003b-d4c3-4499-912b-55d26137231d None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:29.108492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-19d8003b-d4c3-4499-912b-55d26137231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:29.132774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:29.275309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 688/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:46:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:33.848100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca834dd1-a816-4c69-b150-1305a32e6107 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:33.849117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca834dd1-a816-4c69-b150-1305a32e6107 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:33.849403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca834dd1-a816-4c69-b150-1305a32e6107 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:33.850791 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 664/1352] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:46:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:33.859467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeba936-8f4b-463a-bad4-53c265315210 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:33.859467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeba936-8f4b-463a-bad4-53c265315210 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:33.859467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeba936-8f4b-463a-bad4-53c265315210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:33.952110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-abeba936-8f4b-463a-bad4-53c265315210 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:33.953328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-abeba936-8f4b-463a-bad4-53c265315210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:34.034745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:34.076620 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 689/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:46:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:46:35.893680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5de1b-fecc-4946-b2fc-5238ff4c25f9 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:35.894422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5de1b-fecc-4946-b2fc-5238ff4c25f9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:35.894422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5de1b-fecc-4946-b2fc-5238ff4c25f9 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:35.895479 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 665/1354] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:35.901137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6a245-1e3d-44d4-8874-47d6535a72be None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:35.901666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6a245-1e3d-44d4-8874-47d6535a72be None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:35.901789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6a245-1e3d-44d4-8874-47d6535a72be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:35.994650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-54c6a245-1e3d-44d4-8874-47d6535a72be None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:35.995973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-54c6a245-1e3d-44d4-8874-47d6535a72be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:36.114251 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 690/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:46:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:39.724247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54ce9d-878f-4818-8e8d-da80786ad26f None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:39.724980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54ce9d-878f-4818-8e8d-da80786ad26f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:39.724980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54ce9d-878f-4818-8e8d-da80786ad26f None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:39.726470 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 666/1356] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:39.732281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abc36b-13aa-4fca-8374-c6a7f535864d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:39.733143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abc36b-13aa-4fca-8374-c6a7f535864d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:39.733578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abc36b-13aa-4fca-8374-c6a7f535864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:39.783894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:39.820712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-66abc36b-13aa-4fca-8374-c6a7f535864d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:39.822130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-66abc36b-13aa-4fca-8374-c6a7f535864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:39.932991 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 691/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:40.298794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921c11-512c-462b-bfcd-82d4bb143b9b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:40.299073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921c11-512c-462b-bfcd-82d4bb143b9b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:40.299833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921c11-512c-462b-bfcd-82d4bb143b9b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:40.302135 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 667/1358] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 27 09:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:40.309887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74b18-e9df-4533-aac2-dbb946e8f142 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:40.310207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74b18-e9df-4533-aac2-dbb946e8f142 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:40.310578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74b18-e9df-4533-aac2-dbb946e8f142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:40.400298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-66c74b18-e9df-4533-aac2-dbb946e8f142 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:40.401849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-66c74b18-e9df-4533-aac2-dbb946e8f142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:40.461640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:40.539211 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 692/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 09:46:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:40.547634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921c11-512c-462b-bfcd-82d4bb143b9b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:40.549224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e0921c11-512c-462b-bfcd-82d4bb143b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:40.721817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921c11-512c-462b-bfcd-82d4bb143b9b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:40.809243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:40.855441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef30f0a-7537-47a4-9b94-a4c703dd38ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:40.856059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30f0a-7537-47a4-9b94-a4c703dd38ac service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:40.856334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30f0a-7537-47a4-9b94-a4c703dd38ac service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:40.857326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef30f0a-7537-47a4-9b94-a4c703dd38ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:40.859062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5ef30f0a-7537-47a4-9b94-a4c703dd38ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:40.968455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef30f0a-7537-47a4-9b94-a4c703dd38ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:40.970539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef30f0a-7537-47a4-9b94-a4c703dd38ac service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:40.994355 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 668/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:43.682400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3f16-0c90-47b5-a5b3-ea29b24be2e4 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:43.683266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3f16-0c90-47b5-a5b3-ea29b24be2e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:43.683683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3f16-0c90-47b5-a5b3-ea29b24be2e4 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:43.686045 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 693/1361] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:43.691410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb2f66-a1bf-4695-9eab-1ba088d55fa2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:43.691920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb2f66-a1bf-4695-9eab-1ba088d55fa2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:43.692531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb2f66-a1bf-4695-9eab-1ba088d55fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:43.767041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-08cb2f66-a1bf-4695-9eab-1ba088d55fa2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:43.768921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-08cb2f66-a1bf-4695-9eab-1ba088d55fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:43.813184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:43.899381 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 669/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:44.249893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e3f16-0c90-47b5-a5b3-ea29b24be2e4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:44.252630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ad1e3f16-0c90-47b5-a5b3-ea29b24be2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:44.414297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e3f16-0c90-47b5-a5b3-ea29b24be2e4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:44.424972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:44.542159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714238a1-4d4f-4ec7-8389-ba5334d563e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:44.542741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714238a1-4d4f-4ec7-8389-ba5334d563e6 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:44.543020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714238a1-4d4f-4ec7-8389-ba5334d563e6 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:44.543292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714238a1-4d4f-4ec7-8389-ba5334d563e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:44.545027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-714238a1-4d4f-4ec7-8389-ba5334d563e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:44.655352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714238a1-4d4f-4ec7-8389-ba5334d563e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:44.657049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714238a1-4d4f-4ec7-8389-ba5334d563e6 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:44.677876 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 694/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:47.494488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f043c-be28-475c-9051-4530aeeb5a1f None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:47.495000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f043c-be28-475c-9051-4530aeeb5a1f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:47.495208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f043c-be28-475c-9051-4530aeeb5a1f None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:47.496640 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 670/1364] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:47.502682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4cd39-b2ae-4486-8198-0e1d0f3d6dee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:47.503226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4cd39-b2ae-4486-8198-0e1d0f3d6dee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:47.503432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4cd39-b2ae-4486-8198-0e1d0f3d6dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:47.521379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:47.590047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-54e4cd39-b2ae-4486-8198-0e1d0f3d6dee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:47.591629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-54e4cd39-b2ae-4486-8198-0e1d0f3d6dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:47.714033 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 695/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:48.103545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457150f0-8faf-4d32-b59d-dd6e4b021dad None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:48.103974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457150f0-8faf-4d32-b59d-dd6e4b021dad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:48.104362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457150f0-8faf-4d32-b59d-dd6e4b021dad None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:48.106192 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 671/1366] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 27 09:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:48.117919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c7fdf-0f21-4d67-96c1-afedc300875e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:48.118429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c7fdf-0f21-4d67-96c1-afedc300875e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:48.118849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c7fdf-0f21-4d67-96c1-afedc300875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:48.230617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1d8c7fdf-0f21-4d67-96c1-afedc300875e None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:48.232163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1d8c7fdf-0f21-4d67-96c1-afedc300875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:48.289417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:48.456409 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 696/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 09:46:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:48.462146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457150f0-8faf-4d32-b59d-dd6e4b021dad None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:48.462146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-457150f0-8faf-4d32-b59d-dd6e4b021dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:48.698285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457150f0-8faf-4d32-b59d-dd6e4b021dad None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:48.733742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:48.936405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cb510-a6a4-4055-9822-5efa6994f24e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:48.937287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cb510-a6a4-4055-9822-5efa6994f24e service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:48.937672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cb510-a6a4-4055-9822-5efa6994f24e service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:48.938032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cb510-a6a4-4055-9822-5efa6994f24e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:48.944294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6f7cb510-a6a4-4055-9822-5efa6994f24e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:49.048177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cb510-a6a4-4055-9822-5efa6994f24e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:49.050669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cb510-a6a4-4055-9822-5efa6994f24e service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:49.071976 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 672/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:46:52.173257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48384e84-3391-4b19-969a-43590b5b8ef3 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:52.175773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48384e84-3391-4b19-969a-43590b5b8ef3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:52.176587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48384e84-3391-4b19-969a-43590b5b8ef3 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:52.179774 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 697/1369] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:52.185526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05e58-dd57-48bb-86d9-2436e3397dde None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:52.186027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05e58-dd57-48bb-86d9-2436e3397dde None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:52.186318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05e58-dd57-48bb-86d9-2436e3397dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:52.233667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:52.304826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-63c05e58-dd57-48bb-86d9-2436e3397dde None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:52.306945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-63c05e58-dd57-48bb-86d9-2436e3397dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:52.497270 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 673/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:52.817116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04305c-5950-45ab-af8a-571bfc83b36e None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:52.817428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04305c-5950-45ab-af8a-571bfc83b36e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:52.817530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04305c-5950-45ab-af8a-571bfc83b36e None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:52.818936 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 698/1371] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:52.825155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a575d-d25d-4e41-8be3-5c3b7700f63a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:52.825441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a575d-d25d-4e41-8be3-5c3b7700f63a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:52.825715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a575d-d25d-4e41-8be3-5c3b7700f63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:52.903489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6e0a575d-d25d-4e41-8be3-5c3b7700f63a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:52.904857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6e0a575d-d25d-4e41-8be3-5c3b7700f63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:52.971757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:53.026759 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 674/1372] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:53.081184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04305c-5950-45ab-af8a-571bfc83b36e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:53.083400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3d04305c-5950-45ab-af8a-571bfc83b36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:53.270640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:53.276912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04305c-5950-45ab-af8a-571bfc83b36e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:53.377382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7833f-39fe-4ddd-94c8-22eedfb5202c None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:53.378029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7833f-39fe-4ddd-94c8-22eedfb5202c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:53.378029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7833f-39fe-4ddd-94c8-22eedfb5202c None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:53.379458 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 675/1373] 10.222.0.26 () {54 vars in 963 bytes} [Tue Sep 27 09:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:46:53.385056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343fcef-fad6-4302-b67b-9eeef36c316a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:53.385572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343fcef-fad6-4302-b67b-9eeef36c316a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:53.385863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343fcef-fad6-4302-b67b-9eeef36c316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:53.413853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:53.414714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:53.414714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:53.414991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:53.417082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:53.476706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a343fcef-fad6-4302-b67b-9eeef36c316a None None] MFA Rules not processed for user `b324af05ec3e4b35a9bfc188b3b3abec`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:53.478226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a343fcef-fad6-4302-b67b-9eeef36c316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:53.529383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:53.531037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:53.552379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:53.560729 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 699/1374] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:53.597369 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 676/1375] 10.222.0.26 () {58 vars in 1060 bytes} [Tue Sep 27 09:46:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:46:53.606638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:53.608036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:53.713979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131957ad-f843-40fe-9e16-4451e34c0336 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:53.815290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f3a3-1a8c-42da-b7c8-61237ac01d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:53.816068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f3a3-1a8c-42da-b7c8-61237ac01d7d service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:53.816463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f3a3-1a8c-42da-b7c8-61237ac01d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:53.816892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f3a3-1a8c-42da-b7c8-61237ac01d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:53.818893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9f94f3a3-1a8c-42da-b7c8-61237ac01d7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:53.935220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:53.995134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f3a3-1a8c-42da-b7c8-61237ac01d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:53.996639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f3a3-1a8c-42da-b7c8-61237ac01d7d service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:54.027834 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 700/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:46:56.327481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54b63-f5f1-4bf8-b2e9-8fc7962da22c None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:56.328178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54b63-f5f1-4bf8-b2e9-8fc7962da22c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:56.328178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54b63-f5f1-4bf8-b2e9-8fc7962da22c None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:56.329834 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 677/1377] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:46:56.335580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c25e9-d876-4887-99d7-de128a455801 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:56.336109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c25e9-d876-4887-99d7-de128a455801 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:56.336109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c25e9-d876-4887-99d7-de128a455801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:56.414989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b98c25e9-d876-4887-99d7-de128a455801 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:46:56.416593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b98c25e9-d876-4887-99d7-de128a455801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:56.526003 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 701/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:46:56.880955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad54b63-f5f1-4bf8-b2e9-8fc7962da22c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:46:56.882797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2ad54b63-f5f1-4bf8-b2e9-8fc7962da22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:56.945288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:57.052331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad54b63-f5f1-4bf8-b2e9-8fc7962da22c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:46:57.194719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a729bfc-b59f-4e41-a949-81b61678bdfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:46:57.195597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a729bfc-b59f-4e41-a949-81b61678bdfb service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:46:57.196023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a729bfc-b59f-4e41-a949-81b61678bdfb service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:46:57.196434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a729bfc-b59f-4e41-a949-81b61678bdfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:46:57.198895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3a729bfc-b59f-4e41-a949-81b61678bdfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:46:57.216101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:57.303939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a729bfc-b59f-4e41-a949-81b61678bdfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:46:57.305625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a729bfc-b59f-4e41-a949-81b61678bdfb service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:46:57.325816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 678/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:00.004928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428e58e-1fcd-485f-b453-b3c9889826fa None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:00.005809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428e58e-1fcd-485f-b453-b3c9889826fa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:00.005921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428e58e-1fcd-485f-b453-b3c9889826fa None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:00.007663 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 702/1380] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:00.013641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d0abd-3b70-4abd-bb00-617954722fb8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:00.014232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d0abd-3b70-4abd-bb00-617954722fb8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:00.014341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d0abd-3b70-4abd-bb00-617954722fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:00.099524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-667d0abd-3b70-4abd-bb00-617954722fb8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:00.100825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-667d0abd-3b70-4abd-bb00-617954722fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:00.202676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:00.220704 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 679/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:00.578144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428e58e-1fcd-485f-b453-b3c9889826fa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:00.581266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c428e58e-1fcd-485f-b453-b3c9889826fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:00.635782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:00.771435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428e58e-1fcd-485f-b453-b3c9889826fa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:00.904937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:00.906337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c61c81-8893-4ce3-97a3-70dcaa7eaa38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:00.907018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c61c81-8893-4ce3-97a3-70dcaa7eaa38 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:00.907277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c61c81-8893-4ce3-97a3-70dcaa7eaa38 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:00.907678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c61c81-8893-4ce3-97a3-70dcaa7eaa38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:00.909823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-50c61c81-8893-4ce3-97a3-70dcaa7eaa38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:01.009290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c61c81-8893-4ce3-97a3-70dcaa7eaa38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:01.011062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c61c81-8893-4ce3-97a3-70dcaa7eaa38 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:01.034337 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 703/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:03.620050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe378c-7ef7-4e49-89a1-c8271f0957f2 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:03.620825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe378c-7ef7-4e49-89a1-c8271f0957f2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:03.620825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe378c-7ef7-4e49-89a1-c8271f0957f2 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:03.622089 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 680/1383] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:03.628406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2d21-5845-4084-86b7-e4f99351ea26 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:03.629117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2d21-5845-4084-86b7-e4f99351ea26 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:03.629117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2d21-5845-4084-86b7-e4f99351ea26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:03.719910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e68d2d21-5845-4084-86b7-e4f99351ea26 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:03.721608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e68d2d21-5845-4084-86b7-e4f99351ea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:03.813190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:03.852849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 704/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:47:04.218403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe378c-7ef7-4e49-89a1-c8271f0957f2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:04.220122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-03fe378c-7ef7-4e49-89a1-c8271f0957f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:04.410822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe378c-7ef7-4e49-89a1-c8271f0957f2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:04.498177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:04.573094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8abee7-7e63-4c7a-a7ce-9bd77a0a06ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:04.573748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8abee7-7e63-4c7a-a7ce-9bd77a0a06ef service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:04.574079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8abee7-7e63-4c7a-a7ce-9bd77a0a06ef service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:04.574307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8abee7-7e63-4c7a-a7ce-9bd77a0a06ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:04.576105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dd8abee7-7e63-4c7a-a7ce-9bd77a0a06ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:04.710305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8abee7-7e63-4c7a-a7ce-9bd77a0a06ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:04.712502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8abee7-7e63-4c7a-a7ce-9bd77a0a06ef service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:04.736647 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 681/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:07.829093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6bfa-6523-4fb8-8ae0-e8f52893f13b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:07.829997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6bfa-6523-4fb8-8ae0-e8f52893f13b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:07.829997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6bfa-6523-4fb8-8ae0-e8f52893f13b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:07.832575 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 705/1386] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:07.838762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4bf8e-90f8-4033-bb8f-da2a322d6473 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:07.839328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4bf8e-90f8-4033-bb8f-da2a322d6473 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:07.839578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4bf8e-90f8-4033-bb8f-da2a322d6473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:07.918174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4fb4bf8e-90f8-4033-bb8f-da2a322d6473 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:07.919444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4fb4bf8e-90f8-4033-bb8f-da2a322d6473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:07.949466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:08.042244 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 682/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:08.391183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6bfa-6523-4fb8-8ae0-e8f52893f13b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:08.393505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1acd6bfa-6523-4fb8-8ae0-e8f52893f13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:08.602432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6bfa-6523-4fb8-8ae0-e8f52893f13b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:08.667093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:08.731639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018334e-6f52-4b88-a4c8-9cd709b8524f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:08.732257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018334e-6f52-4b88-a4c8-9cd709b8524f service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:08.732533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018334e-6f52-4b88-a4c8-9cd709b8524f service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:08.732886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018334e-6f52-4b88-a4c8-9cd709b8524f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:08.734527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5018334e-6f52-4b88-a4c8-9cd709b8524f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:08.856380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018334e-6f52-4b88-a4c8-9cd709b8524f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:08.859035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018334e-6f52-4b88-a4c8-9cd709b8524f service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:08.880618 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 706/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:11.024512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410ca3b-ce31-457b-894b-f466b850bbcf None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:11.025260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410ca3b-ce31-457b-894b-f466b850bbcf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:11.025260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410ca3b-ce31-457b-894b-f466b850bbcf None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:11.026460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 683/1389] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:47:11.032176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae086-8037-43a9-bc95-3ea721ab1110 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:11.032748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae086-8037-43a9-bc95-3ea721ab1110 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:11.033102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae086-8037-43a9-bc95-3ea721ab1110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:11.107547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-88fae086-8037-43a9-bc95-3ea721ab1110 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:11.108804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-88fae086-8037-43a9-bc95-3ea721ab1110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:11.131122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:11.222809 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 707/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:11.399045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0def4-859d-4ac8-8a3c-f0b22f6f64e1 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:11.399630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0def4-859d-4ac8-8a3c-f0b22f6f64e1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:11.399902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0def4-859d-4ac8-8a3c-f0b22f6f64e1 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:11.401387 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 684/1391] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:11.407622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:11.407973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:11.408911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:11.481944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:11.483214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:11.596374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06ccb1-ca12-4edf-907c-c5f07cc854ea None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:11.596905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06ccb1-ca12-4edf-907c-c5f07cc854ea None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:11.597237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06ccb1-ca12-4edf-907c-c5f07cc854ea None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:11.598618 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 685/1392] 10.222.0.26 () {54 vars in 962 bytes} [Tue Sep 27 09:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:47:11.602411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:11.609610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:11.610169 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 708/1393] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:47:11.610294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:11.610294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:11.694029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] MFA Rules not processed for user `ed91b2c83d57419dac50595f57fc3411`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:11.695144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:11.775589 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 686/1394] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 27 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:11.783959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:11.785431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:11.885317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb65a3-fecf-43c3-af66-130106710d82 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:11.963526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:11.965196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:11.972692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f874a-7222-4a27-9e40-5305a76da398 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:11.973345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f874a-7222-4a27-9e40-5305a76da398 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:11.973618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f874a-7222-4a27-9e40-5305a76da398 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:11.973884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f874a-7222-4a27-9e40-5305a76da398 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:11.975547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea5f874a-7222-4a27-9e40-5305a76da398 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:11.992039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:12.045507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:12.098283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f874a-7222-4a27-9e40-5305a76da398 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:12.099721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f874a-7222-4a27-9e40-5305a76da398 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:12.103948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1c79a-7d59-47bc-8c1d-aacfb6306c9c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:12.119782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 709/1395] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:12.229356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f14a18-1648-4558-81dd-33b9ef6fc073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:12.229884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f14a18-1648-4558-81dd-33b9ef6fc073 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:12.230161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f14a18-1648-4558-81dd-33b9ef6fc073 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:12.230433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f14a18-1648-4558-81dd-33b9ef6fc073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:12.232054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-01f14a18-1648-4558-81dd-33b9ef6fc073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:12.245779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:12.338384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f14a18-1648-4558-81dd-33b9ef6fc073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:12.340304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f14a18-1648-4558-81dd-33b9ef6fc073 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:12.362744 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 687/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:47:12.734087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfdc62-140e-4e76-8c22-e9f961786671 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:12.734428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfdc62-140e-4e76-8c22-e9f961786671 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:12.734925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfdc62-140e-4e76-8c22-e9f961786671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:12.820612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-31bfdc62-140e-4e76-8c22-e9f961786671 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:12.821867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-31bfdc62-140e-4e76-8c22-e9f961786671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:12.927020 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 710/1397] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:12.981894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf612a-fdfd-4787-b2c4-098da8104db7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:12.982170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf612a-fdfd-4787-b2c4-098da8104db7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:12.982440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf612a-fdfd-4787-b2c4-098da8104db7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:12.983863 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 688/1398] 10.222.0.26 () {54 vars in 943 bytes} [Tue Sep 27 09:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:12.992281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aeb48-cff5-4ff1-9873-5ffa58476cf5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:12.992563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aeb48-cff5-4ff1-9873-5ffa58476cf5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:12.992828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aeb48-cff5-4ff1-9873-5ffa58476cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:13.035882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:13.089713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-063aeb48-cff5-4ff1-9873-5ffa58476cf5 None None] MFA Rules not processed for user `c6863dfc2c0143a88060bbf5f0e146df`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:13.090874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-063aeb48-cff5-4ff1-9873-5ffa58476cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:13.226683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 711/1399] 10.222.0.26 () {58 vars in 1040 bytes} [Tue Sep 27 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:13.237672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf612a-fdfd-4787-b2c4-098da8104db7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:13.239315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eccf612a-fdfd-4787-b2c4-098da8104db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:13.422204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf612a-fdfd-4787-b2c4-098da8104db7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:13.425448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:13.551963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ec3ee-a3f6-46a8-ad3a-05fb13b73cbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:13.552579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ec3ee-a3f6-46a8-ad3a-05fb13b73cbe service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:13.552891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ec3ee-a3f6-46a8-ad3a-05fb13b73cbe service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:13.553166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ec3ee-a3f6-46a8-ad3a-05fb13b73cbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:13.554859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-388ec3ee-a3f6-46a8-ad3a-05fb13b73cbe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:13.675099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ec3ee-a3f6-46a8-ad3a-05fb13b73cbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:13.676692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ec3ee-a3f6-46a8-ad3a-05fb13b73cbe service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:13.687166 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 689/1400] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 09:47:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 09:47:14.932238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf42c5-d991-4e3e-b015-9a97074e57fa None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:14.932939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf42c5-d991-4e3e-b015-9a97074e57fa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:14.933816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf42c5-d991-4e3e-b015-9a97074e57fa None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:14.936711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 712/1401] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:47:14.945769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7abf0-deb3-4602-910a-29d914915946 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:14.946419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7abf0-deb3-4602-910a-29d914915946 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:14.946700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7abf0-deb3-4602-910a-29d914915946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:15.021552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:15.061926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d6c7abf0-deb3-4602-910a-29d914915946 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:15.063627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d6c7abf0-deb3-4602-910a-29d914915946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:15.214834 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 690/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:15.588262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf42c5-d991-4e3e-b015-9a97074e57fa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:15.595347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-95bf42c5-d991-4e3e-b015-9a97074e57fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:15.717423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:15.803211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf42c5-d991-4e3e-b015-9a97074e57fa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:15.961276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902455ac-3852-415d-8e0b-bb9f1669d221 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:15.962206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902455ac-3852-415d-8e0b-bb9f1669d221 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:15.962206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902455ac-3852-415d-8e0b-bb9f1669d221 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:15.962442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902455ac-3852-415d-8e0b-bb9f1669d221 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:15.964116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-902455ac-3852-415d-8e0b-bb9f1669d221 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:16.038673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:16.088649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902455ac-3852-415d-8e0b-bb9f1669d221 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:16.090400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902455ac-3852-415d-8e0b-bb9f1669d221 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:16.121737 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 713/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:18.751062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b075d-cd4b-4bcf-9ea3-7833eeb78c7b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:18.752197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b075d-cd4b-4bcf-9ea3-7833eeb78c7b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:18.752197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b075d-cd4b-4bcf-9ea3-7833eeb78c7b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:18.757031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 691/1404] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:18.759169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314e38-fcad-4ccb-9d0f-6a78be1c9233 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:18.759676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314e38-fcad-4ccb-9d0f-6a78be1c9233 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:18.759773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314e38-fcad-4ccb-9d0f-6a78be1c9233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:18.840880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dd314e38-fcad-4ccb-9d0f-6a78be1c9233 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:18.842135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dd314e38-fcad-4ccb-9d0f-6a78be1c9233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:18.966392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 714/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:19.326696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b075d-cd4b-4bcf-9ea3-7833eeb78c7b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:19.328336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ed2b075d-cd4b-4bcf-9ea3-7833eeb78c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:19.476141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:19.522924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b075d-cd4b-4bcf-9ea3-7833eeb78c7b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:19.669393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ba553-c2ba-42a8-a263-02aa6c66f34b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:19.670009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ba553-c2ba-42a8-a263-02aa6c66f34b service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:19.670291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ba553-c2ba-42a8-a263-02aa6c66f34b service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:19.670560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ba553-c2ba-42a8-a263-02aa6c66f34b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:19.672246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-767ba553-c2ba-42a8-a263-02aa6c66f34b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:19.818441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:19.863938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ba553-c2ba-42a8-a263-02aa6c66f34b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:19.865541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ba553-c2ba-42a8-a263-02aa6c66f34b service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:19.890284 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 692/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:22.468275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5429-356a-49c4-a5a0-732bfa57d863 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:22.468945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5429-356a-49c4-a5a0-732bfa57d863 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:22.468945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5429-356a-49c4-a5a0-732bfa57d863 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:22.470790 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 715/1407] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:22.475962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dce68-4864-4e8f-ad60-23dbd329430f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:22.476474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dce68-4864-4e8f-ad60-23dbd329430f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:22.476571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dce68-4864-4e8f-ad60-23dbd329430f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:22.558780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2f6dce68-4864-4e8f-ad60-23dbd329430f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:22.560006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2f6dce68-4864-4e8f-ad60-23dbd329430f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:22.684019 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 693/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:23.037885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc5429-356a-49c4-a5a0-732bfa57d863 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:23.039745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ffc5429-356a-49c4-a5a0-732bfa57d863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:23.084694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:23.220238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc5429-356a-49c4-a5a0-732bfa57d863 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:23.348838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:23.363783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4d410-d928-4b05-bb8a-432ae784e1cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:23.364407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4d410-d928-4b05-bb8a-432ae784e1cb service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:23.364706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4d410-d928-4b05-bb8a-432ae784e1cb service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:23.364975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4d410-d928-4b05-bb8a-432ae784e1cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:23.367098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-45b4d410-d928-4b05-bb8a-432ae784e1cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:23.469500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4d410-d928-4b05-bb8a-432ae784e1cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:23.471219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4d410-d928-4b05-bb8a-432ae784e1cb service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:23.491096 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 716/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:32.682148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=77 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:33.488692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=50 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:37.661946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd2f0-c10c-491a-9394-d670cae9aedd None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:37.662526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd2f0-c10c-491a-9394-d670cae9aedd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:37.662526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd2f0-c10c-491a-9394-d670cae9aedd None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:37.663834 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 694/1410] 10.222.0.26 () {54 vars in 918 bytes} [Tue Sep 27 09:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:37.669758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65c6d-dbbe-4518-9e1e-8a6986c7383d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:37.670285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65c6d-dbbe-4518-9e1e-8a6986c7383d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:37.670285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65c6d-dbbe-4518-9e1e-8a6986c7383d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:37.755060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-67d65c6d-dbbe-4518-9e1e-8a6986c7383d None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:37.756427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-67d65c6d-dbbe-4518-9e1e-8a6986c7383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:37.885988 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 717/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 27 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:37.902152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51809aa-924a-4115-8ed4-6fe2594cff71 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:37.902418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51809aa-924a-4115-8ed4-6fe2594cff71 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:37.902677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51809aa-924a-4115-8ed4-6fe2594cff71 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:37.904033 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 695/1412] 10.222.0.26 () {54 vars in 918 bytes} [Tue Sep 27 09:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:37.909869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c3e5d-be2d-4b96-a27e-4e41d4f2c625 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:37.910135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c3e5d-be2d-4b96-a27e-4e41d4f2c625 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:37.910422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c3e5d-be2d-4b96-a27e-4e41d4f2c625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:37.999576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:38.003901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3c7c3e5d-be2d-4b96-a27e-4e41d4f2c625 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:38.006000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3c7c3e5d-be2d-4b96-a27e-4e41d4f2c625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:38.136214 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 718/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 27 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:41.032413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b58d-cdda-4e81-b80e-3ce944b4de83 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:41.033025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b58d-cdda-4e81-b80e-3ce944b4de83 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:41.033526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b58d-cdda-4e81-b80e-3ce944b4de83 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:41.034913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:47:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:41.040821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6ab35-2098-4477-8718-a2c85bf41469 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:41.041308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6ab35-2098-4477-8718-a2c85bf41469 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:41.041308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6ab35-2098-4477-8718-a2c85bf41469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:41.121405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-63b6ab35-2098-4477-8718-a2c85bf41469 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:41.122637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63b6ab35-2098-4477-8718-a2c85bf41469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:41.181274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:41.226843 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:41.579654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b58d-cdda-4e81-b80e-3ce944b4de83 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:41.581352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a735b58d-cdda-4e81-b80e-3ce944b4de83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:41.737717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b58d-cdda-4e81-b80e-3ce944b4de83 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:41.814792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:41.855773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30038489-cb5f-440e-a124-510f2cf80a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:41.856423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038489-cb5f-440e-a124-510f2cf80a69 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:41.856686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038489-cb5f-440e-a124-510f2cf80a69 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:41.856985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038489-cb5f-440e-a124-510f2cf80a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:41.858579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30038489-cb5f-440e-a124-510f2cf80a69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:41.953960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30038489-cb5f-440e-a124-510f2cf80a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:41.955633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30038489-cb5f-440e-a124-510f2cf80a69 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:41.974109 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 697/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:43.699362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af592c54-cb66-4186-924b-eb7e5bc7f0df None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:43.699921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af592c54-cb66-4186-924b-eb7e5bc7f0df None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:43.699921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af592c54-cb66-4186-924b-eb7e5bc7f0df None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:43.701186 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:43.706942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae07169-a5db-432b-965e-c6758a8d5596 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:43.707379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae07169-a5db-432b-965e-c6758a8d5596 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:43.707474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae07169-a5db-432b-965e-c6758a8d5596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:43.787440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9ae07169-a5db-432b-965e-c6758a8d5596 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:43.788689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9ae07169-a5db-432b-965e-c6758a8d5596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:43.817292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:43.902276 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:44.140649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af592c54-cb66-4186-924b-eb7e5bc7f0df None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:44.142266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-af592c54-cb66-4186-924b-eb7e5bc7f0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:44.294987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af592c54-cb66-4186-924b-eb7e5bc7f0df None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:44.339389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:44.418628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b613c45-abb2-4819-b1f3-10ec361245b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:44.419248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b613c45-abb2-4819-b1f3-10ec361245b0 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:44.419533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b613c45-abb2-4819-b1f3-10ec361245b0 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:44.419796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b613c45-abb2-4819-b1f3-10ec361245b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:44.421445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6b613c45-abb2-4819-b1f3-10ec361245b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:44.519351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b613c45-abb2-4819-b1f3-10ec361245b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:44.520983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b613c45-abb2-4819-b1f3-10ec361245b0 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:44.539499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 721/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:46.306972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657e86e-f6b9-4a2a-848b-c462640c5487 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:46.307622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657e86e-f6b9-4a2a-848b-c462640c5487 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:46.307622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657e86e-f6b9-4a2a-848b-c462640c5487 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:46.308877 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:46.314177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def5ca-ff4e-41a3-a462-f59f20032559 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:46.314806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def5ca-ff4e-41a3-a462-f59f20032559 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:46.314806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def5ca-ff4e-41a3-a462-f59f20032559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:46.374344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:46.399434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f9def5ca-ff4e-41a3-a462-f59f20032559 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:46.400693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9def5ca-ff4e-41a3-a462-f59f20032559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:46.508352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:46.677491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657e86e-f6b9-4a2a-848b-c462640c5487 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:46.679433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6657e86e-f6b9-4a2a-848b-c462640c5487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:46.838877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657e86e-f6b9-4a2a-848b-c462640c5487 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:46.851579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:46.957752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca0943-a6de-48c6-a08e-76c3006a302b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:46.958242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca0943-a6de-48c6-a08e-76c3006a302b service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:46.958511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca0943-a6de-48c6-a08e-76c3006a302b service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:46.958780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca0943-a6de-48c6-a08e-76c3006a302b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:46.960392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f8ca0943-a6de-48c6-a08e-76c3006a302b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:47.058250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca0943-a6de-48c6-a08e-76c3006a302b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:47.059865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca0943-a6de-48c6-a08e-76c3006a302b service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:47.078964 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 700/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:48.827379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aed5a-356d-4483-bea0-77c2f9bb28e6 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:48.827974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aed5a-356d-4483-bea0-77c2f9bb28e6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:48.827974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aed5a-356d-4483-bea0-77c2f9bb28e6 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:48.829348 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:48.834798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773367a-cc6a-4db8-a150-0338b74c0e50 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:48.835300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773367a-cc6a-4db8-a150-0338b74c0e50 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:48.835403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773367a-cc6a-4db8-a150-0338b74c0e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:48.857422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:48.922150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7773367a-cc6a-4db8-a150-0338b74c0e50 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:48.924108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7773367a-cc6a-4db8-a150-0338b74c0e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:49.027128 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:49.193255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999aed5a-356d-4483-bea0-77c2f9bb28e6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:49.194681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-999aed5a-356d-4483-bea0-77c2f9bb28e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:49.317386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:49.356711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999aed5a-356d-4483-bea0-77c2f9bb28e6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:49.478998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698559bb-edf2-44d7-995a-5317191072ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:49.479517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698559bb-edf2-44d7-995a-5317191072ac service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:49.479797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698559bb-edf2-44d7-995a-5317191072ac service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:49.480065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698559bb-edf2-44d7-995a-5317191072ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:49.481863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-698559bb-edf2-44d7-995a-5317191072ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:49.702312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:49.718628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698559bb-edf2-44d7-995a-5317191072ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:49.720318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698559bb-edf2-44d7-995a-5317191072ac service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:49.739542 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 724/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:51.432549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135cc63-236b-43b2-a919-257058ea207d None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:51.433152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135cc63-236b-43b2-a919-257058ea207d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:51.433152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135cc63-236b-43b2-a919-257058ea207d None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:51.434450 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:51.439530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627124-148b-47f7-9673-333d444d933d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:51.440039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627124-148b-47f7-9673-333d444d933d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:51.440039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627124-148b-47f7-9673-333d444d933d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:51.534309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f9627124-148b-47f7-9673-333d444d933d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:51.535530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9627124-148b-47f7-9673-333d444d933d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:51.653569 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:51.816824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135cc63-236b-43b2-a919-257058ea207d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:51.818410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4135cc63-236b-43b2-a919-257058ea207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:51.909649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:51.975574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135cc63-236b-43b2-a919-257058ea207d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:52.096036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc8947-85c3-47e1-9256-edf4bb16d033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:52.096552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc8947-85c3-47e1-9256-edf4bb16d033 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:52.096859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc8947-85c3-47e1-9256-edf4bb16d033 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:52.097124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc8947-85c3-47e1-9256-edf4bb16d033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:52.098723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c2fc8947-85c3-47e1-9256-edf4bb16d033 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:52.150386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:52.193723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc8947-85c3-47e1-9256-edf4bb16d033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:52.195310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc8947-85c3-47e1-9256-edf4bb16d033 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:52.213795 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 703/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:53.898534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365301e-bc79-4006-8758-a0623c70da5b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:53.899093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365301e-bc79-4006-8758-a0623c70da5b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:53.899093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365301e-bc79-4006-8758-a0623c70da5b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:53.900318 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:53.905690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf604b-1f6c-4a17-a34e-8367dafc1f0c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:53.906118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf604b-1f6c-4a17-a34e-8367dafc1f0c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:53.906277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf604b-1f6c-4a17-a34e-8367dafc1f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:53.993911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6faf604b-1f6c-4a17-a34e-8367dafc1f0c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:53.995169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6faf604b-1f6c-4a17-a34e-8367dafc1f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:54.099250 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:54.259530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365301e-bc79-4006-8758-a0623c70da5b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:54.260996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5365301e-bc79-4006-8758-a0623c70da5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:54.315983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:54.419889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365301e-bc79-4006-8758-a0623c70da5b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:54.539539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca91bb-5f1a-4d95-858e-970a621b7c92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:54.540172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca91bb-5f1a-4d95-858e-970a621b7c92 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:54.540443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca91bb-5f1a-4d95-858e-970a621b7c92 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:54.540707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca91bb-5f1a-4d95-858e-970a621b7c92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:54.542672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cbca91bb-5f1a-4d95-858e-970a621b7c92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:54.550658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:54.645906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca91bb-5f1a-4d95-858e-970a621b7c92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:54.647542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca91bb-5f1a-4d95-858e-970a621b7c92 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:54.667910 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 727/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:56.309400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66018b30-126f-44fe-a02f-f679ce5ace9a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:56.310014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66018b30-126f-44fe-a02f-f679ce5ace9a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:56.310014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66018b30-126f-44fe-a02f-f679ce5ace9a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:56.311261 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:56.316535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d50f4-613a-4645-b674-d500230a0d78 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:56.317047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d50f4-613a-4645-b674-d500230a0d78 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:56.317047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d50f4-613a-4645-b674-d500230a0d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:56.402564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f01d50f4-613a-4645-b674-d500230a0d78 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:56.403788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f01d50f4-613a-4645-b674-d500230a0d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:56.474548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:56.505140 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:56.670111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66018b30-126f-44fe-a02f-f679ce5ace9a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:56.671695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66018b30-126f-44fe-a02f-f679ce5ace9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:56.692398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:56.833357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66018b30-126f-44fe-a02f-f679ce5ace9a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:56.947411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:56.975980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd2c86-c7d5-44b5-9150-6d55c9535f01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:56.976594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd2c86-c7d5-44b5-9150-6d55c9535f01 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:56.976895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd2c86-c7d5-44b5-9150-6d55c9535f01 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:56.977160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd2c86-c7d5-44b5-9150-6d55c9535f01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:56.978742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62fd2c86-c7d5-44b5-9150-6d55c9535f01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:57.077438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd2c86-c7d5-44b5-9150-6d55c9535f01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:57.079447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd2c86-c7d5-44b5-9150-6d55c9535f01 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:57.100129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 706/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:58.753705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442295b-8257-400a-b259-5d2478a1ae02 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:58.754332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442295b-8257-400a-b259-5d2478a1ae02 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:58.754332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442295b-8257-400a-b259-5d2478a1ae02 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:58.755556 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:47:58.761605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7fc9a-5c34-4565-817e-ddfa3b3142a6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:58.762203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7fc9a-5c34-4565-817e-ddfa3b3142a6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:58.762203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7fc9a-5c34-4565-817e-ddfa3b3142a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:58.849503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ade7fc9a-5c34-4565-817e-ddfa3b3142a6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:47:58.850747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ade7fc9a-5c34-4565-817e-ddfa3b3142a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:58.894469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:58.951208 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:47:59.131977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442295b-8257-400a-b259-5d2478a1ae02 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:47:59.133533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e442295b-8257-400a-b259-5d2478a1ae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:59.294174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442295b-8257-400a-b259-5d2478a1ae02 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:47:59.369854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:59.452199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd68289-65ae-400b-bd36-3ee6fde5b4b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:47:59.452851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd68289-65ae-400b-bd36-3ee6fde5b4b3 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:47:59.453125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd68289-65ae-400b-bd36-3ee6fde5b4b3 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:47:59.453393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd68289-65ae-400b-bd36-3ee6fde5b4b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:47:59.455392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-acd68289-65ae-400b-bd36-3ee6fde5b4b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:47:59.586514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd68289-65ae-400b-bd36-3ee6fde5b4b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:47:59.588090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd68289-65ae-400b-bd36-3ee6fde5b4b3 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:47:59.615573 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 730/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:01.289676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171761fc-2b3e-463b-9ef1-2ac2270de5eb None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:01.291116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171761fc-2b3e-463b-9ef1-2ac2270de5eb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:01.291116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171761fc-2b3e-463b-9ef1-2ac2270de5eb None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:01.293761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:48:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:01.299501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af1fe-3275-4250-898d-3dd1be5821aa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:01.299944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af1fe-3275-4250-898d-3dd1be5821aa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:01.300194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af1fe-3275-4250-898d-3dd1be5821aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:01.381104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-023af1fe-3275-4250-898d-3dd1be5821aa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:01.382395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-023af1fe-3275-4250-898d-3dd1be5821aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:01.384648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:01.498447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:01.668281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171761fc-2b3e-463b-9ef1-2ac2270de5eb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:01.670741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-171761fc-2b3e-463b-9ef1-2ac2270de5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:01.838011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171761fc-2b3e-463b-9ef1-2ac2270de5eb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:01.866433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:01.965390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9416274-49a3-4972-8c61-9c554c6b33ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:01.966004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9416274-49a3-4972-8c61-9c554c6b33ae service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:01.966278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9416274-49a3-4972-8c61-9c554c6b33ae service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:01.966545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9416274-49a3-4972-8c61-9c554c6b33ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:01.968135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a9416274-49a3-4972-8c61-9c554c6b33ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:02.071949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9416274-49a3-4972-8c61-9c554c6b33ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:02.073604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9416274-49a3-4972-8c61-9c554c6b33ae service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:02.096215 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 709/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:04.483842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094680e2-b1d5-4dc8-a626-b7167e49f5cd None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:04.484482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094680e2-b1d5-4dc8-a626-b7167e49f5cd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:04.484482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094680e2-b1d5-4dc8-a626-b7167e49f5cd None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:04.485807 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:04.491460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaf226-da1a-46a9-acb7-f4439f67d11a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:04.492029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaf226-da1a-46a9-acb7-f4439f67d11a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:04.492029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaf226-da1a-46a9-acb7-f4439f67d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:04.536725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:04.566496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7bdaf226-da1a-46a9-acb7-f4439f67d11a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:04.567733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bdaf226-da1a-46a9-acb7-f4439f67d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:04.674772 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:05.017402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094680e2-b1d5-4dc8-a626-b7167e49f5cd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:05.018929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-094680e2-b1d5-4dc8-a626-b7167e49f5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:05.129043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094680e2-b1d5-4dc8-a626-b7167e49f5cd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:05.163001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:05.209093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e752e12-abb2-4fb1-99d9-5096415ad22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:05.209607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e752e12-abb2-4fb1-99d9-5096415ad22f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:05.209884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e752e12-abb2-4fb1-99d9-5096415ad22f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:05.210181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e752e12-abb2-4fb1-99d9-5096415ad22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:05.211816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e752e12-abb2-4fb1-99d9-5096415ad22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:05.329024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e752e12-abb2-4fb1-99d9-5096415ad22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:05.330491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e752e12-abb2-4fb1-99d9-5096415ad22f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:05.349380 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 733/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:07.988319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e3fd-f478-4fda-bdbf-5592e3e8485b None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:07.988877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e3fd-f478-4fda-bdbf-5592e3e8485b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:07.988877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e3fd-f478-4fda-bdbf-5592e3e8485b None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:07.990169 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:07.995964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991155df-08f8-43f8-904a-5446d9fb0308 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:07.996424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991155df-08f8-43f8-904a-5446d9fb0308 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:07.996537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991155df-08f8-43f8-904a-5446d9fb0308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:08.068848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-991155df-08f8-43f8-904a-5446d9fb0308 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:08.070101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-991155df-08f8-43f8-904a-5446d9fb0308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:08.084198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:08.194363 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:08.549368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e3fd-f478-4fda-bdbf-5592e3e8485b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:08.551749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4000e3fd-f478-4fda-bdbf-5592e3e8485b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:08.671587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e3fd-f478-4fda-bdbf-5592e3e8485b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:08.685883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:08.776190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061eec68-92a8-4a5f-b675-3725dd346d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:08.777705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eec68-92a8-4a5f-b675-3725dd346d6a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:08.777705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eec68-92a8-4a5f-b675-3725dd346d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:08.777705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eec68-92a8-4a5f-b675-3725dd346d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:08.779113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-061eec68-92a8-4a5f-b675-3725dd346d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:08.915894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061eec68-92a8-4a5f-b675-3725dd346d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:08.917637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061eec68-92a8-4a5f-b675-3725dd346d6a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:08.937262 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 712/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:11.698529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cf8a0-815a-404d-930f-d6d74f90067e None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:11.699365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cf8a0-815a-404d-930f-d6d74f90067e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:11.699365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cf8a0-815a-404d-930f-d6d74f90067e None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:11.701811 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:11.708695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41f7a-62cc-45c3-b8e0-38ff5fdde708 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:11.709215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41f7a-62cc-45c3-b8e0-38ff5fdde708 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:11.709311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41f7a-62cc-45c3-b8e0-38ff5fdde708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:11.888502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:11.918275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-70a41f7a-62cc-45c3-b8e0-38ff5fdde708 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:11.919998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-70a41f7a-62cc-45c3-b8e0-38ff5fdde708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:12.032094 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:48:12.384022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9cf8a0-815a-404d-930f-d6d74f90067e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:12.385862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fe9cf8a0-815a-404d-930f-d6d74f90067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:12.495610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9cf8a0-815a-404d-930f-d6d74f90067e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:12.537444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:12.576756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e3105-67c4-4305-b557-ec1128acb9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:12.577351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3105-67c4-4305-b557-ec1128acb9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:12.577637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3105-67c4-4305-b557-ec1128acb9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:12.577980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3105-67c4-4305-b557-ec1128acb9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:12.580502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c26e3105-67c4-4305-b557-ec1128acb9f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:12.712951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e3105-67c4-4305-b557-ec1128acb9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:12.714399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e3105-67c4-4305-b557-ec1128acb9f1 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:12.732312 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 736/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:15.394539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d882-c823-4f03-a3a5-a4a645890d5a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:15.395111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d882-c823-4f03-a3a5-a4a645890d5a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:15.395111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d882-c823-4f03-a3a5-a4a645890d5a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:15.396436 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:15.403976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904779d-6c52-4867-b215-5b6702490d2f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:15.404692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904779d-6c52-4867-b215-5b6702490d2f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:15.404692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904779d-6c52-4867-b215-5b6702490d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:15.475034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6904779d-6c52-4867-b215-5b6702490d2f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:15.476322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6904779d-6c52-4867-b215-5b6702490d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:15.498528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:15.582314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:15.925705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d882-c823-4f03-a3a5-a4a645890d5a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:15.927284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b921d882-c823-4f03-a3a5-a4a645890d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:16.046775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d882-c823-4f03-a3a5-a4a645890d5a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:16.071940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:16.135194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6af7c7-0100-4b9c-a363-17f07d09bf0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:16.135790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6af7c7-0100-4b9c-a363-17f07d09bf0d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:16.136084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6af7c7-0100-4b9c-a363-17f07d09bf0d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:16.136365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6af7c7-0100-4b9c-a363-17f07d09bf0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:16.138056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fa6af7c7-0100-4b9c-a363-17f07d09bf0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:16.262491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6af7c7-0100-4b9c-a363-17f07d09bf0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:16.263838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6af7c7-0100-4b9c-a363-17f07d09bf0d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:16.284996 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 715/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:18.956055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f25fa-66ea-4bb0-88b1-5f33fae72e7c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:18.956615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f25fa-66ea-4bb0-88b1-5f33fae72e7c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:18.956615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f25fa-66ea-4bb0-88b1-5f33fae72e7c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:18.957947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:18.964058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f5695-68ed-4edb-b308-2d7247349be7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:18.964556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f5695-68ed-4edb-b308-2d7247349be7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:18.964556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f5695-68ed-4edb-b308-2d7247349be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:19.035177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:19.039299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cb9f5695-68ed-4edb-b308-2d7247349be7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:19.041431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb9f5695-68ed-4edb-b308-2d7247349be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:19.150317 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:19.487765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f25fa-66ea-4bb0-88b1-5f33fae72e7c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:19.489385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-228f25fa-66ea-4bb0-88b1-5f33fae72e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:19.609555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f25fa-66ea-4bb0-88b1-5f33fae72e7c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:19.672138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:19.700503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896c7b4-52d1-4ca3-94b6-3a89e090c0e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:19.701161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896c7b4-52d1-4ca3-94b6-3a89e090c0e4 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:19.701441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896c7b4-52d1-4ca3-94b6-3a89e090c0e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:19.701736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896c7b4-52d1-4ca3-94b6-3a89e090c0e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:19.703375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4896c7b4-52d1-4ca3-94b6-3a89e090c0e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:19.829583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896c7b4-52d1-4ca3-94b6-3a89e090c0e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:19.831023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896c7b4-52d1-4ca3-94b6-3a89e090c0e4 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:19.851184 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 739/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:22.402973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790cb1c-d0f6-4bb8-bfd5-6294b1f01dc2 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:22.403585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790cb1c-d0f6-4bb8-bfd5-6294b1f01dc2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:22.403585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790cb1c-d0f6-4bb8-bfd5-6294b1f01dc2 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:22.404885 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:22.411278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffbef-ab7b-4c86-96cd-b61d6bafe5db None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:22.411820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffbef-ab7b-4c86-96cd-b61d6bafe5db None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:22.411820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffbef-ab7b-4c86-96cd-b61d6bafe5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:22.486909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1edffbef-ab7b-4c86-96cd-b61d6bafe5db None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:22.488164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1edffbef-ab7b-4c86-96cd-b61d6bafe5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:22.519646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:22.593204 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:22.931140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790cb1c-d0f6-4bb8-bfd5-6294b1f01dc2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:22.932605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3790cb1c-d0f6-4bb8-bfd5-6294b1f01dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:23.050105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790cb1c-d0f6-4bb8-bfd5-6294b1f01dc2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:23.080695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:23.129886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e4c75-2b1c-405f-89ed-ce6a7b09c334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:23.130504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e4c75-2b1c-405f-89ed-ce6a7b09c334 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:23.130778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e4c75-2b1c-405f-89ed-ce6a7b09c334 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:23.131047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e4c75-2b1c-405f-89ed-ce6a7b09c334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:23.132685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-654e4c75-2b1c-405f-89ed-ce6a7b09c334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:23.253979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e4c75-2b1c-405f-89ed-ce6a7b09c334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:23.255572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e4c75-2b1c-405f-89ed-ce6a7b09c334 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:23.275493 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 718/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:25.892998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33669-8c7c-4110-85c2-03e2e58e114c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:25.893571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33669-8c7c-4110-85c2-03e2e58e114c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:25.893571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33669-8c7c-4110-85c2-03e2e58e114c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:25.895008 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:25.901038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d7b6-514b-4442-8105-9b645b4d110a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:25.901514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d7b6-514b-4442-8105-9b645b4d110a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:25.901641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d7b6-514b-4442-8105-9b645b4d110a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:25.975707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-06f7d7b6-514b-4442-8105-9b645b4d110a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:25.976931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06f7d7b6-514b-4442-8105-9b645b4d110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:25.986454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:26.076411 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:48:26.427446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33669-8c7c-4110-85c2-03e2e58e114c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:26.429020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0de33669-8c7c-4110-85c2-03e2e58e114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:26.538350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33669-8c7c-4110-85c2-03e2e58e114c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:26.599187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:26.621379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdc13f-af77-426e-b158-0a2077bf4b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:26.621999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdc13f-af77-426e-b158-0a2077bf4b8d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:26.622272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdc13f-af77-426e-b158-0a2077bf4b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:26.622540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdc13f-af77-426e-b158-0a2077bf4b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:26.624147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cbbdc13f-af77-426e-b158-0a2077bf4b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:26.770976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdc13f-af77-426e-b158-0a2077bf4b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:26.772443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdc13f-af77-426e-b158-0a2077bf4b8d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:26.792315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 742/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:29.317552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490be829-27cb-46b1-8c1d-9636e70971e8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:29.318177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490be829-27cb-46b1-8c1d-9636e70971e8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:29.318177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490be829-27cb-46b1-8c1d-9636e70971e8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:29.319541 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:29.325225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e435f-73af-47e7-982c-9ab0a9d13e07 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:29.325671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e435f-73af-47e7-982c-9ab0a9d13e07 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:29.325792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e435f-73af-47e7-982c-9ab0a9d13e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:29.394556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-368e435f-73af-47e7-982c-9ab0a9d13e07 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:29.395784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-368e435f-73af-47e7-982c-9ab0a9d13e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:29.435695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:29.498487 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:29.848969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490be829-27cb-46b1-8c1d-9636e70971e8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:29.850825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-490be829-27cb-46b1-8c1d-9636e70971e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:29.962154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490be829-27cb-46b1-8c1d-9636e70971e8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:30.002003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:30.039969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5dba7-41db-4142-9713-377f2d8fb6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:30.040577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5dba7-41db-4142-9713-377f2d8fb6bc service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:30.041170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5dba7-41db-4142-9713-377f2d8fb6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:30.041434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5dba7-41db-4142-9713-377f2d8fb6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:30.043367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3af5dba7-41db-4142-9713-377f2d8fb6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:30.162332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5dba7-41db-4142-9713-377f2d8fb6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:30.163843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5dba7-41db-4142-9713-377f2d8fb6bc service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:30.183901 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 721/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:32.727424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2564a-9757-4e84-8635-f8252d9fac0b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:32.728007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2564a-9757-4e84-8635-f8252d9fac0b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:32.728007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2564a-9757-4e84-8635-f8252d9fac0b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:32.729343 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:32.734873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12916-c6dd-46a3-92a5-4880b60d72e4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:32.735411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12916-c6dd-46a3-92a5-4880b60d72e4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:32.735518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12916-c6dd-46a3-92a5-4880b60d72e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:32.804542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c4a12916-c6dd-46a3-92a5-4880b60d72e4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:32.805839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c4a12916-c6dd-46a3-92a5-4880b60d72e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:32.825633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:32.908265 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:33.248527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2564a-9757-4e84-8635-f8252d9fac0b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:33.250082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8a2564a-9757-4e84-8635-f8252d9fac0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:33.363256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2564a-9757-4e84-8635-f8252d9fac0b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:33.433744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:33.441173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e4788-bf4f-4d60-b428-196cef7b9f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:33.441912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4788-bf4f-4d60-b428-196cef7b9f5e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:33.442186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4788-bf4f-4d60-b428-196cef7b9f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:33.442457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4788-bf4f-4d60-b428-196cef7b9f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:33.444078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5e7e4788-bf4f-4d60-b428-196cef7b9f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:33.559410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e4788-bf4f-4d60-b428-196cef7b9f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:33.560899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e4788-bf4f-4d60-b428-196cef7b9f5e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:33.579241 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 745/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:36.027220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff672327-c1a5-4db2-9fd4-cf23f48b2abc None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:36.027823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff672327-c1a5-4db2-9fd4-cf23f48b2abc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:36.027823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff672327-c1a5-4db2-9fd4-cf23f48b2abc None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:36.029146 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:48:36.035424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f67fef-4ba6-46a5-af7b-ddfbb3568a19 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:36.035888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f67fef-4ba6-46a5-af7b-ddfbb3568a19 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:36.035998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f67fef-4ba6-46a5-af7b-ddfbb3568a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:36.106792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f8f67fef-4ba6-46a5-af7b-ddfbb3568a19 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:48:36.108024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8f67fef-4ba6-46a5-af7b-ddfbb3568a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:36.164547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:36.219503 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:48:36.553528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff672327-c1a5-4db2-9fd4-cf23f48b2abc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:48:36.555150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff672327-c1a5-4db2-9fd4-cf23f48b2abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:36.679555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff672327-c1a5-4db2-9fd4-cf23f48b2abc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:48:36.742498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:36.770610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a486a-b67b-4cc4-8521-f41c0b7c9919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:48:36.771224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a486a-b67b-4cc4-8521-f41c0b7c9919 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:48:36.771499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a486a-b67b-4cc4-8521-f41c0b7c9919 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:48:36.771756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a486a-b67b-4cc4-8521-f41c0b7c9919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:48:36.773408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fc9a486a-b67b-4cc4-8521-f41c0b7c9919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:48:36.892675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a486a-b67b-4cc4-8521-f41c0b7c9919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:48:36.894403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a486a-b67b-4cc4-8521-f41c0b7c9919 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:48:36.918219 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 724/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:48:46.217954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=22 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:46.915488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=66 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:01.893121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b72058-7018-4944-8246-68b11822a511 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:01.894486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b72058-7018-4944-8246-68b11822a511 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:01.894486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b72058-7018-4944-8246-68b11822a511 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:01.895918 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 747/1471] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:49:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:01.905161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b788-08b2-4e92-9614-333ca2f4d1e4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:01.906660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b788-08b2-4e92-9614-333ca2f4d1e4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:01.906660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b788-08b2-4e92-9614-333ca2f4d1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:01.989910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4827b788-08b2-4e92-9614-333ca2f4d1e4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:01.991149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4827b788-08b2-4e92-9614-333ca2f4d1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:02.125899 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 725/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:49:02.591610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199b4d-85fd-4141-91f1-5006ae145cff None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:02.591888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199b4d-85fd-4141-91f1-5006ae145cff None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:02.592156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199b4d-85fd-4141-91f1-5006ae145cff None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:02.593940 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 748/1473] 10.222.0.26 () {54 vars in 962 bytes} [Tue Sep 27 09:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:02.600160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d30-cfad-4577-83e2-88911a2e96e4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:02.600447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d30-cfad-4577-83e2-88911a2e96e4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:02.600822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d30-cfad-4577-83e2-88911a2e96e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:02.697739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dda77d30-cfad-4577-83e2-88911a2e96e4 None None] MFA Rules not processed for user `ed91b2c83d57419dac50595f57fc3411`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:02.699460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dda77d30-cfad-4577-83e2-88911a2e96e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:02.727274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:02.797587 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 726/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 27 09:49:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:49:02.805969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0199b4d-85fd-4141-91f1-5006ae145cff None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:49:02.807611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d0199b4d-85fd-4141-91f1-5006ae145cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:02.927346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0199b4d-85fd-4141-91f1-5006ae145cff None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:49:03.009018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e34881-a13e-4f4e-b809-9fdcb868de12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:49:03.009618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34881-a13e-4f4e-b809-9fdcb868de12 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:03.009889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34881-a13e-4f4e-b809-9fdcb868de12 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:03.010154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34881-a13e-4f4e-b809-9fdcb868de12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:03.011759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-51e34881-a13e-4f4e-b809-9fdcb868de12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:03.069939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:03.145357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e34881-a13e-4f4e-b809-9fdcb868de12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:49:03.147092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e34881-a13e-4f4e-b809-9fdcb868de12 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:49:03.168770 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 749/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:49:05.745085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad2e2-6d8c-4004-8391-9f96c6880bcd None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:05.745957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad2e2-6d8c-4004-8391-9f96c6880bcd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:05.745957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad2e2-6d8c-4004-8391-9f96c6880bcd None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:05.747896 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 727/1476] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:49:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:05.756868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d2638-bca1-4be7-babd-399c65ecc887 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:05.757746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d2638-bca1-4be7-babd-399c65ecc887 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:05.757746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d2638-bca1-4be7-babd-399c65ecc887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:05.831682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-906d2638-bca1-4be7-babd-399c65ecc887 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:05.832938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-906d2638-bca1-4be7-babd-399c65ecc887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:05.938512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 750/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:49:06.265874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ad2e2-6d8c-4004-8391-9f96c6880bcd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:49:06.268089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-118ad2e2-6d8c-4004-8391-9f96c6880bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:06.409162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ad2e2-6d8c-4004-8391-9f96c6880bcd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:49:06.464190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:06.519287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602a951-3823-4486-a36a-c144d0fb8121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:49:06.519790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602a951-3823-4486-a36a-c144d0fb8121 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:06.520060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602a951-3823-4486-a36a-c144d0fb8121 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:06.520326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602a951-3823-4486-a36a-c144d0fb8121 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:06.520991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602a951-3823-4486-a36a-c144d0fb8121 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:49:06.522738 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-0602a951-3823-4486-a36a-c144d0fb8121 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:49:06.522738 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:49:06.522738 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:49:06.523866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602a951-3823-4486-a36a-c144d0fb8121 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:49:06.543790 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 728/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Sep 27 09:49:06 2022] GET /identity/v3/projects => generated 2553 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:49:09.032953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f9d0a-875a-45f0-a253-b5b4f41e423f None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:09.033504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f9d0a-875a-45f0-a253-b5b4f41e423f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:09.033504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f9d0a-875a-45f0-a253-b5b4f41e423f None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:09.034852 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 751/1479] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:09.040440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcd5d7-6e49-4e58-a436-dcf80ef52383 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:09.040918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcd5d7-6e49-4e58-a436-dcf80ef52383 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:09.041034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcd5d7-6e49-4e58-a436-dcf80ef52383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:09.113518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ebdcd5d7-6e49-4e58-a436-dcf80ef52383 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:09.114746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ebdcd5d7-6e49-4e58-a436-dcf80ef52383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:09.221129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 729/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:49:09.575745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f9d0a-875a-45f0-a253-b5b4f41e423f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:49:09.577470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3d1f9d0a-875a-45f0-a253-b5b4f41e423f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:09.588872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:09.744258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f9d0a-875a-45f0-a253-b5b4f41e423f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:49:09.825640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:09.859576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c51f9-ca78-4a3d-84b1-b0be4a5bebd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:49:09.860183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c51f9-ca78-4a3d-84b1-b0be4a5bebd8 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:09.860456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c51f9-ca78-4a3d-84b1-b0be4a5bebd8 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:09.860946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c51f9-ca78-4a3d-84b1-b0be4a5bebd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:09.862881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-590c51f9-ca78-4a3d-84b1-b0be4a5bebd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:09.965931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c51f9-ca78-4a3d-84b1-b0be4a5bebd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:49:09.967533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c51f9-ca78-4a3d-84b1-b0be4a5bebd8 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:49:09.987352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 752/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:49:12.711016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1667-6f30-4368-8da1-dc3c58caae50 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:12.711557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1667-6f30-4368-8da1-dc3c58caae50 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:12.711557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1667-6f30-4368-8da1-dc3c58caae50 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:12.713154 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 730/1482] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:12.720003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de03a3c-a78b-4054-9410-3cdf53e22907 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:12.720747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de03a3c-a78b-4054-9410-3cdf53e22907 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:12.720972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de03a3c-a78b-4054-9410-3cdf53e22907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:12.796010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8de03a3c-a78b-4054-9410-3cdf53e22907 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:12.797270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8de03a3c-a78b-4054-9410-3cdf53e22907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:12.853418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:12.912963 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 753/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:49:13.303829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f1667-6f30-4368-8da1-dc3c58caae50 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:49:13.306460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a74f1667-6f30-4368-8da1-dc3c58caae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:13.344999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:13.464264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f1667-6f30-4368-8da1-dc3c58caae50 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:49:13.555726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdf9df-f22b-4c7b-b0f4-ccee4f42f445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:49:13.556344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdf9df-f22b-4c7b-b0f4-ccee4f42f445 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:13.556707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdf9df-f22b-4c7b-b0f4-ccee4f42f445 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:13.556925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdf9df-f22b-4c7b-b0f4-ccee4f42f445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:13.558574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4cbdf9df-f22b-4c7b-b0f4-ccee4f42f445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:13.642138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:13.681127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdf9df-f22b-4c7b-b0f4-ccee4f42f445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:49:13.682574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdf9df-f22b-4c7b-b0f4-ccee4f42f445 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:49:13.701893 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 731/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:49:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:49:16.333178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34b1d1-5b45-4cfb-8dfd-72950351032d None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:16.334082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34b1d1-5b45-4cfb-8dfd-72950351032d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:16.334082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34b1d1-5b45-4cfb-8dfd-72950351032d None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:16.336135 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 754/1485] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:49:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:16.342172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e44c9-0426-4b23-9c9b-1a89f0c6daa7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:16.342598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e44c9-0426-4b23-9c9b-1a89f0c6daa7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:16.342745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e44c9-0426-4b23-9c9b-1a89f0c6daa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:16.432651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dc2e44c9-0426-4b23-9c9b-1a89f0c6daa7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:16.434544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dc2e44c9-0426-4b23-9c9b-1a89f0c6daa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:16.546062 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 732/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:49:16.932020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34b1d1-5b45-4cfb-8dfd-72950351032d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:49:16.934567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a34b1d1-5b45-4cfb-8dfd-72950351032d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:17.074464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34b1d1-5b45-4cfb-8dfd-72950351032d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:49:17.164905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:17.167340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb0a0f-b179-46b4-98b8-590bce8fa3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:49:17.168697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb0a0f-b179-46b4-98b8-590bce8fa3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:17.168979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb0a0f-b179-46b4-98b8-590bce8fa3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:17.169725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb0a0f-b179-46b4-98b8-590bce8fa3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:17.172675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-25fb0a0f-b179-46b4-98b8-590bce8fa3f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:17.307953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb0a0f-b179-46b4-98b8-590bce8fa3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:49:17.309556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb0a0f-b179-46b4-98b8-590bce8fa3f2 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:49:17.332371 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 755/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:49:19.998005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ad54a-82b7-4585-a19e-8c1a79c4e8d1 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:19.998597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ad54a-82b7-4585-a19e-8c1a79c4e8d1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:19.998597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ad54a-82b7-4585-a19e-8c1a79c4e8d1 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:19.999789 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 733/1488] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:20.006576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1d3d-d203-4015-abb6-a5439a0b42ca None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:20.007339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1d3d-d203-4015-abb6-a5439a0b42ca None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:20.007531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d1d3d-d203-4015-abb6-a5439a0b42ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:20.085981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-408d1d3d-d203-4015-abb6-a5439a0b42ca None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:20.087197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-408d1d3d-d203-4015-abb6-a5439a0b42ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:20.144114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:20.191756 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 756/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:49:20.541137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ad54a-82b7-4585-a19e-8c1a79c4e8d1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:49:20.543877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-046ad54a-82b7-4585-a19e-8c1a79c4e8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:20.587502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:20.662117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ad54a-82b7-4585-a19e-8c1a79c4e8d1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:49:20.753071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b627e-1d81-466c-8078-bb3219212b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:49:20.753673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b627e-1d81-466c-8078-bb3219212b9a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:20.753957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b627e-1d81-466c-8078-bb3219212b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:20.754231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b627e-1d81-466c-8078-bb3219212b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:20.755854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-df9b627e-1d81-466c-8078-bb3219212b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:20.848732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:20.889118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b627e-1d81-466c-8078-bb3219212b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:49:20.890571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b627e-1d81-466c-8078-bb3219212b9a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:49:20.910470 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 734/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:49:23.579318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344e329-ab57-4b80-b431-9ab161efad2e None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:23.579920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344e329-ab57-4b80-b431-9ab161efad2e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:23.579920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344e329-ab57-4b80-b431-9ab161efad2e None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:23.581508 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 757/1491] 10.222.0.26 () {54 vars in 927 bytes} [Tue Sep 27 09:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:49:23.587534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317350c-bec3-4f79-8bfe-457eb4a95a58 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:23.588005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317350c-bec3-4f79-8bfe-457eb4a95a58 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:23.588136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317350c-bec3-4f79-8bfe-457eb4a95a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:23.660430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f317350c-bec3-4f79-8bfe-457eb4a95a58 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:49:23.661680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f317350c-bec3-4f79-8bfe-457eb4a95a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:23.764810 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 735/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Sep 27 09:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:49:24.107017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344e329-ab57-4b80-b431-9ab161efad2e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:49:24.109579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2344e329-ab57-4b80-b431-9ab161efad2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:24.234661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344e329-ab57-4b80-b431-9ab161efad2e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:49:24.310938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e6dc-9402-454c-94ff-9d425e12fc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:49:24.311598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e6dc-9402-454c-94ff-9d425e12fc73 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:49:24.311807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e6dc-9402-454c-94ff-9d425e12fc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:49:24.312066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e6dc-9402-454c-94ff-9d425e12fc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:49:24.313676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4141e6dc-9402-454c-94ff-9d425e12fc73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:49:24.323568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:24.434704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e6dc-9402-454c-94ff-9d425e12fc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:49:24.436085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e6dc-9402-454c-94ff-9d425e12fc73 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:49:24.455208 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 758/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:49:33.763284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=82 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:34.452680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=50 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:50:40.780675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82e142-1f00-4932-bba1-729926c24bf2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:40.781291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82e142-1f00-4932-bba1-729926c24bf2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:40.781291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82e142-1f00-4932-bba1-729926c24bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:40.882739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ba82e142-1f00-4932-bba1-729926c24bf2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:50:40.883893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ba82e142-1f00-4932-bba1-729926c24bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:50:41.004968 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 736/1494] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:50:41.013560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdbc68-c5ae-4adb-971f-6af90379d260 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.014089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdbc68-c5ae-4adb-971f-6af90379d260 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.014089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdbc68-c5ae-4adb-971f-6af90379d260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.035483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-18bdbc68-c5ae-4adb-971f-6af90379d260 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:50:41.036686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18bdbc68-c5ae-4adb-971f-6af90379d260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:50:41.047064 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 759/1495] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:50:41.053393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82e142-1f00-4932-bba1-729926c24bf2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.063975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82e142-1f00-4932-bba1-729926c24bf2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.069331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.069931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.070205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.070473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.071125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:50:41.071426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.072557 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:41.072557 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:50:41.072557 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:50:41.073486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.086217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 737/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:50:41.095147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdbc68-c5ae-4adb-971f-6af90379d260 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.107272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdbc68-c5ae-4adb-971f-6af90379d260 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.113252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.113851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.114129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.114379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.114998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.117075 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:41.117075 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 09:50:41.117075 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 09:50:41.118276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.164469 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 760/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:50:41 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:50:41.168868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.180760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9d91e-978f-4f2a-906f-67c208dc47ab admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.244644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:50:41.252501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.253165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.253438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.253711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.254349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.255393 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:41.255393 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:50:41.255393 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:50:41.257372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.281285 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 738/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:50:41 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:50:41.287695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.301366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0ecea-0714-45ee-b360-7f7b3a5b662f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.306838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.307446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.307875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.308143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.308834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.309854 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:41.309854 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 09:50:41.309854 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 09:50:41.310741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.324777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 761/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:50:41.329259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.340883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571fd3a-d829-4ce0-93b4-675063373e7a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.348305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c9815-7d6d-4e92-afc2-1f61d4165723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.349150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c9815-7d6d-4e92-afc2-1f61d4165723 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.349409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c9815-7d6d-4e92-afc2-1f61d4165723 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.349769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c9815-7d6d-4e92-afc2-1f61d4165723 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.350410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c9815-7d6d-4e92-afc2-1f61d4165723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.351528 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-147c9815-7d6d-4e92-afc2-1f61d4165723 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:41.351528 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:50:41.351528 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:50:41.352527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c9815-7d6d-4e92-afc2-1f61d4165723 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.370746 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 739/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:50:41.375261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.386579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d65de0-2b90-48ae-9a45-172f62479621 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.391908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a666ca-8738-47b6-bb4c-c724fab6f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.392587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a666ca-8738-47b6-bb4c-c724fab6f89d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.392901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a666ca-8738-47b6-bb4c-c724fab6f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.393173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a666ca-8738-47b6-bb4c-c724fab6f89d admin admin] PATH_INFO: `/v3/projects/a88dc3718f78465aab80b4a51099169a/users/f8d0ac715c2147dea6e7acd81a24f137/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.409454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a666ca-8738-47b6-bb4c-c724fab6f89d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88dc3718f78465aab80b4a51099169a, user_id=f8d0ac715c2147dea6e7acd81a24f137, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.410487 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-a0a666ca-8738-47b6-bb4c-c724fab6f89d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:41.410487 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 09:50:41.410487 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 09:50:41.412447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a666ca-8738-47b6-bb4c-c724fab6f89d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.431405 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 762/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:50:41 2022] PUT /identity/v3/projects/a88dc3718f78465aab80b4a51099169a/users/f8d0ac715c2147dea6e7acd81a24f137/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:50:41.444593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e9f4-07e0-480b-9baa-d4d7291d7aee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.444904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e9f4-07e0-480b-9baa-d4d7291d7aee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.445205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e9f4-07e0-480b-9baa-d4d7291d7aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.478082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a5f5e9f4-07e0-480b-9baa-d4d7291d7aee None None] MFA Rules not processed for user `f8d0ac715c2147dea6e7acd81a24f137`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:50:41.479315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a5f5e9f4-07e0-480b-9baa-d4d7291d7aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:50:41.553464 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 740/1502] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:50:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:50:41.567606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0605cb-359a-4a49-917b-2fd62cb14abe None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.567950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0605cb-359a-4a49-917b-2fd62cb14abe None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.568208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0605cb-359a-4a49-917b-2fd62cb14abe None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.569777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 763/1503] 10.222.0.26 () {54 vars in 963 bytes} [Tue Sep 27 09:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:50:41.575000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e682b-c5a2-4f6e-a6d9-d553d4891d83 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.575281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e682b-c5a2-4f6e-a6d9-d553d4891d83 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.575578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e682b-c5a2-4f6e-a6d9-d553d4891d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.640234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bf9e682b-c5a2-4f6e-a6d9-d553d4891d83 None None] MFA Rules not processed for user `b324af05ec3e4b35a9bfc188b3b3abec`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:50:41.641476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bf9e682b-c5a2-4f6e-a6d9-d553d4891d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:50:41.721425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 741/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Tue Sep 27 09:50:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:50:41.729516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0605cb-359a-4a49-917b-2fd62cb14abe None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.740495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0605cb-359a-4a49-917b-2fd62cb14abe None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.744332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.744953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.745219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.745480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.754247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.755675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.765405 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 764/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:50:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:50:41.823566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1b7a-e8c6-42cb-be00-a08ac7e18de6 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.824063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1b7a-e8c6-42cb-be00-a08ac7e18de6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.824207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1b7a-e8c6-42cb-be00-a08ac7e18de6 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.825422 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 742/1506] 10.222.0.26 () {52 vars in 830 bytes} [Tue Sep 27 09:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:50:41.830875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:41.905383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594541d0-bede-4324-bda5-28690b187c0d service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:41.910592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:41.911215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:41.911486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:41.911752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] PATH_INFO: `/v3/users/f8d0ac715c2147dea6e7acd81a24f137` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:41.913267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d0ac715c2147dea6e7acd81a24f137)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:41.914287 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:41.914287 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 09:50:41.914287 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 09:50:41.915484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:41.990746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d0ac715c2147dea6e7acd81a24f137'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:41.991476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d0ac715c2147dea6e7acd81a24f137'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.006176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d0ac715c2147dea6e7acd81a24f137'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.022955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d0ac715c2147dea6e7acd81a24f137'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.023852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d0ac715c2147dea6e7acd81a24f137'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.046939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:50:42.057152 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 765/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:50:41 2022] DELETE /identity/v3/users/f8d0ac715c2147dea6e7acd81a24f137 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:50:42.061661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:42.061934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:42.062202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:42.081750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:50:42.082961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:50:42.093230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:50:42.155076 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 743/1508] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:50:42.165962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:42.167994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:50:42.255375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9cf5c-40c3-4eb1-a7b9-945138720403 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:42.261514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089d57b-783a-45fb-8df3-360219e46c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:42.262485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d57b-783a-45fb-8df3-360219e46c0a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:42.262927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d57b-783a-45fb-8df3-360219e46c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:42.263333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d57b-783a-45fb-8df3-360219e46c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:42.278217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089d57b-783a-45fb-8df3-360219e46c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:42.279712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089d57b-783a-45fb-8df3-360219e46c0a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:42.290560 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 766/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:50:42.658086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:50:42.659925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:50:42.672969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5a29e-46e1-42da-add4-a39e2e6ac3d3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:50:42.678346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:50:42.678346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:50:42.678584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:50:42.678957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] PATH_INFO: `/v3/projects/a88dc3718f78465aab80b4a51099169a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:50:42.680350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a88dc3718f78465aab80b4a51099169a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:50:42.681401 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:50:42.681401 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:50:42.681401 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:50:42.682623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:50:42.751066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a88dc3718f78465aab80b4a51099169a', 'request_id': 'req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.765588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a88dc3718f78465aab80b4a51099169a', 'request_id': 'req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.777370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a88dc3718f78465aab80b4a51099169a', 'request_id': 'req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.791597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] The token cache is being invalidate because project a88dc3718f78465aab80b4a51099169a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:50:42.791882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.793122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb7e5d5f-2f83-4c1c-9257-7110cb4efb64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:50:42.794199 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 744/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:50:42 2022] DELETE /identity/v3/projects/a88dc3718f78465aab80b4a51099169a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:50:52.285566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:50:52.787375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:01.843173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50f1f-b9ce-450e-937a-4bbf35c9f7d2 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:01.843813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50f1f-b9ce-450e-937a-4bbf35c9f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:01.843813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50f1f-b9ce-450e-937a-4bbf35c9f7d2 None None] PATH_INFO: `/v3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:01.845024 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 767/1511] 10.222.0.16 () {54 vars in 925 bytes} [Tue Sep 27 09:54:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:54:01.853059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd9ad0-dfde-4c33-8baa-f5bf140893b6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:01.853798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd9ad0-dfde-4c33-8baa-f5bf140893b6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:01.853907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd9ad0-dfde-4c33-8baa-f5bf140893b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:01.917056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-77fd9ad0-dfde-4c33-8baa-f5bf140893b6 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:01.918735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-77fd9ad0-dfde-4c33-8baa-f5bf140893b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:02.002784 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 745/1512] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Sep 27 09:54:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:54:05.779934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a86753-ab43-4e4c-adc4-440bdbd124af None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:05.780542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a86753-ab43-4e4c-adc4-440bdbd124af None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:05.780542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a86753-ab43-4e4c-adc4-440bdbd124af None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:05.781890 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 768/1513] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 27 09:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:54:05.788245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061ed1-b64c-4de9-8d11-11255849ea63 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:05.788901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061ed1-b64c-4de9-8d11-11255849ea63 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:05.788901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061ed1-b64c-4de9-8d11-11255849ea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:05.810841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-82061ed1-b64c-4de9-8d11-11255849ea63 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:05.812127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-82061ed1-b64c-4de9-8d11-11255849ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:05.825489 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 746/1514] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 09:54:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:54:05.834664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a86753-ab43-4e4c-adc4-440bdbd124af None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:54:05.850619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a86753-ab43-4e4c-adc4-440bdbd124af None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:54:05.856951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca25b9-ea7e-4f15-8499-4511ced0eaa4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:54:05.857491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca25b9-ea7e-4f15-8499-4511ced0eaa4 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:05.857770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca25b9-ea7e-4f15-8499-4511ced0eaa4 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:05.858046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca25b9-ea7e-4f15-8499-4511ced0eaa4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:05.868527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca25b9-ea7e-4f15-8499-4511ced0eaa4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:54:05.870161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca25b9-ea7e-4f15-8499-4511ced0eaa4 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:54:05.881934 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 769/1515] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:54:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:54:15.805926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=47 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:15.877281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:19.099123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df92761-d454-4630-84d2-5724f41f481c None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:19.099743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df92761-d454-4630-84d2-5724f41f481c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:19.099743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df92761-d454-4630-84d2-5724f41f481c None None] PATH_INFO: `/v3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:19.101158 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 747/1516] 10.222.0.55 () {54 vars in 925 bytes} [Tue Sep 27 09:54:19 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:54:19.107562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca41f4-bcd5-434a-a6ff-4068047e1101 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:19.108143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca41f4-bcd5-434a-a6ff-4068047e1101 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:19.108263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca41f4-bcd5-434a-a6ff-4068047e1101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:19.131405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-06ca41f4-bcd5-434a-a6ff-4068047e1101 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:19.132553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-06ca41f4-bcd5-434a-a6ff-4068047e1101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:19.143392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 770/1517] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Sep 27 09:54:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:54:28.088417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4383fe-23df-4c27-88f3-f6acfcd839b8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:28.088912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4383fe-23df-4c27-88f3-f6acfcd839b8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:28.089025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4383fe-23df-4c27-88f3-f6acfcd839b8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:28.090334 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 748/1518] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 27 09:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:54:28.095379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b7cdb-4329-4f02-900c-246e26fa9ef4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:28.095825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b7cdb-4329-4f02-900c-246e26fa9ef4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:28.095939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b7cdb-4329-4f02-900c-246e26fa9ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:28.116848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fa4b7cdb-4329-4f02-900c-246e26fa9ef4 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:28.118037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fa4b7cdb-4329-4f02-900c-246e26fa9ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:28.126946 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 771/1519] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 09:54:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:54:28.134629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4383fe-23df-4c27-88f3-f6acfcd839b8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:54:28.149858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4383fe-23df-4c27-88f3-f6acfcd839b8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:54:28.155972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b79a9-be12-4c82-8276-9c9aa413104b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:54:28.156618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b79a9-be12-4c82-8276-9c9aa413104b service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:28.156886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b79a9-be12-4c82-8276-9c9aa413104b service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:28.157172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b79a9-be12-4c82-8276-9c9aa413104b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:28.166474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b79a9-be12-4c82-8276-9c9aa413104b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:54:28.168061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b79a9-be12-4c82-8276-9c9aa413104b service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:54:28.178710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 749/1520] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:54:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:54:38.111248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:38.174036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:45.464717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d60d0-4a4a-4d93-a7f4-a1b09245443c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:45.465324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d60d0-4a4a-4d93-a7f4-a1b09245443c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:45.465324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d60d0-4a4a-4d93-a7f4-a1b09245443c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:45.466600 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 772/1521] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:54:45.472180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af441-f445-43a8-ba4b-fe96c79c3139 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:45.472959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af441-f445-43a8-ba4b-fe96c79c3139 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:45.473139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af441-f445-43a8-ba4b-fe96c79c3139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:45.500069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-219af441-f445-43a8-ba4b-fe96c79c3139 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:45.501288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-219af441-f445-43a8-ba4b-fe96c79c3139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:45.583543 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 750/1522] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:54:45.751157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d60d0-4a4a-4d93-a7f4-a1b09245443c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:54:45.753225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c88d60d0-4a4a-4d93-a7f4-a1b09245443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:45.872935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d60d0-4a4a-4d93-a7f4-a1b09245443c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:54:45.879175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040735d-321b-4763-b487-0a68857659df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:54:45.879896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040735d-321b-4763-b487-0a68857659df service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:45.880178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040735d-321b-4763-b487-0a68857659df service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:45.880894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040735d-321b-4763-b487-0a68857659df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:45.893134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040735d-321b-4763-b487-0a68857659df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:54:45.894753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040735d-321b-4763-b487-0a68857659df service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:54:45.905919 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 773/1523] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:54:47.579848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3b2a5-f7b5-4b93-a62f-84e1d5b262bc None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:47.580517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3b2a5-f7b5-4b93-a62f-84e1d5b262bc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:47.580517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3b2a5-f7b5-4b93-a62f-84e1d5b262bc None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:47.581769 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 751/1524] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 09:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:54:47.588462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a168629-4574-4f7a-a928-294962501dc2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:47.588960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a168629-4574-4f7a-a928-294962501dc2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:47.589076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a168629-4574-4f7a-a928-294962501dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:47.611945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3a168629-4574-4f7a-a928-294962501dc2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:47.613205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3a168629-4574-4f7a-a928-294962501dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:47.624245 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 774/1525] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 09:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:54:47.796549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3b2a5-f7b5-4b93-a62f-84e1d5b262bc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:54:47.814719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3b2a5-f7b5-4b93-a62f-84e1d5b262bc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:54:47.821996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545007d1-71d4-4a6f-85b5-81e1941056ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:54:47.822636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545007d1-71d4-4a6f-85b5-81e1941056ad service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:47.822903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545007d1-71d4-4a6f-85b5-81e1941056ad service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:47.823236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545007d1-71d4-4a6f-85b5-81e1941056ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:47.832534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545007d1-71d4-4a6f-85b5-81e1941056ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:54:47.834403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545007d1-71d4-4a6f-85b5-81e1941056ad service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:54:47.845467 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 752/1526] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:54:52.281572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca90441-8d40-47fb-be23-4bb14ff17ff4 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:52.282232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca90441-8d40-47fb-be23-4bb14ff17ff4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:52.282232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca90441-8d40-47fb-be23-4bb14ff17ff4 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:52.283992 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 775/1527] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:54:52.291664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e00b-09ac-403d-99ca-5db4aee61a19 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:52.292143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e00b-09ac-403d-99ca-5db4aee61a19 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:52.292143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e00b-09ac-403d-99ca-5db4aee61a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:52.315347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6bc3e00b-09ac-403d-99ca-5db4aee61a19 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:52.316573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6bc3e00b-09ac-403d-99ca-5db4aee61a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:52.325876 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 753/1528] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:54:52.674779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca90441-8d40-47fb-be23-4bb14ff17ff4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:54:52.676545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eca90441-8d40-47fb-be23-4bb14ff17ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:52.739796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca90441-8d40-47fb-be23-4bb14ff17ff4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:54:52.744300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c04e1d-2683-47a2-b4e9-943c9725333d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:54:52.744951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04e1d-2683-47a2-b4e9-943c9725333d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:52.745219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04e1d-2683-47a2-b4e9-943c9725333d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:52.745487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04e1d-2683-47a2-b4e9-943c9725333d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:52.755971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c04e1d-2683-47a2-b4e9-943c9725333d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:54:52.757467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c04e1d-2683-47a2-b4e9-943c9725333d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:54:52.768561 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 776/1529] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:54:55.225158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848656c1-590b-4a19-b88a-042e696d6ed7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:55.225711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848656c1-590b-4a19-b88a-042e696d6ed7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:55.225711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848656c1-590b-4a19-b88a-042e696d6ed7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:55.226974 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 754/1530] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 09:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:54:55.232299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce30d-71e2-44b0-a554-9dfbf0e1fe38 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:55.232765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce30d-71e2-44b0-a554-9dfbf0e1fe38 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:55.232909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce30d-71e2-44b0-a554-9dfbf0e1fe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:55.258280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-333ce30d-71e2-44b0-a554-9dfbf0e1fe38 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:54:55.259501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-333ce30d-71e2-44b0-a554-9dfbf0e1fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:55.268796 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 777/1531] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 09:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:54:55.610082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848656c1-590b-4a19-b88a-042e696d6ed7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:54:55.613071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-848656c1-590b-4a19-b88a-042e696d6ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:54:55.628854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848656c1-590b-4a19-b88a-042e696d6ed7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:54:55.632691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbef48-4b2a-41a0-841b-6129a7f1c0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:54:55.633326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbef48-4b2a-41a0-841b-6129a7f1c0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:54:55.633602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbef48-4b2a-41a0-841b-6129a7f1c0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:54:55.633869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbef48-4b2a-41a0-841b-6129a7f1c0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:54:55.642807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbef48-4b2a-41a0-841b-6129a7f1c0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:54:55.644232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbef48-4b2a-41a0-841b-6129a7f1c0c4 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:54:55.654225 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 755/1532] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:54:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:55:05.250509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=78 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:55:05.650038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=42 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:55:47.126309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe22fd4-52e8-49d3-8e02-dbcb68ebeb72 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:55:47.126878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe22fd4-52e8-49d3-8e02-dbcb68ebeb72 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:55:47.126878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe22fd4-52e8-49d3-8e02-dbcb68ebeb72 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:55:47.128423 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 778/1533] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:55:47.133835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210943-d98d-4e00-99c8-19e817ca6fcf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:55:47.134270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210943-d98d-4e00-99c8-19e817ca6fcf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:55:47.134487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210943-d98d-4e00-99c8-19e817ca6fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:55:47.155244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bb210943-d98d-4e00-99c8-19e817ca6fcf None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:55:47.156444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bb210943-d98d-4e00-99c8-19e817ca6fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:55:47.165821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 756/1534] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:55:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:55:57.149855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:20.576159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca35c5-05a6-4611-821e-3c2c88ee004d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:20.576774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca35c5-05a6-4611-821e-3c2c88ee004d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:20.576774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca35c5-05a6-4611-821e-3c2c88ee004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:20.619575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3dca35c5-05a6-4611-821e-3c2c88ee004d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:20.621200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3dca35c5-05a6-4611-821e-3c2c88ee004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:20.635445 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 779/1535] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:20.645392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2810d2-7c62-41ff-92a1-8059bec9a384 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:20.645965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2810d2-7c62-41ff-92a1-8059bec9a384 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:20.645965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2810d2-7c62-41ff-92a1-8059bec9a384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:20.682917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5c2810d2-7c62-41ff-92a1-8059bec9a384 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:20.684149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5c2810d2-7c62-41ff-92a1-8059bec9a384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:20.700469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca35c5-05a6-4611-821e-3c2c88ee004d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:20.701171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 757/1536] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:20.719170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca35c5-05a6-4611-821e-3c2c88ee004d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:20.724454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:20.725477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:20.726051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:20.727157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:20.728244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:20.728775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:20.730022 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:56:20.730022 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 09:56:20.730022 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 09:56:20.731264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:20.757734 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 780/1537] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:20.784607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2810d2-7c62-41ff-92a1-8059bec9a384 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:20.801607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2810d2-7c62-41ff-92a1-8059bec9a384 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:20.825747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:20.826809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:20.827037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:20.827297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:20.828022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:20.829076 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:56:20.829076 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:56:20.829076 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:56:20.830282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:20.878442 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 758/1538] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:20 2022] POST /identity/v3/projects => generated 397 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:20.883076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:20.895934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719892ed-1700-409e-96e9-f1aca19174eb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:20.972286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:20.972933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:20.973210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:20.973478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:20.974132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:20.975192 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:56:20.975192 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 09:56:20.975192 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 09:56:20.976499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.012546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 781/1539] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:20 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:21.016781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.028701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf60d1-669c-4bca-be28-36e39ae2cee3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.039212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.039865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.040421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.040775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.041440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.043235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.055930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 759/1540] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:21.060448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.071670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1787f1-ac99-4a99-9b7e-a53270152577 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.080813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912ddb2-2c40-450a-9663-2a88391b1ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.081661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912ddb2-2c40-450a-9663-2a88391b1ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.081941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912ddb2-2c40-450a-9663-2a88391b1ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.082207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912ddb2-2c40-450a-9663-2a88391b1ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.082861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912ddb2-2c40-450a-9663-2a88391b1ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.088713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912ddb2-2c40-450a-9663-2a88391b1ba8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.104289 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 782/1541] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:21.109169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.120381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9290897-fde0-458c-b827-0e65ad96d3c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.131798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.132620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.132961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.133237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] PATH_INFO: `/v3/projects/5a71603028ff42b4847878e7ede6aebe/users/8ea0f9af73d5435088120d793a61bef3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.162332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a71603028ff42b4847878e7ede6aebe, user_id=8ea0f9af73d5435088120d793a61bef3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.163497 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:56:21.163497 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:56:21.163497 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:56:21.165877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.191048 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 760/1542] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:21 2022] PUT /identity/v3/projects/5a71603028ff42b4847878e7ede6aebe/users/8ea0f9af73d5435088120d793a61bef3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:21.205423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f542-efb8-4c98-9989-7c7de46cdc22 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.206284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f542-efb8-4c98-9989-7c7de46cdc22 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.206284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f542-efb8-4c98-9989-7c7de46cdc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.251435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8554f542-efb8-4c98-9989-7c7de46cdc22 None None] MFA Rules not processed for user `8ea0f9af73d5435088120d793a61bef3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.252714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8554f542-efb8-4c98-9989-7c7de46cdc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.334934 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 783/1543] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.339871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.360999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1299-976c-453c-af9d-e7e29386e27a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.361428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1299-976c-453c-af9d-e7e29386e27a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.361696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1299-976c-453c-af9d-e7e29386e27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.406524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c44e1299-976c-453c-af9d-e7e29386e27a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.407738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c44e1299-976c-453c-af9d-e7e29386e27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.470929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba3ad-9dba-465b-8ba3-56131833b406 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.477601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e67a7-a4a5-43d8-ba8d-c18cf32eccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.478221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e67a7-a4a5-43d8-ba8d-c18cf32eccdb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.478552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e67a7-a4a5-43d8-ba8d-c18cf32eccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.478824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e67a7-a4a5-43d8-ba8d-c18cf32eccdb admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.479457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e67a7-a4a5-43d8-ba8d-c18cf32eccdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.480543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:21.482140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e67a7-a4a5-43d8-ba8d-c18cf32eccdb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.514248 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 784/1544] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:21.515564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de567-95ee-4222-984d-8fad4ebdf6a1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.515897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de567-95ee-4222-984d-8fad4ebdf6a1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.516177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de567-95ee-4222-984d-8fad4ebdf6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.536418 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 761/1545] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/projects => generated 397 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:21.539653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cbdbc-bcf6-47d0-8f87-2af2cd87f5e6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.539944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cbdbc-bcf6-47d0-8f87-2af2cd87f5e6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.540222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cbdbc-bcf6-47d0-8f87-2af2cd87f5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.584173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f86de567-95ee-4222-984d-8fad4ebdf6a1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.592291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f86de567-95ee-4222-984d-8fad4ebdf6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.609266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e94cbdbc-bcf6-47d0-8f87-2af2cd87f5e6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.610852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e94cbdbc-bcf6-47d0-8f87-2af2cd87f5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.701367 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 785/1546] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.702653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652b200-c119-40d8-b068-2ff1efa3f6bf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.702944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652b200-c119-40d8-b068-2ff1efa3f6bf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.703204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652b200-c119-40d8-b068-2ff1efa3f6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.718930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 762/1547] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.718930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cbdbc-bcf6-47d0-8f87-2af2cd87f5e6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.726278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7652b200-c119-40d8-b068-2ff1efa3f6bf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.727849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7652b200-c119-40d8-b068-2ff1efa3f6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.729055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cbdbc-bcf6-47d0-8f87-2af2cd87f5e6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.733664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94871e5a-3c06-4179-8861-cc551507e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.734272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871e5a-3c06-4179-8861-cc551507e797 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.734547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871e5a-3c06-4179-8861-cc551507e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.734818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871e5a-3c06-4179-8861-cc551507e797 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.735447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94871e5a-3c06-4179-8861-cc551507e797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.737139 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 786/1548] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.737432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94871e5a-3c06-4179-8861-cc551507e797 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.738439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136908-1294-4fd0-bbc0-495270e1f7bc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.738724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136908-1294-4fd0-bbc0-495270e1f7bc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.738987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136908-1294-4fd0-bbc0-495270e1f7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.760084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-61136908-1294-4fd0-bbc0-495270e1f7bc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.761294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-61136908-1294-4fd0-bbc0-495270e1f7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.763966 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 763/1549] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/users => generated 305 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:21.768297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739a12-cd69-4fd5-8939-7910e072bece None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.768566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739a12-cd69-4fd5-8939-7910e072bece None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.768850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739a12-cd69-4fd5-8939-7910e072bece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.771254 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 787/1550] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.771254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42c734-e64d-432f-9ca3-2770a1375c68 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.771426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42c734-e64d-432f-9ca3-2770a1375c68 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.771672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42c734-e64d-432f-9ca3-2770a1375c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.777872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:21.790148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c1739a12-cd69-4fd5-8939-7910e072bece None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.791476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c1739a12-cd69-4fd5-8939-7910e072bece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.797800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2e42c734-e64d-432f-9ca3-2770a1375c68 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.799071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2e42c734-e64d-432f-9ca3-2770a1375c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.800331 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 764/1551] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.801931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f8d2-f493-4ab7-971f-495f8858ab9c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.802197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f8d2-f493-4ab7-971f-495f8858ab9c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.802465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f8d2-f493-4ab7-971f-495f8858ab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.813432 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 788/1552] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.814850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcd948-9030-4fb0-adac-3329cbd53446 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.815125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcd948-9030-4fb0-adac-3329cbd53446 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.815394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcd948-9030-4fb0-adac-3329cbd53446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.829243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9317f8d2-f493-4ab7-971f-495f8858ab9c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.830542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9317f8d2-f493-4ab7-971f-495f8858ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.834984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cafcd948-9030-4fb0-adac-3329cbd53446 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.836245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cafcd948-9030-4fb0-adac-3329cbd53446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.839421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 765/1553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.840629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48a774-d25c-4037-a862-7db4fedabdff None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.840946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48a774-d25c-4037-a862-7db4fedabdff None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.841215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48a774-d25c-4037-a862-7db4fedabdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.849643 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 789/1554] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.851209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcd948-9030-4fb0-adac-3329cbd53446 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.865691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcd948-9030-4fb0-adac-3329cbd53446 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.865884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3b48a774-d25c-4037-a862-7db4fedabdff None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.865884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3b48a774-d25c-4037-a862-7db4fedabdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.868786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec9c4b-013a-43c0-af75-83b6f1364bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.869430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec9c4b-013a-43c0-af75-83b6f1364bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.869699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec9c4b-013a-43c0-af75-83b6f1364bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.869996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec9c4b-013a-43c0-af75-83b6f1364bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.870616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec9c4b-013a-43c0-af75-83b6f1364bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:21.870917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec9c4b-013a-43c0-af75-83b6f1364bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.872525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec9c4b-013a-43c0-af75-83b6f1364bc3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.873197 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 766/1555] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.876996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48a774-d25c-4037-a862-7db4fedabdff None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.883799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48a774-d25c-4037-a862-7db4fedabdff None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.888714 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 790/1556] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:21.888714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721ac7f-4969-4307-9b9b-4ce1885db798 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.888714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721ac7f-4969-4307-9b9b-4ce1885db798 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.888969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fa038-8341-451e-a62b-a302ac337a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.889065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721ac7f-4969-4307-9b9b-4ce1885db798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.889132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fa038-8341-451e-a62b-a302ac337a72 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.889376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fa038-8341-451e-a62b-a302ac337a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.889590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fa038-8341-451e-a62b-a302ac337a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.890221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fa038-8341-451e-a62b-a302ac337a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.891800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fa038-8341-451e-a62b-a302ac337a72 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.902731 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 767/1557] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:21.904157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec998856-430f-4dfe-8e6f-c7d474fcebeb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.904460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec998856-430f-4dfe-8e6f-c7d474fcebeb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.904730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec998856-430f-4dfe-8e6f-c7d474fcebeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.909533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8721ac7f-4969-4307-9b9b-4ce1885db798 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.910846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8721ac7f-4969-4307-9b9b-4ce1885db798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.919684 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 791/1558] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.919684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181e28f-1f1b-4a08-aa3c-d7afa0318c78 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.919684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181e28f-1f1b-4a08-aa3c-d7afa0318c78 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.919684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181e28f-1f1b-4a08-aa3c-d7afa0318c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.924921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ec998856-430f-4dfe-8e6f-c7d474fcebeb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.926464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec998856-430f-4dfe-8e6f-c7d474fcebeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.934414 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 768/1559] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.935620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd923062-a27a-49c8-9431-cebee080b076 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.935897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd923062-a27a-49c8-9431-cebee080b076 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.936165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd923062-a27a-49c8-9431-cebee080b076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.944508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d181e28f-1f1b-4a08-aa3c-d7afa0318c78 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.945951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d181e28f-1f1b-4a08-aa3c-d7afa0318c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.953883 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 792/1560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.954683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181e28f-1f1b-4a08-aa3c-d7afa0318c78 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.956006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bd923062-a27a-49c8-9431-cebee080b076 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:21.957196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bd923062-a27a-49c8-9431-cebee080b076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:21.965283 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 769/1561] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:21.965972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd923062-a27a-49c8-9431-cebee080b076 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.973846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181e28f-1f1b-4a08-aa3c-d7afa0318c78 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.975373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd923062-a27a-49c8-9431-cebee080b076 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:21.980056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.980391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:21.980689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.980978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.981064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:21.981266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:21.981351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.981534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:21.982330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:21.982501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:21.982632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.982800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:21.984245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.984518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:21.996711 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 770/1562] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:21.997523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:21.998069 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 793/1563] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:21.999036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.009757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f87ffe-1f40-4d31-bf5d-9d93aa6adf14 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.015064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.015779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.016057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.016333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] PATH_INFO: `/v3/projects/fe46a7ae8eab451e941f83e19be3b539/users/49f35168484b42feb22d106d745e6546/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.016824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df89e-0bc0-4bbd-90f8-a098df4ded49 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.023491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.024104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.024515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.024806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.025453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.027373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.032897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe46a7ae8eab451e941f83e19be3b539, user_id=49f35168484b42feb22d106d745e6546, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.035563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.051059 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 794/1564] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:21 2022] PUT /identity/v3/projects/fe46a7ae8eab451e941f83e19be3b539/users/49f35168484b42feb22d106d745e6546/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:22.055719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.057533 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 771/1565] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:21 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.058353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.119854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10ad77-d4bb-42e1-8aad-54c3341de67e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.120199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6ff2c-71cb-46c7-8001-d1fc18b8f76a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.124629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.124973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.125273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.125577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.125687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.125839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.125933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.126106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.126573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:22.126749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:22.126918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.127047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.128512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.128615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.130390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:22.141763 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 772/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:22.142683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.143088 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 795/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:22.143919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.153797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46038e-0ea3-4ae9-9817-c10b0411e0cc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.157747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d6575-d5cf-4d72-a45b-d59dbd58d452 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.159837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.160472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.160780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.161046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.161719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:22.162014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.162821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.163453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.163765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.163888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.164071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.164720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:22.165055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.166627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.177771 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 796/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:22.178601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.187200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 773/1569] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:22.187200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.189107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740302d9-a84c-4d5c-8122-a624eaafcc59 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.194009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.194759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.195044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.195363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.195979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.197508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46643c-05ac-49cc-bac9-c565daa97212 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.197954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.202511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.203173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.203475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.203941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.204608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.206763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.226884 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 797/1570] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.227945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.235600 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 774/1571] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.236682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.283680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:22.291408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1d497-8b78-4734-b308-16ca2eb42695 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.301666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c3e7-49fa-4465-83f1-0f1996fb4f99 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.306489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.307097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.307353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.307621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.308255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.310200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.334319 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 775/1572] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.335464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.345499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e5ad9-6919-4a7f-b8e3-c9a8f53f5c4a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.351335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.351987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.352291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.352618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.353353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.355208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.355363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.355840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.356119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.356354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.357031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.358628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.369330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 798/1573] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:22.370224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.380345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ff55-365d-4369-8179-fab1e73b1660 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.385731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.386335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.386616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.390281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.390416 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 776/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.390515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.390610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.390791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.425780 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 799/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/projects => generated 401 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.425780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.463183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52a0a9-735c-4932-9c7b-2e1f7e2510c8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.471953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:22.493657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a1d71-7196-44e1-8b9c-e680adb8b70c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.498525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.499138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.499412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.499699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.500360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.502332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.521494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.522096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.522373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.522676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.523524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.525586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.532385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 800/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.533413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.559616 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 777/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/projects => generated 393 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.560852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.606259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4b0ed-6a97-413c-a985-041e8f616899 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.630166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:22.653017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ea379-9e17-4ced-a609-f9f105e0f209 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.658342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.658988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.659283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.659558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.660217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.662338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.688774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.689405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.689675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.690096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.690740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.691777 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 778/1578] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.692673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.693882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.705103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafec1f-f4f1-47f3-ab8d-8d4caed16125 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.719069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.719886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.720270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.720635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.721855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.723697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.728067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 801/1579] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/users => generated 301 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.729441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.736760 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 779/1580] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:22.738979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.744791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5bf73-5737-4c16-994b-077c4f3cb6a3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.750348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d7e1d-4eec-4481-a12c-b98c7a28e69e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.750520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.751108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.751386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.751658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] PATH_INFO: `/v3/projects/fe46a7ae8eab451e941f83e19be3b539/users/49f35168484b42feb22d106d745e6546/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.753683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe46a7ae8eab451e941f83e19be3b539, user_id=49f35168484b42feb22d106d745e6546, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.755740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.756372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.756472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.756742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.757586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.758163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.760057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.768162 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 802/1581] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:22 2022] PUT /identity/v3/projects/fe46a7ae8eab451e941f83e19be3b539/users/49f35168484b42feb22d106d745e6546/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:22.777868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.787170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 780/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.788443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.799415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83965-b0c0-4b80-8c26-b3f66d3e3cd7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.834852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:22.852932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82041ad-db33-49be-84c2-e1b1c50c4e38 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.857869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.858506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.858777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.859072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.859705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.861963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.867137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.868161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.868803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.869247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.870032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.873235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.891193 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 803/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.891193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.901085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b47d-bae8-43cd-a881-46138289c6fa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.903747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 781/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.905813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.906419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.906697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.906967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.907579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.907678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.909600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.917974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e199c-24e4-4838-9cf1-563ef991a6ac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.923805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.924459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.924833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.925104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.925748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.927837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.933938 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 804/1585] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:22.937679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.939293 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 782/1586] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:22.940324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.948472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca7f3b-c017-4d01-b0e8-a12087526627 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.950625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bd276-3b16-445f-a465-223ebf23181e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.953844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf76f74-074e-4b2e-b95d-892c7c229e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.954454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf76f74-074e-4b2e-b95d-892c7c229e5a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.954731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf76f74-074e-4b2e-b95d-892c7c229e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.955003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf76f74-074e-4b2e-b95d-892c7c229e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.955646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf76f74-074e-4b2e-b95d-892c7c229e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.955842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.956495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.956804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.957077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.957307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf76f74-074e-4b2e-b95d-892c7c229e5a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.957698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.959282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.969318 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 805/1587] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:22.973852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54adf3-afe8-45cf-a58d-583857bbafd7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.974019 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 783/1588] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:22.974106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54adf3-afe8-45cf-a58d-583857bbafd7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.974106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54adf3-afe8-45cf-a58d-583857bbafd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.974290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:22.983166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00febc-9d85-442a-9aca-c8495d7781e7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:22.987972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:22.988601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:22.988917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:22.989206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:22.989840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:22.992706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:22.999790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7c54adf3-afe8-45cf-a58d-583857bbafd7 None None] MFA Rules not processed for user `49f35168484b42feb22d106d745e6546`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.001055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c54adf3-afe8-45cf-a58d-583857bbafd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:23.003769 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 784/1589] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.004599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.015228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e361-0317-43ad-b303-0e2109353d96 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.021090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.021698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.021977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.022249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.022978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.024574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.036479 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 785/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.037436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.047643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2f087-df13-47d3-883c-ee1fd074d916 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.052565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.053198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.053476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.053745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.054531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.056159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.066130 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 786/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.066945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.074644 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 806/1592] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:22 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:23.076082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54adf3-afe8-45cf-a58d-583857bbafd7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.078615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e399cc3-d302-42d8-b22a-29850ce5c81e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.083389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.083999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.084269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.084538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.085303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.086614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54adf3-afe8-45cf-a58d-583857bbafd7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.087069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.091222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.091831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.092103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.092757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.093031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.094624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.101355 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 787/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.102026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.109543 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 807/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.109543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.112914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fc2c3-fb06-43e7-b9c2-4450c2044186 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.117972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.118594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.118907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.119184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.119909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.121260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8c20d-33ad-4593-a5b9-ba45161bf66b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.121632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.126163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.126922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.127197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.127472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] PATH_INFO: `/v3/projects/6c6a155df2a04d928c0e39a0b04ebe26/users/3cde508c0fb74264acc5faa0acbbf35f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.132265 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 788/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.136493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.143307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6a155df2a04d928c0e39a0b04ebe26, user_id=3cde508c0fb74264acc5faa0acbbf35f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.145932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.146274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3c7a-44df-4dfe-90ff-c3a585915424 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.151105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.151814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.152049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.152396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.153075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.154680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.161805 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 808/1596] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:23 2022] PUT /identity/v3/projects/6c6a155df2a04d928c0e39a0b04ebe26/users/3cde508c0fb74264acc5faa0acbbf35f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:23.162647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.165905 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 789/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.166948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.179228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31491-80bf-4626-ae51-7fe26750d042 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.182993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.234385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42227b3c-02c4-45db-b8c5-0b7ccab9eea5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.239370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e798d1-2c9e-447c-a930-179e06179650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.239984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e798d1-2c9e-447c-a930-179e06179650 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.240262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e798d1-2c9e-447c-a930-179e06179650 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.240565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e798d1-2c9e-447c-a930-179e06179650 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.241245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e798d1-2c9e-447c-a930-179e06179650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.241970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.242583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.242858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.243035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e798d1-2c9e-447c-a930-179e06179650 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.243134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.243755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.244564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.245374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.259785 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 790/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.259978 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 809/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.260063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.260144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6f15f-ce2b-43ae-a875-d7866adc9c36 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.260144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6f15f-ce2b-43ae-a875-d7866adc9c36 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.260144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6f15f-ce2b-43ae-a875-d7866adc9c36 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.261146 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 810/1600] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 27 09:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:23.261980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6f15f-ce2b-43ae-a875-d7866adc9c36 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.269856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ea6d0-2f8d-4c89-8243-91c0c43ce4f1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.275053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6f15f-ce2b-43ae-a875-d7866adc9c36 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.275303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.275912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.277120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.277120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.277120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.278718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.279679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.280343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.280643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.280942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] PATH_INFO: `/v3/projects/681fc4282a6c4fd8ba4f078705f4ca5c/users/023730e34b2d4e2a996bfa068a97413c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.289697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 791/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:23.290527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.300143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087babe6-01b3-4582-aca8-c1d6fa9cc5be admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.300305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681fc4282a6c4fd8ba4f078705f4ca5c, user_id=023730e34b2d4e2a996bfa068a97413c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.303391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.305484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229596-6e7c-46e7-a2e7-a9ffe5705af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.306141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229596-6e7c-46e7-a2e7-a9ffe5705af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.306417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229596-6e7c-46e7-a2e7-a9ffe5705af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.306696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229596-6e7c-46e7-a2e7-a9ffe5705af8 admin admin] PATH_INFO: `/v3/projects/a5b2045c08fb42b888d2fcb7a9cd396f/users/25d7b2696d6745fd944d3131269c70a2/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.318142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 811/1602] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:23 2022] PUT /identity/v3/projects/681fc4282a6c4fd8ba4f078705f4ca5c/users/023730e34b2d4e2a996bfa068a97413c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:23.318966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.324715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f229596-6e7c-46e7-a2e7-a9ffe5705af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b2045c08fb42b888d2fcb7a9cd396f, user_id=25d7b2696d6745fd944d3131269c70a2, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.327383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f229596-6e7c-46e7-a2e7-a9ffe5705af8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.342081 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 792/1603] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:23 2022] PUT /identity/v3/projects/a5b2045c08fb42b888d2fcb7a9cd396f/users/25d7b2696d6745fd944d3131269c70a2/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:23.352913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8dead-fa36-44e2-a848-18f04a0bf8d3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.352913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8dead-fa36-44e2-a848-18f04a0bf8d3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.352913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8dead-fa36-44e2-a848-18f04a0bf8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.375990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-02f8dead-fa36-44e2-a848-18f04a0bf8d3 None None] MFA Rules not processed for user `3cde508c0fb74264acc5faa0acbbf35f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.377320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02f8dead-fa36-44e2-a848-18f04a0bf8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:23.400436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffbaa6-4bb4-4570-b044-b2807e6da198 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.443842 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 793/1604] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:23.448135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8dead-fa36-44e2-a848-18f04a0bf8d3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.462218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.462927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.463635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.463910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] PATH_INFO: `/v3/projects/60dba847e0ea4cc391cd50f10685ece0/users/71a19206cba24022a57ca1b6995037cd/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.479760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60dba847e0ea4cc391cd50f10685ece0, user_id=71a19206cba24022a57ca1b6995037cd, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.482483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.497708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 812/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:23 2022] PUT /identity/v3/projects/60dba847e0ea4cc391cd50f10685ece0/users/71a19206cba24022a57ca1b6995037cd/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:23.498646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.510474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.515590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8dead-fa36-44e2-a848-18f04a0bf8d3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.568682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298b2e-a090-478b-ba8d-e8a26c5c1482 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.573929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236dbee-bce7-4092-a754-3d518325e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.574608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236dbee-bce7-4092-a754-3d518325e5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.574887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236dbee-bce7-4092-a754-3d518325e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.575161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236dbee-bce7-4092-a754-3d518325e5c2 admin admin] PATH_INFO: `/v3/projects/815877f2e9b445829f59f5f3978a0724/users/408201d0dd294b62a63825150eebb461/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.578696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.582470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.583152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.583424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.583692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] PATH_INFO: `/v3/projects/1881e013b4b844cba1afce9fd21f8dd7/users/b4cebc85e62046d28fab159c6bb13e15/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.593293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236dbee-bce7-4092-a754-3d518325e5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=815877f2e9b445829f59f5f3978a0724, user_id=408201d0dd294b62a63825150eebb461, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.595966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236dbee-bce7-4092-a754-3d518325e5c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.599051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1881e013b4b844cba1afce9fd21f8dd7, user_id=b4cebc85e62046d28fab159c6bb13e15, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.602063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.616139 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 813/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:23 2022] PUT /identity/v3/projects/815877f2e9b445829f59f5f3978a0724/users/408201d0dd294b62a63825150eebb461/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:23.617936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c186e-b31a-47d6-9610-4400ea5f5b0c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.618421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 794/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:23 2022] PUT /identity/v3/projects/1881e013b4b844cba1afce9fd21f8dd7/users/b4cebc85e62046d28fab159c6bb13e15/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:23.618525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c186e-b31a-47d6-9610-4400ea5f5b0c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.618525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c186e-b31a-47d6-9610-4400ea5f5b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.622885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.666775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c30c186e-b31a-47d6-9610-4400ea5f5b0c None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.667942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c30c186e-b31a-47d6-9610-4400ea5f5b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:23.686756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1ae68-65d5-4edb-9ebb-1ba92758c8d6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.691395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c3bca-dc94-461f-80ce-b583df8c42a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.692064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c3bca-dc94-461f-80ce-b583df8c42a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.692337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c3bca-dc94-461f-80ce-b583df8c42a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.692605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c3bca-dc94-461f-80ce-b583df8c42a5 admin admin] PATH_INFO: `/v3/projects/f4a54f2fc90f451ca9ab24ceb02b129f/users/94c82956b7484f94bcaf437ba12711a3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.707650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c3bca-dc94-461f-80ce-b583df8c42a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a54f2fc90f451ca9ab24ceb02b129f, user_id=94c82956b7484f94bcaf437ba12711a3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.710503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c3bca-dc94-461f-80ce-b583df8c42a5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.725447 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 795/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:23 2022] PUT /identity/v3/projects/f4a54f2fc90f451ca9ab24ceb02b129f/users/94c82956b7484f94bcaf437ba12711a3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:23.727318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9ed3-f8d7-4ecc-a8cc-01f457717795 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.727594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9ed3-f8d7-4ecc-a8cc-01f457717795 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.727868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9ed3-f8d7-4ecc-a8cc-01f457717795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.749785 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 814/1609] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:23.750733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93b450-f260-4d65-8ae8-2a42126bf058 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.751015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93b450-f260-4d65-8ae8-2a42126bf058 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.751285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93b450-f260-4d65-8ae8-2a42126bf058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.761338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e20a9ed3-f8d7-4ecc-a8cc-01f457717795 None None] MFA Rules not processed for user `023730e34b2d4e2a996bfa068a97413c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.762856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e20a9ed3-f8d7-4ecc-a8cc-01f457717795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:23.781547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ec93b450-f260-4d65-8ae8-2a42126bf058 None None] MFA Rules not processed for user `25d7b2696d6745fd944d3131269c70a2`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.782769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ec93b450-f260-4d65-8ae8-2a42126bf058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:23.832540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.840597 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 796/1610] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:23.841391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9ed3-f8d7-4ecc-a8cc-01f457717795 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:23.857151 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 815/1611] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:23.858479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6b4f7-bd14-4a17-9ad7-842cb9e8e843 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.858760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6b4f7-bd14-4a17-9ad7-842cb9e8e843 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.859027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6b4f7-bd14-4a17-9ad7-842cb9e8e843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.886661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8ca6b4f7-bd14-4a17-9ad7-842cb9e8e843 None None] MFA Rules not processed for user `71a19206cba24022a57ca1b6995037cd`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.887929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8ca6b4f7-bd14-4a17-9ad7-842cb9e8e843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:23.907886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9ed3-f8d7-4ecc-a8cc-01f457717795 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:23.912575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81121b-c881-4fb8-9101-2e597d44b76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:23.913403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81121b-c881-4fb8-9101-2e597d44b76d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.913783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81121b-c881-4fb8-9101-2e597d44b76d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.914102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81121b-c881-4fb8-9101-2e597d44b76d admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.914810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81121b-c881-4fb8-9101-2e597d44b76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:23.916886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81121b-c881-4fb8-9101-2e597d44b76d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:23.933045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.947945 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 797/1612] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:23.949080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0099-5613-46e2-b4b9-dbdf5016e7aa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.949363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0099-5613-46e2-b4b9-dbdf5016e7aa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.949644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0099-5613-46e2-b4b9-dbdf5016e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.960525 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 816/1613] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:23.965610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289b9af-5f91-41e0-ad44-acdf2f4fedb7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:23.965993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289b9af-5f91-41e0-ad44-acdf2f4fedb7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:23.966280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289b9af-5f91-41e0-ad44-acdf2f4fedb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:23.971442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ba6b0099-5613-46e2-b4b9-dbdf5016e7aa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.972695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ba6b0099-5613-46e2-b4b9-dbdf5016e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:23.995914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d289b9af-5f91-41e0-ad44-acdf2f4fedb7 None None] MFA Rules not processed for user `408201d0dd294b62a63825150eebb461`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:23.997229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d289b9af-5f91-41e0-ad44-acdf2f4fedb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:24.039394 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 798/1614] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:24.043774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb418b6-b4f6-4eb6-bbff-1ab5d0fb5f7a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.044143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb418b6-b4f6-4eb6-bbff-1ab5d0fb5f7a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.044253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb418b6-b4f6-4eb6-bbff-1ab5d0fb5f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.067293 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 817/1615] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:24.068154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289b9af-5f91-41e0-ad44-acdf2f4fedb7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.073198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1bb418b6-b4f6-4eb6-bbff-1ab5d0fb5f7a None None] MFA Rules not processed for user `94c82956b7484f94bcaf437ba12711a3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:24.074429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1bb418b6-b4f6-4eb6-bbff-1ab5d0fb5f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:24.140679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289b9af-5f91-41e0-ad44-acdf2f4fedb7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.141256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 799/1616] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:24.142096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb418b6-b4f6-4eb6-bbff-1ab5d0fb5f7a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.149744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.150364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.150653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.150984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.154930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb418b6-b4f6-4eb6-bbff-1ab5d0fb5f7a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.160442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.161099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.161369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.161637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.162265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.164548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.180179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.197093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 800/1617] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.197093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.216795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.218400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.228616 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 818/1618] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:24 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:24.229453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.236138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.240161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6704d-fab3-4a59-aa03-4584c96b0f66 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.267067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc5dbd-1877-4d04-bcfa-49036a2c35a0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.272453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.274015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.274354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.274635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.275280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.277243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.306166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.306781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.307068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.307327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.307957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.310037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.313747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 801/1619] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/projects => generated 403 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.313747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.335243 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 819/1620] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.336780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.391582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d46d2a-c9a6-4951-a465-550f7f490417 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.396292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.396942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.397224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.397563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.398473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.400232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.402591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52e43f-8047-437f-8ab6-47a651788f39 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.432808 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 802/1621] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.434230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.464056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.464752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.465129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.465410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.477582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.479424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.490724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 820/1622] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:24.491763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.501431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db08f-45e8-4b7b-a0e4-1f4c283d02e4 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.507105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.507380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed15d-b65c-49e7-8dd5-84905000476e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.508114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.508371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.508649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.509317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.511267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.512591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.513412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.513821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.514262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.515170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.517228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.528277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.549865 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 821/1623] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.550768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.551782 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 803/1624] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.552930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.595579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.632605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72eabf-33be-4c05-89a5-800f5a7444db admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.638271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.639009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.639290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.639874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.640691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.642661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.643954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827c99c-9933-46eb-b192-39c88ed450df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.650068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.650703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.651105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.651283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.651984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.654236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.674205 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 804/1625] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/users => generated 308 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.675238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.680481 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 822/1626] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.681676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.687088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f402d-5f62-4f4a-ab44-6b8ff5074e9c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.693007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.693903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.694074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ac27b-7efc-4547-9ee0-a0f135ad01ba admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.694611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.695078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.696025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.697951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.700288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.701002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.701386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.701723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.702419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.704752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.711903 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 805/1627] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:24.713076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.724182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96881e4d-b3b4-43f0-b942-43b6007f99ec admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.730383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.731861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.732281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.732728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.733536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.735662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.748144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 823/1628] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/users => generated 298 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.749921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.763792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c1db-2a25-4be6-9854-e3ab386020af admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.769779 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 806/1629] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/users => generated 303 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.770946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.772635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.773737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.774301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.774690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.775412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.777596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.791043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11112-6c73-4068-9686-986a8724cca0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.799530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.800434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.800741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.801012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.802098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.803725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.820719 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 807/1630] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:24.820719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.828903 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 824/1631] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:24 2022] POST /identity/v3/users => generated 302 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:24.830148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.833844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e07dfc-02d6-4117-8401-f8bd36be6538 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.839579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.840447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.840788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.841576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.842365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.843993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.844528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0b653-5bfd-41ec-b349-d100a93e329b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.854596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.856479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.857609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.858244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] PATH_INFO: `/v3/projects/17f3acdcf8a645898bbc1d44b0778be9/users/12d9b17377ac46a2b77eb0957690dbaa/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.861183 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 808/1632] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:24.862516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.877304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faeab8c-6ac2-44f0-a0d5-4a81b647b4d0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:24.879218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f3acdcf8a645898bbc1d44b0778be9, user_id=12d9b17377ac46a2b77eb0957690dbaa, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.882459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.883067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:24.883867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:24.884163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:24.884552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:24.885392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:24.887223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:24.907460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 809/1633] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:24.907460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.915160 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 825/1634] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:24 2022] PUT /identity/v3/projects/17f3acdcf8a645898bbc1d44b0778be9/users/12d9b17377ac46a2b77eb0957690dbaa/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:24.919193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:24.923794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73315af-d3e7-48ab-b74b-813bc9b77938 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.029101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.029742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.030023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.030286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.030947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.038754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.053210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bf4be-aa43-4103-b8ed-9090c26d5403 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.063602 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 810/1635] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.063602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.065954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.067803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.068352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.068735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] PATH_INFO: `/v3/projects/2e256e7d04da47679d60ea6549f50e7b/users/891c58769fc54d58a36cde92f5b358f4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.088857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf36a8e-9c36-40e5-8889-82c7e09b7637 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.100283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.101046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.102138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.102311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.102952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.106112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.108269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e256e7d04da47679d60ea6549f50e7b, user_id=891c58769fc54d58a36cde92f5b358f4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.112863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.127741 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 811/1636] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.129703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.133498 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 826/1637] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:24 2022] PUT /identity/v3/projects/2e256e7d04da47679d60ea6549f50e7b/users/891c58769fc54d58a36cde92f5b358f4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:25.134385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.141153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:25.154071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfae44-7aca-4781-9708-6098e28366fc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.213946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:25.233414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.233611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aba47-a317-408d-8f41-74be1fa1afb6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.234172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.234515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.234855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] PATH_INFO: `/v3/projects/dc9d0df2395744b0baafe373c8913b91/users/e0808d9ea0734660b695111afab964e3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.238125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.238791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.239062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.239333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] PATH_INFO: `/v3/projects/8ed90b8ad6af4db9b88de6e726ac062c/users/a3e92d0a645e41ee886d2408342c38b3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.253612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9d0df2395744b0baafe373c8913b91, user_id=e0808d9ea0734660b695111afab964e3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.256026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.261072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed90b8ad6af4db9b88de6e726ac062c, user_id=a3e92d0a645e41ee886d2408342c38b3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.265254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.273649 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 812/1638] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/dc9d0df2395744b0baafe373c8913b91/users/e0808d9ea0734660b695111afab964e3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:25.274826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.286222 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 827/1639] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/8ed90b8ad6af4db9b88de6e726ac062c/users/a3e92d0a645e41ee886d2408342c38b3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:25.287655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.365392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ecd4c-a610-4249-aa67-b323169db106 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.380631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47902869-62f1-4793-9022-958681a263c9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.387092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.388250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.388812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.389214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] PATH_INFO: `/v3/projects/3eb11513a2234b449d33a3ff704ab242/users/6e05516fc0c84f7eb8b68d1ee80e991d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.412825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb11513a2234b449d33a3ff704ab242, user_id=6e05516fc0c84f7eb8b68d1ee80e991d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.415862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.439390 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 828/1640] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/3eb11513a2234b449d33a3ff704ab242/users/6e05516fc0c84f7eb8b68d1ee80e991d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:25.440787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.447316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.449047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.449457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.449816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.451465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.453888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.469244 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 813/1641] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.470273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.488133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc43237-6ca9-404e-809c-54bdb11c4966 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.534928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:25.559040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7423-9c0b-4185-b985-6cdd5e60b1a4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.567444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a094d-0ce4-49d4-8d13-a630f6b7112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.568273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a094d-0ce4-49d4-8d13-a630f6b7112e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.568554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a094d-0ce4-49d4-8d13-a630f6b7112e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.568861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a094d-0ce4-49d4-8d13-a630f6b7112e admin admin] PATH_INFO: `/v3/projects/4f94985e103a4cd4bc330a2065f48a4d/users/109304194e3b4dc09da005ded7076a86/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.587524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.588213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.588528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.588834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.589487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.591106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.591567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a094d-0ce4-49d4-8d13-a630f6b7112e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f94985e103a4cd4bc330a2065f48a4d, user_id=109304194e3b4dc09da005ded7076a86, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.594621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a094d-0ce4-49d4-8d13-a630f6b7112e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.621249 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 829/1642] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/4f94985e103a4cd4bc330a2065f48a4d/users/109304194e3b4dc09da005ded7076a86/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:25.622987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8596136-ae94-4103-86f2-58e2bc3abbdf None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.623550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8596136-ae94-4103-86f2-58e2bc3abbdf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.624276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8596136-ae94-4103-86f2-58e2bc3abbdf None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.626274 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 830/1643] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:25.626646 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 814/1644] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.627599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8596136-ae94-4103-86f2-58e2bc3abbdf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.627716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.639330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed5b7d-b2d8-41f9-a5d9-116c6146dfa7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.688326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:25.720606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.721412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.721695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.721969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.722774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.724588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.737510 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 815/1645] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.738365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.742770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8596136-ae94-4103-86f2-58e2bc3abbdf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.747612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.748480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.750517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.750517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.750517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.751593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.759785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cba758-2ff4-4450-b787-76271725da79 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.764103 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 831/1646] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.765090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.768536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.773646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.773928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.774201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.774868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.776913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.784630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7962c5a-81d8-4c82-90ce-afee8a0a5688 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.792858 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 816/1647] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.792858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.797233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.797966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.798340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.798649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] PATH_INFO: `/v3/projects/17f3acdcf8a645898bbc1d44b0778be9/users/12d9b17377ac46a2b77eb0957690dbaa/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.801493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f3acdcf8a645898bbc1d44b0778be9, user_id=12d9b17377ac46a2b77eb0957690dbaa, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.802663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273ead-09f8-4898-a801-6b5545d6b601 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.805105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.808266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d34a1-1a51-4b56-bd39-3a93753f0ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.809242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d34a1-1a51-4b56-bd39-3a93753f0ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.809519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d34a1-1a51-4b56-bd39-3a93753f0ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.809853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d34a1-1a51-4b56-bd39-3a93753f0ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.810568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d34a1-1a51-4b56-bd39-3a93753f0ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.812504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d34a1-1a51-4b56-bd39-3a93753f0ef4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.824755 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 832/1648] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/17f3acdcf8a645898bbc1d44b0778be9/users/12d9b17377ac46a2b77eb0957690dbaa/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:25.825880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.835423 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 817/1649] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:25.836793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da71d-2e63-4d43-bfc3-24636f5afcdf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.837063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da71d-2e63-4d43-bfc3-24636f5afcdf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.837334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da71d-2e63-4d43-bfc3-24636f5afcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.865587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-417da71d-2e63-4d43-bfc3-24636f5afcdf None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:25.867255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-417da71d-2e63-4d43-bfc3-24636f5afcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:25.922181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c2ef-4c6d-45b0-a249-cee404cb7b1e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:25.930749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:25.931405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:25.932022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:25.933960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] PATH_INFO: `/v3/projects/2e256e7d04da47679d60ea6549f50e7b/users/891c58769fc54d58a36cde92f5b358f4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:25.936209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e256e7d04da47679d60ea6549f50e7b, user_id=891c58769fc54d58a36cde92f5b358f4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:25.941470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:25.971068 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 818/1651] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:56:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:25.971682 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 833/1651] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/2e256e7d04da47679d60ea6549f50e7b/users/891c58769fc54d58a36cde92f5b358f4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:25.973920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da71d-2e63-4d43-bfc3-24636f5afcdf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.974990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:25.993592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da71d-2e63-4d43-bfc3-24636f5afcdf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:26.041767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:26.077711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f5279-e753-4f49-a310-1c36952cdbef admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:26.088185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510e22d-3945-41dc-b5b4-48d8ed48bc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:26.088359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:26.089241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.089534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.089633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510e22d-3945-41dc-b5b4-48d8ed48bc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.089934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] PATH_INFO: `/v3/projects/8ed90b8ad6af4db9b88de6e726ac062c/users/a3e92d0a645e41ee886d2408342c38b3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.092442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510e22d-3945-41dc-b5b4-48d8ed48bc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.092549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed90b8ad6af4db9b88de6e726ac062c, user_id=a3e92d0a645e41ee886d2408342c38b3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:26.092898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510e22d-3945-41dc-b5b4-48d8ed48bc24 admin admin] PATH_INFO: `/v3/projects/dc9d0df2395744b0baafe373c8913b91/users/e0808d9ea0734660b695111afab964e3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.095620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510e22d-3945-41dc-b5b4-48d8ed48bc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9d0df2395744b0baafe373c8913b91, user_id=e0808d9ea0734660b695111afab964e3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:26.096238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:26.099901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510e22d-3945-41dc-b5b4-48d8ed48bc24 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:26.114504 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 819/1652] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/8ed90b8ad6af4db9b88de6e726ac062c/users/a3e92d0a645e41ee886d2408342c38b3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:26.115561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:26.117651 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 834/1653] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:25 2022] PUT /identity/v3/projects/dc9d0df2395744b0baafe373c8913b91/users/e0808d9ea0734660b695111afab964e3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:26.119220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06d4a2-b4c4-4cdf-b2f8-6c4e34d23a22 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.119615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06d4a2-b4c4-4cdf-b2f8-6c4e34d23a22 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.120113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06d4a2-b4c4-4cdf-b2f8-6c4e34d23a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.163009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aa06d4a2-b4c4-4cdf-b2f8-6c4e34d23a22 None None] MFA Rules not processed for user `12d9b17377ac46a2b77eb0957690dbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:26.165262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa06d4a2-b4c4-4cdf-b2f8-6c4e34d23a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:26.224912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a019aa-9a78-4011-9552-9d34762d623f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:26.250525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:26.300431 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 835/1654] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:26.302843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06d4a2-b4c4-4cdf-b2f8-6c4e34d23a22 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:26.308721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7419e-aea5-4cda-8a40-9cf49dda2de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:26.309428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7419e-aea5-4cda-8a40-9cf49dda2de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.309711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7419e-aea5-4cda-8a40-9cf49dda2de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.309985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7419e-aea5-4cda-8a40-9cf49dda2de6 admin admin] PATH_INFO: `/v3/projects/3eb11513a2234b449d33a3ff704ab242/users/6e05516fc0c84f7eb8b68d1ee80e991d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.313189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7419e-aea5-4cda-8a40-9cf49dda2de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb11513a2234b449d33a3ff704ab242, user_id=6e05516fc0c84f7eb8b68d1ee80e991d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:26.315966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7419e-aea5-4cda-8a40-9cf49dda2de6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:26.325214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06d4a2-b4c4-4cdf-b2f8-6c4e34d23a22 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:26.328744 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 820/1655] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:26 2022] PUT /identity/v3/projects/3eb11513a2234b449d33a3ff704ab242/users/6e05516fc0c84f7eb8b68d1ee80e991d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:26.330384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a861f6-1edf-4202-8a72-484692e4ed3c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.331277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a861f6-1edf-4202-8a72-484692e4ed3c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.331564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a861f6-1edf-4202-8a72-484692e4ed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.389832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-20a861f6-1edf-4202-8a72-484692e4ed3c None None] MFA Rules not processed for user `891c58769fc54d58a36cde92f5b358f4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:26.391451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-20a861f6-1edf-4202-8a72-484692e4ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:26.442251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:26.443495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.443781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.444057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] PATH_INFO: `/v3/projects/4f94985e103a4cd4bc330a2065f48a4d/users/109304194e3b4dc09da005ded7076a86/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.453021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f94985e103a4cd4bc330a2065f48a4d, user_id=109304194e3b4dc09da005ded7076a86, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:26.461532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:26.476578 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 836/1656] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:26 2022] PUT /identity/v3/projects/4f94985e103a4cd4bc330a2065f48a4d/users/109304194e3b4dc09da005ded7076a86/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:26.477719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:26.483351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:26.505193 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 821/1657] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:26.505193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0c46-5057-4a8a-9669-29b9305955ac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.505193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0c46-5057-4a8a-9669-29b9305955ac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.505193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0c46-5057-4a8a-9669-29b9305955ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.537055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-380d0c46-5057-4a8a-9669-29b9305955ac None None] MFA Rules not processed for user `a3e92d0a645e41ee886d2408342c38b3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:26.538593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-380d0c46-5057-4a8a-9669-29b9305955ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:26.590125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d4418-6e51-4a8b-8ec1-74f6b17e07fd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:26.595410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcce5c-e877-4df2-823a-76a7c336e33e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:26.596046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcce5c-e877-4df2-823a-76a7c336e33e service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.596325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcce5c-e877-4df2-823a-76a7c336e33e service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.596606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcce5c-e877-4df2-823a-76a7c336e33e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.606452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcce5c-e877-4df2-823a-76a7c336e33e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:26.608074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcce5c-e877-4df2-823a-76a7c336e33e service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:26.622268 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 837/1658] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:26.623600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13cd0c-342f-4c1d-8be9-881351c4b2ea None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.623929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13cd0c-342f-4c1d-8be9-881351c4b2ea None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.624200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13cd0c-342f-4c1d-8be9-881351c4b2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.657914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 822/1659] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:26.657914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ecd96-1a29-461c-8ee4-c84221c081a8 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.657914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ecd96-1a29-461c-8ee4-c84221c081a8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.657914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ecd96-1a29-461c-8ee4-c84221c081a8 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.657914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 823/1660] 10.222.0.26 () {54 vars in 961 bytes} [Tue Sep 27 09:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:26.657914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ca39-a7ab-4714-95eb-80eb33a4aef7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.657914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ca39-a7ab-4714-95eb-80eb33a4aef7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.658452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ca39-a7ab-4714-95eb-80eb33a4aef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.668844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1c13cd0c-342f-4c1d-8be9-881351c4b2ea None None] MFA Rules not processed for user `e0808d9ea0734660b695111afab964e3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:26.670186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1c13cd0c-342f-4c1d-8be9-881351c4b2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:26.690616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9346ca39-a7ab-4714-95eb-80eb33a4aef7 None None] MFA Rules not processed for user `6e05516fc0c84f7eb8b68d1ee80e991d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:26.691855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9346ca39-a7ab-4714-95eb-80eb33a4aef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:26.729181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:26.776297 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 838/1661] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:26.778184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e2e41-6d1a-48c9-b8bb-91802c654932 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.778659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e2e41-6d1a-48c9-b8bb-91802c654932 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.778939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e2e41-6d1a-48c9-b8bb-91802c654932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.808592 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 824/1662] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:26.809857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd296a75-05f3-4b34-83d4-286f823908b9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.810148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd296a75-05f3-4b34-83d4-286f823908b9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.810425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd296a75-05f3-4b34-83d4-286f823908b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.819788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9c8e2e41-6d1a-48c9-b8bb-91802c654932 None None] MFA Rules not processed for user `109304194e3b4dc09da005ded7076a86`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:26.822000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c8e2e41-6d1a-48c9-b8bb-91802c654932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:26.830764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fd296a75-05f3-4b34-83d4-286f823908b9 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:26.832058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fd296a75-05f3-4b34-83d4-286f823908b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:26.895978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:26.904444 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 839/1663] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:26.905443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e2e41-6d1a-48c9-b8bb-91802c654932 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:26.909768 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 825/1664] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:26.918880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd296a75-05f3-4b34-83d4-286f823908b9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:26.920724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e2e41-6d1a-48c9-b8bb-91802c654932 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:26.926946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:26.927549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.927952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.928126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:26.933792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd296a75-05f3-4b34-83d4-286f823908b9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:26.942340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:26.943648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:26.943840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:26.944132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.035880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.037635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.068309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.070424 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 840/1665] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:26 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:27.071567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.082475 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 826/1666] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:27.191077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:27.206150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4f395-01ab-4e70-9ca8-d52de167e28d service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:27.223960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:27.224859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.225148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.225445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.239763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.245313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.284689 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 841/1667] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:27.305153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:27.318750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:27.328811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50516749-c5bf-4d9a-b6f8-7d4fe93763d1 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:27.334790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84725bae-dad6-41e8-8f7a-d2aaae7fcb2e service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:27.340238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:27.341362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.341897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.342490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.345712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:27.346360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.346640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.346908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.361289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.363255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.363450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.365173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.376476 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 842/1668] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:27 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:27.380893 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 827/1669] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:27.467873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:27.475111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:27.493139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f270ef3-e9de-45f4-9a82-e3aea347514e service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:27.511961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1b17-ee6b-4849-8fcb-1bc38f797965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:27.512594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1b17-ee6b-4849-8fcb-1bc38f797965 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.513015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1b17-ee6b-4849-8fcb-1bc38f797965 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.513354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1b17-ee6b-4849-8fcb-1bc38f797965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.564879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:27.596418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe563e6a-4a01-479a-835c-db3699f81ee9 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:27.612829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:27.613999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.613999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.613999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] PATH_INFO: `/v3/users/408201d0dd294b62a63825150eebb461` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.617820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=408201d0dd294b62a63825150eebb461)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.618856 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:56:27.618856 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 09:56:27.618856 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 09:56:27.620072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.636104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e1b17-ee6b-4849-8fcb-1bc38f797965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.638365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e1b17-ee6b-4849-8fcb-1bc38f797965 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.650782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 843/1670] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:27 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:27.652761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855069b-36ab-4586-a637-c7348756ea49 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.653237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855069b-36ab-4586-a637-c7348756ea49 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.653724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855069b-36ab-4586-a637-c7348756ea49 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.655280 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 844/1671] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:27.663408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e63dd-21c5-451f-b812-407e09a6bd97 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.664639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e63dd-21c5-451f-b812-407e09a6bd97 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.665154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e63dd-21c5-451f-b812-407e09a6bd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.709480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2a8e63dd-21c5-451f-b812-407e09a6bd97 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:27.714589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2a8e63dd-21c5-451f-b812-407e09a6bd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:27.731860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '408201d0dd294b62a63825150eebb461'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:27.737443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '408201d0dd294b62a63825150eebb461'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:27.742800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 845/1672] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:56:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:27.777251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e63dd-21c5-451f-b812-407e09a6bd97 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:27.791603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '408201d0dd294b62a63825150eebb461'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:27.794448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e63dd-21c5-451f-b812-407e09a6bd97 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:27.803351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baed7e2-af29-46a7-9871-dc3e3255686f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:27.804315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baed7e2-af29-46a7-9871-dc3e3255686f service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.804769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baed7e2-af29-46a7-9871-dc3e3255686f service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.805483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baed7e2-af29-46a7-9871-dc3e3255686f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.810685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3baed7e2-af29-46a7-9871-dc3e3255686f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:27.818158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '408201d0dd294b62a63825150eebb461'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:27.829110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baed7e2-af29-46a7-9871-dc3e3255686f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:27.833100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baed7e2-af29-46a7-9871-dc3e3255686f service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:27.864080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 846/1673] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:27.867592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '408201d0dd294b62a63825150eebb461'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:27.871795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:27.884180 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 828/1674] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:27 2022] DELETE /identity/v3/users/408201d0dd294b62a63825150eebb461 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:27.890828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:27.891582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:27.892059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:27.949844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:27.951990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:28.118000 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 847/1675] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:28.125156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:28.126765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:28.240964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b378f0-dcdb-49e0-8ac5-1795b6c4b327 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:28.251095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:28.251999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:28.252477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:28.252995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:28.275549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:28.279310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:28.313029 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 829/1676] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:28.527626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:28.527838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:28.529254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:28.529746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:28.565175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a203fd-d3aa-4a84-8424-549f5d8df876 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:28.576777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:28.580180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:28.581229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:28.581892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] PATH_INFO: `/v3/users/71a19206cba24022a57ca1b6995037cd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:28.584791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a19206cba24022a57ca1b6995037cd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:28.588593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:28.727562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a19206cba24022a57ca1b6995037cd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:28.728279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a19206cba24022a57ca1b6995037cd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:28.746953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a19206cba24022a57ca1b6995037cd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:28.747578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eb743-ada1-4865-9727-12c2eefa0adc service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:28.761421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a19206cba24022a57ca1b6995037cd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:28.763364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560767b7-02ac-450d-ac03-ceacd3e00038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:28.763881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a19206cba24022a57ca1b6995037cd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:28.764053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560767b7-02ac-450d-ac03-ceacd3e00038 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:28.764282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560767b7-02ac-450d-ac03-ceacd3e00038 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:28.766691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560767b7-02ac-450d-ac03-ceacd3e00038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:28.771570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-560767b7-02ac-450d-ac03-ceacd3e00038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:28.806814 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 848/1677] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:28 2022] DELETE /identity/v3/users/71a19206cba24022a57ca1b6995037cd => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:28.806814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:28.812410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:28.850969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:28.937170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560767b7-02ac-450d-ac03-ceacd3e00038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:28.938964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560767b7-02ac-450d-ac03-ceacd3e00038 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:28.947735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f6710-0dc1-4162-8d95-0427fe689f8c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:28.963477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:28.964609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:28.965784 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 830/1678] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:28.967016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:28.967300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:28.968398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:28.968625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:28.974916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] PATH_INFO: `/v3/users/94c82956b7484f94bcaf437ba12711a3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:28.978141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c82956b7484f94bcaf437ba12711a3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:28.983037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:28.994983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:28.996249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.005857 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 831/1679] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:29.006844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:29.008758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.039284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1edee-b4f0-40e6-8e16-1a44326afaa3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:29.047983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:29.048967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:29.049653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:29.052414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] PATH_INFO: `/v3/projects/815877f2e9b445829f59f5f3978a0724` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:29.056823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815877f2e9b445829f59f5f3978a0724)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:29.060211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:29.143756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c82956b7484f94bcaf437ba12711a3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.144742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c82956b7484f94bcaf437ba12711a3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.158410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:29.169231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c82956b7484f94bcaf437ba12711a3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.191005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c82956b7484f94bcaf437ba12711a3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.193651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1404fbb1-4cad-4aa9-abbd-69df72897db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c82956b7484f94bcaf437ba12711a3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.213476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815877f2e9b445829f59f5f3978a0724', 'request_id': 'req-82252bac-1554-4e6e-b7c8-ce63c4aea797'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.227751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815877f2e9b445829f59f5f3978a0724', 'request_id': 'req-82252bac-1554-4e6e-b7c8-ce63c4aea797'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.246355 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 849/1680] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:28 2022] DELETE /identity/v3/users/94c82956b7484f94bcaf437ba12711a3 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:29.253240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ead95b-0afd-4202-bf34-c2ca3c7d0514 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:29.255737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ead95b-0afd-4202-bf34-c2ca3c7d0514 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:29.256154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ead95b-0afd-4202-bf34-c2ca3c7d0514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:29.257912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815877f2e9b445829f59f5f3978a0724', 'request_id': 'req-82252bac-1554-4e6e-b7c8-ce63c4aea797'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.297017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] The token cache is being invalidate because project 815877f2e9b445829f59f5f3978a0724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:29.297017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.297017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.297017 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 832/1681] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:29 2022] DELETE /identity/v3/projects/815877f2e9b445829f59f5f3978a0724 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:29.300742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:29.300742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.318240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-93ead95b-0afd-4202-bf34-c2ca3c7d0514 None None] MFA Rules not processed for user `023730e34b2d4e2a996bfa068a97413c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:29.322101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-93ead95b-0afd-4202-bf34-c2ca3c7d0514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.458410 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 850/1682] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:29.460724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:29.460724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:29.461110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:29.477563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82252bac-1554-4e6e-b7c8-ce63c4aea797 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:29.522188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:29.528153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.588005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:29.588755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:29.589108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:29.589448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:29.598054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.665709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:29.725046 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 851/1683] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:29.725046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:29.728961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.757858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e90e12-1931-4d47-ba25-0333b0a7f615 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:29.775760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:29.776405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:29.776745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:29.777001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] PATH_INFO: `/v3/users/109304194e3b4dc09da005ded7076a86` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:29.779597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109304194e3b4dc09da005ded7076a86)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:29.781592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:29.794503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:29.796610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:29.813796 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 833/1684] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:29.814827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:29.817228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:29.837428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24add75-c254-4373-a72b-ba25600826f1 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:29.857382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:29.858046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:29.858528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:29.860134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] PATH_INFO: `/v3/users/25d7b2696d6745fd944d3131269c70a2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:29.860833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d7b2696d6745fd944d3131269c70a2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:29.862912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:29.894331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:29.909995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109304194e3b4dc09da005ded7076a86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.911318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109304194e3b4dc09da005ded7076a86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.933056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109304194e3b4dc09da005ded7076a86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.944236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109304194e3b4dc09da005ded7076a86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.945379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109304194e3b4dc09da005ded7076a86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.972645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d7b2696d6745fd944d3131269c70a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.982687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d7b2696d6745fd944d3131269c70a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:29.990965 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 852/1685] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:29 2022] DELETE /identity/v3/users/109304194e3b4dc09da005ded7076a86 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:29.992938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:29.994814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.011719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d7b2696d6745fd944d3131269c70a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.033846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d7b2696d6745fd944d3131269c70a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.068057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d7b2696d6745fd944d3131269c70a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.082579 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 834/1686] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:29 2022] DELETE /identity/v3/users/25d7b2696d6745fd944d3131269c70a2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:30.083338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:30.087312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.178543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac036-9259-48c4-a4bb-ac2d65428e75 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:30.216494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca28ed-0a9b-4c77-9b29-04f6f15ec812 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:30.226032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649d026-116a-40de-b106-33b26578e9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:30.226731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649d026-116a-40de-b106-33b26578e9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.227014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649d026-116a-40de-b106-33b26578e9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.227290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649d026-116a-40de-b106-33b26578e9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.230056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8649d026-116a-40de-b106-33b26578e9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.263527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:30.284472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:30.285563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.286141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.286641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.289500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.419898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649d026-116a-40de-b106-33b26578e9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:30.421149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:30.421542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649d026-116a-40de-b106-33b26578e9b9 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:30.422821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:30.472500 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 835/1687] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:30.472843 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 853/1688] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:30.473874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:30.477259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbe158-d99e-4318-9df2-33bd3c65694b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.477533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.477979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbe158-d99e-4318-9df2-33bd3c65694b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.478403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbe158-d99e-4318-9df2-33bd3c65694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.506878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7868f-9868-4405-a225-68afbf8fa816 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:30.525556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:30.526225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.526585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.526757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] PATH_INFO: `/v3/projects/60dba847e0ea4cc391cd50f10685ece0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.528157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60dba847e0ea4cc391cd50f10685ece0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:30.529234 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 09:56:30.529234 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 09:56:30.529234 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 09:56:30.530437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:30.547750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2bcbe158-d99e-4318-9df2-33bd3c65694b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:30.550671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2bcbe158-d99e-4318-9df2-33bd3c65694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.574601 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 836/1689] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:30.591344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbe158-d99e-4318-9df2-33bd3c65694b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:30.628793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbe158-d99e-4318-9df2-33bd3c65694b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:30.667511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60dba847e0ea4cc391cd50f10685ece0', 'request_id': 'req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.683099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60dba847e0ea4cc391cd50f10685ece0', 'request_id': 'req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.693439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60dba847e0ea4cc391cd50f10685ece0', 'request_id': 'req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.716556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:30.740423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] The token cache is being invalidate because project 60dba847e0ea4cc391cd50f10685ece0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:30.740750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.745124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d2b73b3b-c5cb-42c0-8e6e-885a77a59e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:30.750103 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 854/1690] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:30 2022] DELETE /identity/v3/projects/60dba847e0ea4cc391cd50f10685ece0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:30.750103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6bff-2cdc-4654-b2ab-16823a9da367 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.750103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6bff-2cdc-4654-b2ab-16823a9da367 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.750562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6bff-2cdc-4654-b2ab-16823a9da367 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.752563 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 855/1691] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:30.753671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da6bff-2cdc-4654-b2ab-16823a9da367 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:30.754081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d95a8-44fa-4653-979e-64e57c221877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:30.755406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d95a8-44fa-4653-979e-64e57c221877 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.755692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d95a8-44fa-4653-979e-64e57c221877 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.755977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d95a8-44fa-4653-979e-64e57c221877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.758798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da5d95a8-44fa-4653-979e-64e57c221877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.771598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8da6bff-2cdc-4654-b2ab-16823a9da367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.874179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da6bff-2cdc-4654-b2ab-16823a9da367 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:30.876863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d95a8-44fa-4653-979e-64e57c221877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:30.878314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d95a8-44fa-4653-979e-64e57c221877 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:30.879726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:30.880588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.881074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.881565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] PATH_INFO: `/v3/users/a3e92d0a645e41ee886d2408342c38b3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.883217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e92d0a645e41ee886d2408342c38b3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:30.885629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:30.890059 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 837/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:30.891489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:30.892542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:30.892542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:30.914756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:30.915991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:30.957077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:31.021199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e92d0a645e41ee886d2408342c38b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.022017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e92d0a645e41ee886d2408342c38b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.049085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e92d0a645e41ee886d2408342c38b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.066951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e92d0a645e41ee886d2408342c38b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.068856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e92d0a645e41ee886d2408342c38b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.112414 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 856/1693] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:30 2022] DELETE /identity/v3/users/a3e92d0a645e41ee886d2408342c38b3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:31.113443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:31.117918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.143103 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 838/1694] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:56:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:31.158247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:31.160170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.300901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b227f5-0901-4a30-b1ab-cd53803c9994 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:31.313341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:31.314361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:31.314573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:31.314826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] PATH_INFO: `/v3/projects/4f94985e103a4cd4bc330a2065f48a4d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:31.319249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f94985e103a4cd4bc330a2065f48a4d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:31.323820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:31.333939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583c52f-fa3c-4a12-aa94-79273f89e2ae None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:31.361855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:31.415021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f94985e103a4cd4bc330a2065f48a4d', 'request_id': 'req-094524a2-2136-48e5-bae1-a65dffcb5918'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.442993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f94985e103a4cd4bc330a2065f48a4d', 'request_id': 'req-094524a2-2136-48e5-bae1-a65dffcb5918'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.453525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f94985e103a4cd4bc330a2065f48a4d', 'request_id': 'req-094524a2-2136-48e5-bae1-a65dffcb5918'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.480711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] The token cache is being invalidate because project 4f94985e103a4cd4bc330a2065f48a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:31.481052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.482402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-094524a2-2136-48e5-bae1-a65dffcb5918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.484728 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 857/1695] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:31 2022] DELETE /identity/v3/projects/4f94985e103a4cd4bc330a2065f48a4d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:31.490894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15938864-4911-4899-ae1c-d0558a8a69e1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:31.491213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15938864-4911-4899-ae1c-d0558a8a69e1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:31.491487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15938864-4911-4899-ae1c-d0558a8a69e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:31.497587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e84953-0d4b-4f21-83a2-f9c0dbb328ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:31.498231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e84953-0d4b-4f21-83a2-f9c0dbb328ee service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:31.498498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e84953-0d4b-4f21-83a2-f9c0dbb328ee service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:31.498769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e84953-0d4b-4f21-83a2-f9c0dbb328ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:31.501730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-25e84953-0d4b-4f21-83a2-f9c0dbb328ee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.555703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-15938864-4911-4899-ae1c-d0558a8a69e1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:31.556699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15938864-4911-4899-ae1c-d0558a8a69e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.609003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:31.701395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 858/1696] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:31.703502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:31.704153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:31.704602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:31.728815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e84953-0d4b-4f21-83a2-f9c0dbb328ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:31.730605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e84953-0d4b-4f21-83a2-f9c0dbb328ee service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:31.764818 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 839/1697] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:31.773576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] MFA Rules not processed for user `b4cebc85e62046d28fab159c6bb13e15`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:31.774059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:31.774303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:31.774438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.774591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:31.809363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:31.811152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.826073 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 840/1698] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:31.830047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:31.832288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.853101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae553ba-094e-450b-aaad-2d6e3c6b73b2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:31.858169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:31.858829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:31.859110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:31.859385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] PATH_INFO: `/v3/projects/f4a54f2fc90f451ca9ab24ceb02b129f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:31.860787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a54f2fc90f451ca9ab24ceb02b129f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:31.862914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:31.877150 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 859/1699] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:31.878185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:31.879863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.897739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3547b9-0f08-48ac-a28c-56851b8e2d53 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:31.903469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:31.904247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:31.904584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:31.904969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] PATH_INFO: `/v3/projects/a5b2045c08fb42b888d2fcb7a9cd396f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:31.906170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b2045c08fb42b888d2fcb7a9cd396f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:31.908406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:31.935941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a54f2fc90f451ca9ab24ceb02b129f', 'request_id': 'req-06f9b3f3-3e58-4335-8186-0b0c457b8d14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.943661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a54f2fc90f451ca9ab24ceb02b129f', 'request_id': 'req-06f9b3f3-3e58-4335-8186-0b0c457b8d14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.956272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a54f2fc90f451ca9ab24ceb02b129f', 'request_id': 'req-06f9b3f3-3e58-4335-8186-0b0c457b8d14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.973782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] The token cache is being invalidate because project f4a54f2fc90f451ca9ab24ceb02b129f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:31.974104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.975533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:31.977471 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 841/1700] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:31 2022] DELETE /identity/v3/projects/f4a54f2fc90f451ca9ab24ceb02b129f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:31.978025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:31.979647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:31.993992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b2045c08fb42b888d2fcb7a9cd396f', 'request_id': 'req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.007895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b2045c08fb42b888d2fcb7a9cd396f', 'request_id': 'req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.014954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b2045c08fb42b888d2fcb7a9cd396f', 'request_id': 'req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.022914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:32.028865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] The token cache is being invalidate because project a5b2045c08fb42b888d2fcb7a9cd396f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:32.029163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.030003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.032025 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 860/1701] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:31 2022] DELETE /identity/v3/projects/a5b2045c08fb42b888d2fcb7a9cd396f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:32.032597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.034724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.067116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9b3f3-3e58-4335-8186-0b0c457b8d14 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.090706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:32.126287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30addc4-db46-4b82-a5a9-fd104eaf3dbf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.131304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.131872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.132171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.132517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.133213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:32.133648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.135269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.136717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.137703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.138495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.138947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] PATH_INFO: `/v3/projects/8ed90b8ad6af4db9b88de6e726ac062c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.140272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed90b8ad6af4db9b88de6e726ac062c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.142518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.150686 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 861/1702] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:32.151682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.153725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.170166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d1988-e82d-4348-9769-ed50cc030fc6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.177629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.178318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.178596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.178906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.179687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.182528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.202527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed90b8ad6af4db9b88de6e726ac062c', 'request_id': 'req-e8bb5135-44f4-4632-8876-46410e50fc55'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.210073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed90b8ad6af4db9b88de6e726ac062c', 'request_id': 'req-e8bb5135-44f4-4632-8876-46410e50fc55'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.217071 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 862/1703] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:32 2022] POST /identity/v3/projects => generated 401 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:32.217910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.220181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.222146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed90b8ad6af4db9b88de6e726ac062c', 'request_id': 'req-e8bb5135-44f4-4632-8876-46410e50fc55'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.235622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] The token cache is being invalidate because project 8ed90b8ad6af4db9b88de6e726ac062c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:32.235922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.236761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.238572 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 842/1704] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:31 2022] DELETE /identity/v3/projects/8ed90b8ad6af4db9b88de6e726ac062c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:32.239110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.241245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.298189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b9779-2edc-42bc-94bf-973a34e0fa8e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.305030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.305823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.306169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.306560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] PATH_INFO: `/v3/users/6e05516fc0c84f7eb8b68d1ee80e991d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.308125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e05516fc0c84f7eb8b68d1ee80e991d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.310449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.314721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb5135-44f4-4632-8876-46410e50fc55 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.320288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.321296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.321818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.322264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] PATH_INFO: `/v3/users/891c58769fc54d58a36cde92f5b358f4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.323673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=891c58769fc54d58a36cde92f5b358f4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.325953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.346395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:32.404153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891c58769fc54d58a36cde92f5b358f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.405686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891c58769fc54d58a36cde92f5b358f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.406573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e05516fc0c84f7eb8b68d1ee80e991d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.407560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e05516fc0c84f7eb8b68d1ee80e991d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.417253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891c58769fc54d58a36cde92f5b358f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.422605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e05516fc0c84f7eb8b68d1ee80e991d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.430246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891c58769fc54d58a36cde92f5b358f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.435465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e05516fc0c84f7eb8b68d1ee80e991d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.436153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e05516fc0c84f7eb8b68d1ee80e991d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.449312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891c58769fc54d58a36cde92f5b358f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:32.458934 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 843/1705] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:32 2022] DELETE /identity/v3/users/891c58769fc54d58a36cde92f5b358f4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:32.462499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 863/1706] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:32 2022] DELETE /identity/v3/users/6e05516fc0c84f7eb8b68d1ee80e991d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:32.462499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.462775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.465834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.478228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.546258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:32.552446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e21786-0a7a-4e93-b0d8-9e22ef630d38 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.595785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91c4b-0748-461e-ab32-3107aadc497c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.606147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9858b9b-c812-45bd-a7f9-91113c276cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.606147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9858b9b-c812-45bd-a7f9-91113c276cff admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.606147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9858b9b-c812-45bd-a7f9-91113c276cff admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.606147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9858b9b-c812-45bd-a7f9-91113c276cff admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.608307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9858b9b-c812-45bd-a7f9-91113c276cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.609993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9858b9b-c812-45bd-a7f9-91113c276cff admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.644699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 864/1707] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:32 2022] POST /identity/v3/users => generated 307 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:32.645095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8faef-2845-4051-a5eb-f46ba8bbd02f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.645512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8faef-2845-4051-a5eb-f46ba8bbd02f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.645883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8faef-2845-4051-a5eb-f46ba8bbd02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.671139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7fd8faef-2845-4051-a5eb-f46ba8bbd02f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:32.672696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7fd8faef-2845-4051-a5eb-f46ba8bbd02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.696724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 865/1708] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:32.696724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea9e98-3c12-4690-be55-00201c5c7548 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.696724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea9e98-3c12-4690-be55-00201c5c7548 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.696724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea9e98-3c12-4690-be55-00201c5c7548 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.696724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 866/1709] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:32.696724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea9e98-3c12-4690-be55-00201c5c7548 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.703620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-deea9e98-3c12-4690-be55-00201c5c7548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.712124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.712951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.714424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.714711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.715711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.719321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.771681 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 844/1710] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:32 2022] POST /identity/v3/projects => generated 373 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:32.772743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:32.804412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:32.820509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea9e98-3c12-4690-be55-00201c5c7548 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.873927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0d4c0-c3c7-4e6c-bdda-b288603f452e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:32.879441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3de7a-475a-468f-90ea-4218e5026f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.880097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3de7a-475a-468f-90ea-4218e5026f03 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.880376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3de7a-475a-468f-90ea-4218e5026f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.880649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3de7a-475a-468f-90ea-4218e5026f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.881383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3de7a-475a-468f-90ea-4218e5026f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.884790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3de7a-475a-468f-90ea-4218e5026f03 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:32.900411 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 845/1711] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:32.901908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4e5c-c563-46bc-b184-ea539cfb1db9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.902234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4e5c-c563-46bc-b184-ea539cfb1db9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.902467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4e5c-c563-46bc-b184-ea539cfb1db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.942422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:32.943083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.943360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.943629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.946210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a30f4e5c-c563-46bc-b184-ea539cfb1db9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:32.948243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a30f4e5c-c563-46bc-b184-ea539cfb1db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.951799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:32.968677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 846/1712] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:32.974249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d13a8-9ccf-40f9-9acb-fc88db37e5d0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:32.974532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d13a8-9ccf-40f9-9acb-fc88db37e5d0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:32.974806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d13a8-9ccf-40f9-9acb-fc88db37e5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:32.984731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:32.989384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.017071 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 867/1713] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:33.018217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.019718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.023017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-730d13a8-9ccf-40f9-9acb-fc88db37e5d0 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:33.025406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-730d13a8-9ccf-40f9-9acb-fc88db37e5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.034011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c81c2-9b17-42fa-a0c3-65846037fa78 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.039388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.040048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.040331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.040591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.041395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.043816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.088233 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 868/1714] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:33 2022] POST /identity/v3/users => generated 287 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:33.088233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.097736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.122842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b03abe-460d-4360-875e-94e6a33d838d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.134168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.134816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.135090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.135355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] PATH_INFO: `/v3/projects/2e256e7d04da47679d60ea6549f50e7b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.137787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e256e7d04da47679d60ea6549f50e7b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.139745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.149030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 847/1715] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:56:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:33.150384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d13a8-9ccf-40f9-9acb-fc88db37e5d0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.163389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d13a8-9ccf-40f9-9acb-fc88db37e5d0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.168518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.169474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.169989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.170382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] PATH_INFO: `/v3/projects/4d40f3dbcee3467f92fc9bcbd4bde2eb/users/5cc03c5c36f14c389cb76fa0b4dc78e0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.181849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:33.219020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d40f3dbcee3467f92fc9bcbd4bde2eb, user_id=5cc03c5c36f14c389cb76fa0b4dc78e0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.220241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e256e7d04da47679d60ea6549f50e7b', 'request_id': 'req-68262a81-099a-4e48-8228-9a132dbd83fb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.221046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.238295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e256e7d04da47679d60ea6549f50e7b', 'request_id': 'req-68262a81-099a-4e48-8228-9a132dbd83fb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.245359 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 848/1716] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:33 2022] PUT /identity/v3/projects/4d40f3dbcee3467f92fc9bcbd4bde2eb/users/5cc03c5c36f14c389cb76fa0b4dc78e0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:33.245904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e256e7d04da47679d60ea6549f50e7b', 'request_id': 'req-68262a81-099a-4e48-8228-9a132dbd83fb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.246288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.270313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] The token cache is being invalidate because project 2e256e7d04da47679d60ea6549f50e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:33.270690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.281245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.286474 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 869/1717] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:33 2022] DELETE /identity/v3/projects/2e256e7d04da47679d60ea6549f50e7b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:33.287261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.288978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.336618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619410cc-d0fa-469b-b861-d4b82c193f40 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.341673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.342406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.342970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.343349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.344178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:33.344583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.346572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.366363 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 849/1718] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:33.367530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.369339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.387684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ffb9b-c8d2-4b67-b445-89432dea3ac4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.392906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.393693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.394030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.394298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.394940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.396921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.402308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68262a81-099a-4e48-8228-9a132dbd83fb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.403576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:33.409533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.413892 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 850/1719] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:33.413892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.414290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.414903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.415850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.417005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] PATH_INFO: `/v3/projects/3eb11513a2234b449d33a3ff704ab242` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.419362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb11513a2234b449d33a3ff704ab242)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.423879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.487616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97bedf-24ba-4539-8aee-14a8d7a3eaaf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.493449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb11513a2234b449d33a3ff704ab242', 'request_id': 'req-14ed99d7-a9fd-4472-ac97-702128382b06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.510466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb11513a2234b449d33a3ff704ab242', 'request_id': 'req-14ed99d7-a9fd-4472-ac97-702128382b06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.518928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb11513a2234b449d33a3ff704ab242', 'request_id': 'req-14ed99d7-a9fd-4472-ac97-702128382b06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.534210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] The token cache is being invalidate because project 3eb11513a2234b449d33a3ff704ab242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:33.534490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.535712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:33.537588 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 870/1720] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:33 2022] DELETE /identity/v3/projects/3eb11513a2234b449d33a3ff704ab242 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:33.538806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.540551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.579197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.580885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.581169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.581439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.586890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.598249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:33.625851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed99d7-a9fd-4472-ac97-702128382b06 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.633039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be6eb4-331c-4bef-810e-97a5d84a9075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.633853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be6eb4-331c-4bef-810e-97a5d84a9075 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.634196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be6eb4-331c-4bef-810e-97a5d84a9075 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.635277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be6eb4-331c-4bef-810e-97a5d84a9075 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.636140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be6eb4-331c-4bef-810e-97a5d84a9075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.638918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be6eb4-331c-4bef-810e-97a5d84a9075 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.654053 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 871/1721] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:33.655511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.655839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.656113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.681434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:33.682913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.700317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 872/1722] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:33.701536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.703303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.709407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.711685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.717472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369d627-a301-4242-8138-8ac166d928d3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.723756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.724785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.725245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.725688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.726634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.729082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.732886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 851/1723] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:33.734734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.739197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.762051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d646e2-3770-4660-b6b2-184854bcb909 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.767319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd2828-ec3a-48b4-81b2-e95fc4543e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.768035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd2828-ec3a-48b4-81b2-e95fc4543e65 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.768318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd2828-ec3a-48b4-81b2-e95fc4543e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.768592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd2828-ec3a-48b4-81b2-e95fc4543e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.769551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd2828-ec3a-48b4-81b2-e95fc4543e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.771427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd2828-ec3a-48b4-81b2-e95fc4543e65 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.789022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 852/1724] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:33.789022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2c27-7877-4224-bf06-163f527ca8ee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.789022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2c27-7877-4224-bf06-163f527ca8ee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.789022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2c27-7877-4224-bf06-163f527ca8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.799975 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 873/1725] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:33 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:33.801631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.838356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0d7b2c27-7877-4224-bf06-163f527ca8ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:33.840579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d7b2c27-7877-4224-bf06-163f527ca8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.893904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea132-d77a-42d4-9780-c2d692fc2918 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:33.904208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d6402-0717-4012-9aa8-9b56a2be7551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:33.905543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6402-0717-4012-9aa8-9b56a2be7551 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.905999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6402-0717-4012-9aa8-9b56a2be7551 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.906414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6402-0717-4012-9aa8-9b56a2be7551 admin admin] PATH_INFO: `/v3/projects/4d40f3dbcee3467f92fc9bcbd4bde2eb/users/5cc03c5c36f14c389cb76fa0b4dc78e0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.909391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d6402-0717-4012-9aa8-9b56a2be7551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d40f3dbcee3467f92fc9bcbd4bde2eb, user_id=5cc03c5c36f14c389cb76fa0b4dc78e0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:33.912169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d6402-0717-4012-9aa8-9b56a2be7551 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:33.915045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:33.928761 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 874/1726] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:33 2022] PUT /identity/v3/projects/4d40f3dbcee3467f92fc9bcbd4bde2eb/users/5cc03c5c36f14c389cb76fa0b4dc78e0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:33.931095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fd898-5258-4553-966e-05761e8463b0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:33.932993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fd898-5258-4553-966e-05761e8463b0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:33.933289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fd898-5258-4553-966e-05761e8463b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:33.957214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3c1fd898-5258-4553-966e-05761e8463b0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:33.958548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3c1fd898-5258-4553-966e-05761e8463b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:33.960846 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 853/1727] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:33.962170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2c27-7877-4224-bf06-163f527ca8ee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:33.976222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2c27-7877-4224-bf06-163f527ca8ee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.067960 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 875/1728] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:34.068878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fd898-5258-4553-966e-05761e8463b0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.079813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac2f33-b330-4784-afa8-f50d9c4282f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.080454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac2f33-b330-4784-afa8-f50d9c4282f8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.081067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac2f33-b330-4784-afa8-f50d9c4282f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.081460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac2f33-b330-4784-afa8-f50d9c4282f8 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.082504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac2f33-b330-4784-afa8-f50d9c4282f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.085153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac2f33-b330-4784-afa8-f50d9c4282f8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.088597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fd898-5258-4553-966e-05761e8463b0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.097135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.097989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.098325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.098622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] PATH_INFO: `/v3/projects/f08ab464ec0846d6a50c15d130cf88a2/users/abb11855f4bd4764b26791b9cab9b043/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.118433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 854/1729] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:33 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:34.118433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55aa4-9fef-43f8-bc2b-d50e8632157d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.118433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55aa4-9fef-43f8-bc2b-d50e8632157d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.118433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55aa4-9fef-43f8-bc2b-d50e8632157d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.121690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08ab464ec0846d6a50c15d130cf88a2, user_id=abb11855f4bd4764b26791b9cab9b043, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.127610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.144469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-10d55aa4-9fef-43f8-bc2b-d50e8632157d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:34.146153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-10d55aa4-9fef-43f8-bc2b-d50e8632157d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:34.158513 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 876/1730] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:34 2022] PUT /identity/v3/projects/f08ab464ec0846d6a50c15d130cf88a2/users/abb11855f4bd4764b26791b9cab9b043/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:34.183337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.226387 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 855/1731] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:34.227547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55aa4-9fef-43f8-bc2b-d50e8632157d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.241176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55aa4-9fef-43f8-bc2b-d50e8632157d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.243736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:34.249221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d5144-5c85-4973-9d07-eed25120b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.249804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5144-5c85-4973-9d07-eed25120b2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.250308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5144-5c85-4973-9d07-eed25120b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.250963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d5144-5c85-4973-9d07-eed25120b2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.251774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d5144-5c85-4973-9d07-eed25120b2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.255344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d5144-5c85-4973-9d07-eed25120b2d2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.278887 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 856/1732] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:34.278887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de303-586a-43ff-a394-72d745ff2455 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.278887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de303-586a-43ff-a394-72d745ff2455 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.278887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de303-586a-43ff-a394-72d745ff2455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.309696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda578a3-82b8-4aa1-88d1-f4176f630a06 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.314528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.315153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.315430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.315704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.316547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:34.316650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.318385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.326417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-184de303-586a-43ff-a394-72d745ff2455 None None] MFA Rules not processed for user `abb11855f4bd4764b26791b9cab9b043`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:34.328600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-184de303-586a-43ff-a394-72d745ff2455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:34.332128 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 877/1733] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:34.333304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.346395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091e00b-25f8-408d-b0a0-d3c962eb77a4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.353052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.353777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.354059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.354412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.355131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:34.355484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.357485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.377257 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 878/1734] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:34.378140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.389639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac457d-6844-4f8c-8607-1dcf795a342f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.394842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.395404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.395685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.395957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.396609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.398542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.417220 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 879/1735] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:34.417220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.431932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:34.437494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c0577-8717-4047-9816-bf6a79b5ab5a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.446445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.447445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.447944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.448491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.449530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.451699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.452517 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 857/1736] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:34.453900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de303-586a-43ff-a394-72d745ff2455 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.467086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de303-586a-43ff-a394-72d745ff2455 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.472338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.473211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.473521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.473954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.475153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.477698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.491051 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 880/1737] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:34.493154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.521755 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 858/1738] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/projects => generated 407 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:34.522732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.583099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486f731-c4e2-409c-a4e8-8fbe2c2ee1fa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.644343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace80294-6493-47b9-b7f0-e50e2425ba51 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.659052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.662882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.663198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.664699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.666565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.670397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.671756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.672031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.672308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d/users/862f94043f044a25b740db652bf3dc02/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.672411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.694107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81a1b6d13954194a01cc43062b0038d, user_id=862f94043f044a25b740db652bf3dc02, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.697609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.711150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:34.739641 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 859/1739] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:34.743882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.747893 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 881/1740] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:34 2022] PUT /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d/users/862f94043f044a25b740db652bf3dc02/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:34.749440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.829905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40eb28-abc5-4430-8ee3-b39d14c86fac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.902210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336aa38-8840-4feb-839c-bec934c41181 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.908594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.909304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.909678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.910039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.910824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.913094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.941822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fb3e6-eab5-4692-902d-bff8f93b9e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.942454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb3e6-eab5-4692-902d-bff8f93b9e77 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.942723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb3e6-eab5-4692-902d-bff8f93b9e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.943948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb3e6-eab5-4692-902d-bff8f93b9e77 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.944698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fb3e6-eab5-4692-902d-bff8f93b9e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.946860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fb3e6-eab5-4692-902d-bff8f93b9e77 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:34.954531 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 860/1741] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:34.955324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:34.971815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b781f-b32b-46eb-aba1-0ecca572d5d0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:34.978092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:34.978995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:34.978995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:34.979189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:34.979818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:34.982138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.003208 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 882/1742] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/users => generated 304 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:35.006311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41a38-e430-4a35-aca7-49fe923a19a3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.006951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41a38-e430-4a35-aca7-49fe923a19a3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.007505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd41a38-e430-4a35-aca7-49fe923a19a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.022788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 861/1743] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:34 2022] POST /identity/v3/users => generated 293 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:35.024139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.037600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7487b2-2b0d-479a-981b-966299027462 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.044117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.045051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.045504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.045913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.046939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.049216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.066792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ebd41a38-e430-4a35-aca7-49fe923a19a3 None None] MFA Rules not processed for user `862f94043f044a25b740db652bf3dc02`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:35.068338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ebd41a38-e430-4a35-aca7-49fe923a19a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:35.089379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 862/1744] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:35.090648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.112647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd466e-a9d0-4feb-b969-2a9617113b71 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.118156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.118788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.119081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.119337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.119994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.121645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.145014 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 863/1745] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:35.146555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.189399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0cbb-b960-403e-9300-b3907336c201 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.202649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.205321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.205750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.207926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.208976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.210977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.214275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:35.236404 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 883/1746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:35.237281 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 864/1747] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:35.239536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd41a38-e430-4a35-aca7-49fe923a19a3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.240354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.263845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd41a38-e430-4a35-aca7-49fe923a19a3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.264813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf6f81-bea2-484e-a0a2-f274b2ea408b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.269945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.270622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.270902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.271176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.272136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.277199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.282127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.282534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.283204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.283586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.285209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.287092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.292850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 884/1748] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:35.293789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.302796 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 865/1749] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:35.304062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.306924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84dbd-07dc-4e53-b494-35dcd2f2b9c6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.314584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.315556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.315844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.316120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.316891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.318950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170c0ae-cd87-43ba-862a-116b73178c13 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.319129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.324642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.325564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.326467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.326750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] PATH_INFO: `/v3/projects/d23ec99b72334ba4ad76bb15521b7fdb/users/c274baa6558447ca810435c59673fbab/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.338684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:35.349763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23ec99b72334ba4ad76bb15521b7fdb, user_id=c274baa6558447ca810435c59673fbab, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.354426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.366134 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 885/1750] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:35 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:35.367967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.400355 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 866/1751] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:35 2022] PUT /identity/v3/projects/d23ec99b72334ba4ad76bb15521b7fdb/users/c274baa6558447ca810435c59673fbab/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:35.401348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.478493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe354-d260-4ebb-8472-571e87c4564f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.572340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75d4fa-5278-4a59-83dc-d93247b12a27 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.586515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.587251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.588589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.588906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] PATH_INFO: `/v3/projects/246bd38c478e448cab936abd93fa094e/users/8daac732ab214a56a401a3487f2b820c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.622357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246bd38c478e448cab936abd93fa094e, user_id=8daac732ab214a56a401a3487f2b820c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.626271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.626597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.627519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.627995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.628486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] PATH_INFO: `/v3/projects/b0bb608f923d4e66be0ac6d38e345721/users/4aaff47516a241e9aa9ad5e2007429e0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.657508 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 867/1752] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:35 2022] PUT /identity/v3/projects/246bd38c478e448cab936abd93fa094e/users/8daac732ab214a56a401a3487f2b820c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:35.658836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0bb608f923d4e66be0ac6d38e345721, user_id=4aaff47516a241e9aa9ad5e2007429e0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.662361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.662938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.697644 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 886/1753] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:35 2022] PUT /identity/v3/projects/b0bb608f923d4e66be0ac6d38e345721/users/4aaff47516a241e9aa9ad5e2007429e0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:35.698745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:35.806840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07c784-e75d-4258-9566-b0a87e103756 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.813790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5540f-3b51-429d-a3de-bde9e578fe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.815735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5540f-3b51-429d-a3de-bde9e578fe64 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.816037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5540f-3b51-429d-a3de-bde9e578fe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.816784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5540f-3b51-429d-a3de-bde9e578fe64 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.817053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5540f-3b51-429d-a3de-bde9e578fe64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.820198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3434e39-3fe0-4af2-ae4c-63a19e4075f2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:35.820529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5540f-3b51-429d-a3de-bde9e578fe64 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.826430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29bf65-ef51-4c65-b397-77d98e6d9bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:35.827424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29bf65-ef51-4c65-b397-77d98e6d9bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.828718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29bf65-ef51-4c65-b397-77d98e6d9bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.829414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29bf65-ef51-4c65-b397-77d98e6d9bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.830327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29bf65-ef51-4c65-b397-77d98e6d9bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:35.833727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29bf65-ef51-4c65-b397-77d98e6d9bd5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:35.851578 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 887/1754] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:35.853583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde1c1-7776-4afa-addf-6d945b7bb8b0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.854078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde1c1-7776-4afa-addf-6d945b7bb8b0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.854549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde1c1-7776-4afa-addf-6d945b7bb8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.862764 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 868/1755] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:35 2022] POST /identity/v3/users => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:35.866843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f1f82-b147-469e-a178-dbb5667ba011 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:35.867292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f1f82-b147-469e-a178-dbb5667ba011 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:35.868397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f1f82-b147-469e-a178-dbb5667ba011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:35.910957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-423f1f82-b147-469e-a178-dbb5667ba011 None None] MFA Rules not processed for user `4aaff47516a241e9aa9ad5e2007429e0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:35.911561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b0fde1c1-7776-4afa-addf-6d945b7bb8b0 None None] MFA Rules not processed for user `8daac732ab214a56a401a3487f2b820c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:35.912364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-423f1f82-b147-469e-a178-dbb5667ba011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:35.912881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b0fde1c1-7776-4afa-addf-6d945b7bb8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:35.921257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:35.943408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:35.999936 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 869/1756] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:36.001504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609e20-d2ef-4977-8fcb-e7c90f1b3811 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.001902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609e20-d2ef-4977-8fcb-e7c90f1b3811 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.002771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609e20-d2ef-4977-8fcb-e7c90f1b3811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.016010 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 888/1757] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:36.021117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde1c1-7776-4afa-addf-6d945b7bb8b0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.034852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde1c1-7776-4afa-addf-6d945b7bb8b0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.040685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.041384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.041660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.041928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] PATH_INFO: `/v3/projects/d23ec99b72334ba4ad76bb15521b7fdb/users/c274baa6558447ca810435c59673fbab/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.044578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23ec99b72334ba4ad76bb15521b7fdb, user_id=c274baa6558447ca810435c59673fbab, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.047995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.061630 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 889/1758] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:36 2022] PUT /identity/v3/projects/d23ec99b72334ba4ad76bb15521b7fdb/users/c274baa6558447ca810435c59673fbab/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:36.073124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.076503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ae609e20-d2ef-4977-8fcb-e7c90f1b3811 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:36.078484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae609e20-d2ef-4977-8fcb-e7c90f1b3811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:36.166908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44827b4-010e-4142-8763-f2806a619429 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.174348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.175756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.176108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.176815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.177548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.179079 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 870/1759] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:36.180134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae609e20-d2ef-4977-8fcb-e7c90f1b3811 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.186108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.199265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae609e20-d2ef-4977-8fcb-e7c90f1b3811 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.205926 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 890/1760] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:36.207141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.207721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5f93b-2e78-4f86-852f-df93bdd68a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.209183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5f93b-2e78-4f86-852f-df93bdd68a44 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.209536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5f93b-2e78-4f86-852f-df93bdd68a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.209822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5f93b-2e78-4f86-852f-df93bdd68a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.210551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5f93b-2e78-4f86-852f-df93bdd68a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.217800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5f93b-2e78-4f86-852f-df93bdd68a44 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.231080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0806f9-9ff5-4033-915f-506b2fefcd4f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.245530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.246166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.246443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.246732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.247515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.249529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.264906 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 871/1761] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:36 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:36.267068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9322b-33a4-42c6-ac62-ecb8042ca273 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.267349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9322b-33a4-42c6-ac62-ecb8042ca273 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.268166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9322b-33a4-42c6-ac62-ecb8042ca273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.297803 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 891/1762] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:36 2022] POST /identity/v3/projects => generated 409 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:36.298940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.317655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-55d9322b-33a4-42c6-ac62-ecb8042ca273 None None] MFA Rules not processed for user `c274baa6558447ca810435c59673fbab`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:36.320156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55d9322b-33a4-42c6-ac62-ecb8042ca273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:36.443526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:36.450677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 872/1763] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:36.451862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9322b-33a4-42c6-ac62-ecb8042ca273 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.466811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b179f-5a9f-42c3-aa22-195098cac654 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.474027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.474864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.474973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.475673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] PATH_INFO: `/v3/projects/c6877dc065e44ab4a5ff07428f46867f/users/db4b607d5bc940d28d3c3b9edb6ef4a2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.481424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9322b-33a4-42c6-ac62-ecb8042ca273 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.488844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.489827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.490301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.490984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.492851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.495127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.497684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:36.500133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6877dc065e44ab4a5ff07428f46867f, user_id=db4b607d5bc940d28d3c3b9edb6ef4a2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.504619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.540247 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 892/1764] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:36 2022] PUT /identity/v3/projects/c6877dc065e44ab4a5ff07428f46867f/users/db4b607d5bc940d28d3c3b9edb6ef4a2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:36.540247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.547950 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 873/1765] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:36 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:36.550537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.554239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:36.650146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39dc3-b40b-4ab1-8f2c-686b25ebdcc0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.657741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.658608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.659064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.659424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.660213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.663049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.708964 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 893/1766] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:36 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:36.710475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.721713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:36.743743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0632f-da0b-4050-b89b-0180f3fea94e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.865124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.865966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.866312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.866769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.895091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e4b36-a641-4e57-af0e-1d87664e711e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:36.903338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:36.905620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:36.906713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:36.907241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:36.951490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:36.953502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:36.964728 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 874/1767] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 09:56:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:36.973438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:36.997943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47053de9-c3dd-4356-b0cc-9c2c9dbff0a8 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:37.009239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:37.010038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:37.010415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:37.010719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:37.011457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:37.015071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:37.019040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:37.060607 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 875/1768] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:37.060607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:37.066662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:37.069659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:37.090129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e59ec9-9e07-4714-ba11-9aa107e386f0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:37.100440 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 894/1769] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:37.102439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:37.105423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:37.106093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:37.106508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:37.106886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:37.107865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:37.109552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:37.126785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b767809-8149-4ddb-8831-6d2a04bd7947 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:37.141928 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 876/1770] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:37.144410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:37.160084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:37.164562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:37.166444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:37.170932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:37.171898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:37.174604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:37.176740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a0df6-a09e-4769-aa30-c62cb9e55b5a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:37.195229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:37.199346 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 895/1771] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:37.204844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:37.206896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:37.207321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:37.210009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] PATH_INFO: `/v3/projects/c6877dc065e44ab4a5ff07428f46867f/users/db4b607d5bc940d28d3c3b9edb6ef4a2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:37.212484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6877dc065e44ab4a5ff07428f46867f, user_id=db4b607d5bc940d28d3c3b9edb6ef4a2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:37.216398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:37.247486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:37.271334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ece710-7761-46e3-8165-f722bd7d6098 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:37.272750 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 877/1772] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:37 2022] PUT /identity/v3/projects/c6877dc065e44ab4a5ff07428f46867f/users/db4b607d5bc940d28d3c3b9edb6ef4a2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:37.274401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:37.475110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a5cda-e6fb-45f5-952a-c5c913ab3bc6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:37.490611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916fc28-fa9a-435f-a684-754009cf4403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:37.491316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916fc28-fa9a-435f-a684-754009cf4403 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:37.491625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916fc28-fa9a-435f-a684-754009cf4403 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:37.491923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916fc28-fa9a-435f-a684-754009cf4403 admin admin] PATH_INFO: `/v3/projects/9f08504febac4265b33fed0f12095b45/users/4b6ee505ea6345f4bf22f29cc5f359f4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:37.529238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:37.530231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:37.530646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:37.531053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] PATH_INFO: `/v3/projects/20bd39a91cb44b129e1179aea3aa4905/users/43eb9a62487d44f4862315c996a22eb6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:37.542720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916fc28-fa9a-435f-a684-754009cf4403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f08504febac4265b33fed0f12095b45, user_id=4b6ee505ea6345f4bf22f29cc5f359f4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:37.552734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916fc28-fa9a-435f-a684-754009cf4403 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:37.596357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 878/1773] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:37 2022] PUT /identity/v3/projects/9f08504febac4265b33fed0f12095b45/users/4b6ee505ea6345f4bf22f29cc5f359f4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:37.598028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d520ac-e885-478b-9ebf-3c3e261e967d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:37.598315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d520ac-e885-478b-9ebf-3c3e261e967d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:37.598587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d520ac-e885-478b-9ebf-3c3e261e967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:37.618917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bd39a91cb44b129e1179aea3aa4905, user_id=43eb9a62487d44f4862315c996a22eb6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:37.629948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:37.692379 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 896/1774] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:37 2022] PUT /identity/v3/projects/20bd39a91cb44b129e1179aea3aa4905/users/43eb9a62487d44f4862315c996a22eb6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:37.707693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:37.786293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c3d520ac-e885-478b-9ebf-3c3e261e967d None None] MFA Rules not processed for user `db4b607d5bc940d28d3c3b9edb6ef4a2`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:37.787865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c3d520ac-e885-478b-9ebf-3c3e261e967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:38.031535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cffb7-a3d7-4d7c-9476-dab4a7b78c8c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:38.062023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:38.062647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.063002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.063170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.063873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:38.083649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:38.131018 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 879/1775] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:38.132834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d520ac-e885-478b-9ebf-3c3e261e967d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:38.146369 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 897/1776] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:38.151675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:38.161771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d520ac-e885-478b-9ebf-3c3e261e967d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:38.169661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:38.170188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfd977-4b26-48e8-9f61-fd9a0b007e3f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:38.171271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.172335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.174149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.175535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:38.178636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:38.188067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:38.188979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.189263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.189536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] PATH_INFO: `/v3/projects/9f08504febac4265b33fed0f12095b45/users/4b6ee505ea6345f4bf22f29cc5f359f4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.194137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f08504febac4265b33fed0f12095b45, user_id=4b6ee505ea6345f4bf22f29cc5f359f4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:38.197263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:38.222632 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 880/1777] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:38.226827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:38.247399 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 898/1778] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:38 2022] PUT /identity/v3/projects/9f08504febac4265b33fed0f12095b45/users/4b6ee505ea6345f4bf22f29cc5f359f4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:38.248355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:38.287455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:38.374290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba2aa4-ca26-44c8-826b-a6618e389eaf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:38.458337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc636dea-b8a7-4c08-bc4d-a70f2d389b5c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:38.472535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87187a-2c81-467b-ab61-dcb2c82ac5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:38.475667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87187a-2c81-467b-ab61-dcb2c82ac5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.477164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87187a-2c81-467b-ab61-dcb2c82ac5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.477458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87187a-2c81-467b-ab61-dcb2c82ac5d7 admin admin] PATH_INFO: `/v3/projects/20bd39a91cb44b129e1179aea3aa4905/users/43eb9a62487d44f4862315c996a22eb6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.482414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87187a-2c81-467b-ab61-dcb2c82ac5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bd39a91cb44b129e1179aea3aa4905, user_id=43eb9a62487d44f4862315c996a22eb6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:38.489692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87187a-2c81-467b-ab61-dcb2c82ac5d7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:38.530619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 899/1779] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:38 2022] PUT /identity/v3/projects/20bd39a91cb44b129e1179aea3aa4905/users/43eb9a62487d44f4862315c996a22eb6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:38.530843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06735c3-877d-4623-81c8-4d902c3bc20f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:38.530843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06735c3-877d-4623-81c8-4d902c3bc20f service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.530843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06735c3-877d-4623-81c8-4d902c3bc20f service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.530843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06735c3-877d-4623-81c8-4d902c3bc20f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.531143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cf763-7f96-4f39-bc23-d8f8becfdc04 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.531143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cf763-7f96-4f39-bc23-d8f8becfdc04 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.531143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cf763-7f96-4f39-bc23-d8f8becfdc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.539252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:38.614781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-577cf763-7f96-4f39-bc23-d8f8becfdc04 None None] MFA Rules not processed for user `4b6ee505ea6345f4bf22f29cc5f359f4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:38.621524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-577cf763-7f96-4f39-bc23-d8f8becfdc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:38.740873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06735c3-877d-4623-81c8-4d902c3bc20f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:38.744046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06735c3-877d-4623-81c8-4d902c3bc20f service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:38.774792 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 881/1780] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:38.776297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.776581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.776887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.809870 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 900/1781] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:38 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:38.814070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b5593-42cf-43b3-94c8-4ceed49ca421 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:38.814365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b5593-42cf-43b3-94c8-4ceed49ca421 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:38.814640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b5593-42cf-43b3-94c8-4ceed49ca421 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:38.817718 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 901/1782] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:38.826403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b5593-42cf-43b3-94c8-4ceed49ca421 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:38.828468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e75b5593-42cf-43b3-94c8-4ceed49ca421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:38.892142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] MFA Rules not processed for user `43eb9a62487d44f4862315c996a22eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:38.894106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:39.040969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:39.086481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b5593-42cf-43b3-94c8-4ceed49ca421 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:39.098060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4e5ce-bb0b-4842-9681-ecde59a5e3f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:39.098867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4e5ce-bb0b-4842-9681-ecde59a5e3f6 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:39.099206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4e5ce-bb0b-4842-9681-ecde59a5e3f6 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:39.099990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4e5ce-bb0b-4842-9681-ecde59a5e3f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:39.123926 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 882/1783] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:39.133670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:39.135159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4e5ce-bb0b-4842-9681-ecde59a5e3f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:39.138279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4e5ce-bb0b-4842-9681-ecde59a5e3f6 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:39.155013 ncdv-master-1 devstack@keystone.service[104131]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 27 09:56:39.155444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:39.174610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 902/1784] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:38 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:39.176830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640db9f-0fee-467d-9e96-2441a76cafe3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:39.181072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640db9f-0fee-467d-9e96-2441a76cafe3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:39.181806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640db9f-0fee-467d-9e96-2441a76cafe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:39.316288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e640db9f-0fee-467d-9e96-2441a76cafe3 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:39.317733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e640db9f-0fee-467d-9e96-2441a76cafe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:39.320190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0376e6-098d-4624-96a1-bb9ab93bfca0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:39.339326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:39.340076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:39.340450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:39.344217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:39.349161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:39.352420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:39.361973 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 883/1785] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:39 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:39.452512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:39.479981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41548-442a-4d43-a819-166a4aae96d3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:39.497841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:39.498742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:39.499486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:39.499865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:39.505494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 903/1786] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Sep 27 09:56:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:39.524792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:39.526677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:39.556603 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 884/1787] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:39.657419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640db9f-0fee-467d-9e96-2441a76cafe3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:39.680963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640db9f-0fee-467d-9e96-2441a76cafe3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:39.707701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:39.708705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:39.709909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:39.710394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:39.714219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:39.715855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:39.740529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:39.767020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc7c1f-660f-49b0-a203-06749d5abcee service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:39.779551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbc16b-be66-427d-9762-23f8da4d4730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:39.780427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc16b-be66-427d-9762-23f8da4d4730 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:39.780820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc16b-be66-427d-9762-23f8da4d4730 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:39.781150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc16b-be66-427d-9762-23f8da4d4730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:39.805385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbc16b-be66-427d-9762-23f8da4d4730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:39.807525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbc16b-be66-427d-9762-23f8da4d4730 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:39.830119 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 885/1788] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:39.874541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:39.894681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:39.942626 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 904/1789] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:39 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:39.990513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540529c-67a3-4c81-93c2-b2a0fc7c76ad None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:39.990513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540529c-67a3-4c81-93c2-b2a0fc7c76ad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:39.990781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540529c-67a3-4c81-93c2-b2a0fc7c76ad None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:39.992881 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 886/1790] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:40.021628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:40.048726 ncdv-master-1 devstack@keystone.service[104132]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 27 09:56:40.049349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:40.085743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0389-69c3-4bd6-a314-a24ec81ce219 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:40.116134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540529c-67a3-4c81-93c2-b2a0fc7c76ad None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:40.118599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7f18-36ba-4205-8326-4716a39aa97f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:40.119575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7f18-36ba-4205-8326-4716a39aa97f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:40.120234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7f18-36ba-4205-8326-4716a39aa97f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:40.120593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7f18-36ba-4205-8326-4716a39aa97f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:40.122696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7f18-36ba-4205-8326-4716a39aa97f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:40.125602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7f18-36ba-4205-8326-4716a39aa97f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:40.142552 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 905/1791] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:40 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:40.160340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540529c-67a3-4c81-93c2-b2a0fc7c76ad None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:40.183918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:40.184940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:40.185275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:40.185718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:40.355605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff4e31-198b-4861-b2f7-9f84135f0392 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:40.355893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff4e31-198b-4861-b2f7-9f84135f0392 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:40.356246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff4e31-198b-4861-b2f7-9f84135f0392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:40.359142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:40.361634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:40.405855 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 887/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:40.405966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-26ff4e31-198b-4861-b2f7-9f84135f0392 None None] MFA Rules not processed for user `862f94043f044a25b740db652bf3dc02`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:40.408635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-26ff4e31-198b-4861-b2f7-9f84135f0392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:40.583571 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 906/1793] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:40.605381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:40.629219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff4e31-198b-4861-b2f7-9f84135f0392 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:40.656971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff4e31-198b-4861-b2f7-9f84135f0392 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:40.670961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:40.671995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:40.673157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:40.674494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:40.692146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:40.694272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:40.724746 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 907/1794] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:40.752976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:40.778005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915bf53-659b-46f6-a0f1-be8ad3caa4b7 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:40.787335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:40.787964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:40.788190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:40.788494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] PATH_INFO: `/v3/users/8daac732ab214a56a401a3487f2b820c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:40.792473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8daac732ab214a56a401a3487f2b820c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:40.794550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:40.990595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8daac732ab214a56a401a3487f2b820c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:40.993122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8daac732ab214a56a401a3487f2b820c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.013292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:41.024403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8daac732ab214a56a401a3487f2b820c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.027510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75050537-5c57-4378-a0a8-420b65ebe0e1 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:41.039147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:41.039947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:41.040230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:41.040553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] PATH_INFO: `/v3/users/4aaff47516a241e9aa9ad5e2007429e0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:41.042205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aaff47516a241e9aa9ad5e2007429e0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:41.047664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:41.064328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8daac732ab214a56a401a3487f2b820c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.098287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86fce77c-7897-49c3-942b-87b0347e18a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8daac732ab214a56a401a3487f2b820c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.106530 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 888/1795] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:40 2022] DELETE /identity/v3/users/8daac732ab214a56a401a3487f2b820c => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:41.112020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b2c06-8973-4d0b-b2eb-495d19cd060b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:41.112493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b2c06-8973-4d0b-b2eb-495d19cd060b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:41.112976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b2c06-8973-4d0b-b2eb-495d19cd060b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:41.182077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aaff47516a241e9aa9ad5e2007429e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.183223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aaff47516a241e9aa9ad5e2007429e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.190737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-897b2c06-8973-4d0b-b2eb-495d19cd060b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:41.194282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-897b2c06-8973-4d0b-b2eb-495d19cd060b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:41.199949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aaff47516a241e9aa9ad5e2007429e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.212025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aaff47516a241e9aa9ad5e2007429e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.214067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12e2800f-a212-4256-87e8-c317e4722c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aaff47516a241e9aa9ad5e2007429e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:41.255963 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 908/1796] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:41 2022] DELETE /identity/v3/users/4aaff47516a241e9aa9ad5e2007429e0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:56:41.260773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:41.261110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:41.261447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:41.285387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:41.334032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:41.337010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:41.358688 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 889/1797] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:41.361328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67b09-b6d7-496f-9dfe-edaceafd3076 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:41.362752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67b09-b6d7-496f-9dfe-edaceafd3076 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:41.363325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67b09-b6d7-496f-9dfe-edaceafd3076 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:41.388937 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 890/1798] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:41 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:41.390614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc67b09-b6d7-496f-9dfe-edaceafd3076 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:41.392808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ecc67b09-b6d7-496f-9dfe-edaceafd3076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:41.457017 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 909/1799] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:41.458256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:41.473143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:41.537298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc67b09-b6d7-496f-9dfe-edaceafd3076 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:41.550399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:41.551048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:41.551333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:41.551956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:41.576172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:41.581030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:41.623287 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 891/1800] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:41.652202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:41.701557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:41.707112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2bc7f-7b2f-4fcb-8c79-f9fefe9a33c9 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:41.709570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:41.721121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc8cf8-0081-48cc-b36e-ed27074a3d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:41.721953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc8cf8-0081-48cc-b36e-ed27074a3d02 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:41.722204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc8cf8-0081-48cc-b36e-ed27074a3d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:41.722551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc8cf8-0081-48cc-b36e-ed27074a3d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:41.748244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc8cf8-0081-48cc-b36e-ed27074a3d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:41.754402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc8cf8-0081-48cc-b36e-ed27074a3d02 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:41.778802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 892/1801] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:41.867777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c9fc4-6031-42af-9f9e-93b07eeade0d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:41.881786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:41.884442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:41.885341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:41.886430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:41.888241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:41.902511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:41.908049 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 910/1802] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:41 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:42.131208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b08f36-3f94-49aa-9d0b-8340adb03bce None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:42.131495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b08f36-3f94-49aa-9d0b-8340adb03bce None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:42.131940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b08f36-3f94-49aa-9d0b-8340adb03bce None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:42.134801 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 893/1803] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:42.150345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:42.152151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:42.198710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:42.222277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabbd3-e48b-43a1-9fc8-bc58de3983a4 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:42.247487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:42.248219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:42.248507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:42.248812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:42.252173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:42.254306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:42.268911 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 911/1804] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:42 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:42.366041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c39c2-8c91-481e-b729-fd5ad390dfb9 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:42.366041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c39c2-8c91-481e-b729-fd5ad390dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:42.366679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c39c2-8c91-481e-b729-fd5ad390dfb9 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:42.368446 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 894/1805] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:42.374964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:42.405857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:42.435252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dab548-b09c-4ef9-962d-fb628b932818 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:42.456960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd20f4-ac12-421d-a6aa-7232ba6ff2f3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:42.457855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd20f4-ac12-421d-a6aa-7232ba6ff2f3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:42.457962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd20f4-ac12-421d-a6aa-7232ba6ff2f3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:42.458161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd20f4-ac12-421d-a6aa-7232ba6ff2f3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:42.459455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd20f4-ac12-421d-a6aa-7232ba6ff2f3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:42.462507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd20f4-ac12-421d-a6aa-7232ba6ff2f3 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:42.474987 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 912/1806] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:42 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:42.667342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c39c2-8c91-481e-b729-fd5ad390dfb9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:42.668312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:42.669545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:42.670040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:42.673939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f5c39c2-8c91-481e-b729-fd5ad390dfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:42.719068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:42.745488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c39c2-8c91-481e-b729-fd5ad390dfb9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:42.762203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] MFA Rules not processed for user `862f94043f044a25b740db652bf3dc02`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:42.770806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:42.776971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:42.780358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:42.783770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:42.784253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] PATH_INFO: `/v3/projects/246bd38c478e448cab936abd93fa094e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:42.797029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=246bd38c478e448cab936abd93fa094e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:42.805553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:42.870580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:42.979051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246bd38c478e448cab936abd93fa094e', 'request_id': 'req-94f97312-da2c-494b-a97e-7e230e23d717'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:42.982512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 913/1807] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:42.983661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:42.990934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:42.994684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246bd38c478e448cab936abd93fa094e', 'request_id': 'req-94f97312-da2c-494b-a97e-7e230e23d717'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.012934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd8e6-c8ca-462e-a10d-d8f237c1012a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:43.015689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246bd38c478e448cab936abd93fa094e', 'request_id': 'req-94f97312-da2c-494b-a97e-7e230e23d717'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.034957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] The token cache is being invalidate because project 246bd38c478e448cab936abd93fa094e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:43.036129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.037522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.047288 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 895/1808] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:42 2022] DELETE /identity/v3/projects/246bd38c478e448cab936abd93fa094e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:43.048475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:43.051175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:43.209455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:43.210780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:43.210780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:43.210780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] PATH_INFO: `/v3/projects/b0bb608f923d4e66be0ac6d38e345721` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:43.215885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0bb608f923d4e66be0ac6d38e345721)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:43.221980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:43.257791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f97312-da2c-494b-a97e-7e230e23d717 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:43.287460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:43.288143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:43.288469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:43.290572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] PATH_INFO: `/v3/users/4b6ee505ea6345f4bf22f29cc5f359f4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:43.293947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6ee505ea6345f4bf22f29cc5f359f4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:43.302245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:43.373429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0bb608f923d4e66be0ac6d38e345721', 'request_id': 'req-b0e1f993-3261-4c92-86bf-dff0d8c7f339'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.387270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0bb608f923d4e66be0ac6d38e345721', 'request_id': 'req-b0e1f993-3261-4c92-86bf-dff0d8c7f339'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.405543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0bb608f923d4e66be0ac6d38e345721', 'request_id': 'req-b0e1f993-3261-4c92-86bf-dff0d8c7f339'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.430164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] The token cache is being invalidate because project b0bb608f923d4e66be0ac6d38e345721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:43.430461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.432286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.440822 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 914/1809] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:42 2022] DELETE /identity/v3/projects/b0bb608f923d4e66be0ac6d38e345721 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:43.455737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:43.457311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:43.461631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6ee505ea6345f4bf22f29cc5f359f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.463907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6ee505ea6345f4bf22f29cc5f359f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.484539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6ee505ea6345f4bf22f29cc5f359f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.501962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6ee505ea6345f4bf22f29cc5f359f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.526209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ff5477b-bcbf-47aa-8041-9c0041193495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6ee505ea6345f4bf22f29cc5f359f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.540147 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 896/1810] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:43 2022] DELETE /identity/v3/users/4b6ee505ea6345f4bf22f29cc5f359f4 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:43.615355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:43.620718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1f993-3261-4c92-86bf-dff0d8c7f339 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:43.786595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:43.788802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:43.791403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:43.792709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] PATH_INFO: `/v3/users/43eb9a62487d44f4862315c996a22eb6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:43.796821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43eb9a62487d44f4862315c996a22eb6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:43.810664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:43.946700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172de65-ab2f-493a-a841-4ef4ffb1f360 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:43.947086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172de65-ab2f-493a-a841-4ef4ffb1f360 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:43.947464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172de65-ab2f-493a-a841-4ef4ffb1f360 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:43.949341 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 897/1811] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:56:43.950603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:43.950960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:43.951467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:43.953475 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 898/1812] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:43.959703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43eb9a62487d44f4862315c996a22eb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.960599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43eb9a62487d44f4862315c996a22eb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.969557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:43.971584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:43.971801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43eb9a62487d44f4862315c996a22eb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.988303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:43.997078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43eb9a62487d44f4862315c996a22eb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:43.998864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74af28a-1775-4311-9926-4a430e00aa7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43eb9a62487d44f4862315c996a22eb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:44.034944 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 915/1813] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:43 2022] DELETE /identity/v3/users/43eb9a62487d44f4862315c996a22eb6 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:44.180421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:44.189611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:44.229334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:44.229939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:44.229939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:44.270360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:44.271827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:44.393678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96679f07-ff2c-4d7b-b651-043e8ccc18c2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:44.434880 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 916/1814] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:56:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:44.594215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:44.595130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:44.595130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:44.595518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:44.598003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:44.618783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:44.621165 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 899/1815] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:43 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 652 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:44.818645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:44.820478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:44.877988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:44.891445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:44.909388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552372b0-5f84-4e8d-9d69-46cec38ac86c tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:44.935502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09849652-cab9-4bd8-a1c0-c08a12773598 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:44.936087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09849652-cab9-4bd8-a1c0-c08a12773598 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:44.936412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09849652-cab9-4bd8-a1c0-c08a12773598 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:44.936686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09849652-cab9-4bd8-a1c0-c08a12773598 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:44.965219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09849652-cab9-4bd8-a1c0-c08a12773598 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:44.969250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09849652-cab9-4bd8-a1c0-c08a12773598 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:45.154338 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 900/1816] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:45.189967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac16f06-4a7e-409f-83bb-fc054b5129fa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:45.206754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:45.207594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.207943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.208365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] PATH_INFO: `/v3/projects/9f08504febac4265b33fed0f12095b45` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.214053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f08504febac4265b33fed0f12095b45)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:45.216326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:45.384240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f08504febac4265b33fed0f12095b45', 'request_id': 'req-96205d23-bcc9-4933-9817-bcad48d56ef8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:45.393969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97de7e5-95ac-45cf-ade4-b5e45633d137 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.394608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97de7e5-95ac-45cf-ade4-b5e45633d137 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.395037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97de7e5-95ac-45cf-ade4-b5e45633d137 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.398474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 901/1817] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:45.420840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f08504febac4265b33fed0f12095b45', 'request_id': 'req-96205d23-bcc9-4933-9817-bcad48d56ef8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:45.442729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f08504febac4265b33fed0f12095b45', 'request_id': 'req-96205d23-bcc9-4933-9817-bcad48d56ef8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:45.444438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97de7e5-95ac-45cf-ade4-b5e45633d137 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:45.478945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] The token cache is being invalidate because project 9f08504febac4265b33fed0f12095b45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:45.479314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:45.480258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-96205d23-bcc9-4933-9817-bcad48d56ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:45.488582 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 917/1818] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:44 2022] DELETE /identity/v3/projects/9f08504febac4265b33fed0f12095b45 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:45.494015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566c593-a34e-4939-8e95-dd7eead70aa7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.494278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566c593-a34e-4939-8e95-dd7eead70aa7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.494757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566c593-a34e-4939-8e95-dd7eead70aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.544905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c566c593-a34e-4939-8e95-dd7eead70aa7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:45.550324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c566c593-a34e-4939-8e95-dd7eead70aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:45.585448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:45.677373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97de7e5-95ac-45cf-ade4-b5e45633d137 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:45.717250 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 918/1819] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:45.727535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c756aa-eea3-4d7c-a058-10ec2d74aaeb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.728644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c756aa-eea3-4d7c-a058-10ec2d74aaeb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.730130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c756aa-eea3-4d7c-a058-10ec2d74aaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.767408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-56c756aa-eea3-4d7c-a058-10ec2d74aaeb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:45.771065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-56c756aa-eea3-4d7c-a058-10ec2d74aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:45.787151 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 919/1820] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:45.793720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c756aa-eea3-4d7c-a058-10ec2d74aaeb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:45.823496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c756aa-eea3-4d7c-a058-10ec2d74aaeb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:45.836866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:45.838302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.838973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.841004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.843368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:45.843906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:45.846976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:45.870772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97de7e5-95ac-45cf-ade4-b5e45633d137 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:45.886100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 920/1821] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:45.886750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:45.888788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.889000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:45.889588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.890906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.896229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:45.902111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:45.908390 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 902/1822] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:45 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:45.911236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2aa1d-5cfb-42e7-a03a-50e2fcb57d63 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:45.917538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a05506-b545-4777-b27e-cda0b0b7c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:45.918534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05506-b545-4777-b27e-cda0b0b7c8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.919183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05506-b545-4777-b27e-cda0b0b7c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.919399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05506-b545-4777-b27e-cda0b0b7c8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.920487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a05506-b545-4777-b27e-cda0b0b7c8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:45.923100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a05506-b545-4777-b27e-cda0b0b7c8b6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:45.968843 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 921/1823] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:45 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:45.973184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:45.986641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:45.987324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:45.987740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:45.993114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b4530-4fb5-488a-91f0-81bd3fb6e685 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:45.995536 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 922/1824] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:46.001736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:46.004704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:46.020394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:46.144185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:46.145495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:46.148607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:46.153438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.153957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.154765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.155747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:46.158821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:46.197540 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 903/1825] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:45 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:56:46.202844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:46.223252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de95544-5b79-46f6-b420-59c4535f92ff admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:46.229902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:46.231231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.231977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.232522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.233721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:46.236787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:46.259941 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 904/1826] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:46.261520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:46.275708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56217175-56b5-4600-aa15-a32cc3976746 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:46.282070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:46.283152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.283628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.284264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.285173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:46.287375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:46.296512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f906-bb9b-48e0-9239-5767de426602 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:46.309985 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 905/1827] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:46.311509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:46.313970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:46.325516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:46.327516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.330161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.330625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.333310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e0ca6-5e16-4115-83b7-65eac1bcd181 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:46.334631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:46.339971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:46.341052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.341707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.342356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] PATH_INFO: `/v3/projects/20bd39a91cb44b129e1179aea3aa4905` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.344842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bd39a91cb44b129e1179aea3aa4905)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:46.345812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:46.347494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:46.356703 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 923/1828] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:46 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:46.358450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:46.398693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff618d-d6de-4320-83fc-f6a226be6714 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:46.431981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2a561-40b4-42cb-9be9-69810b30d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:46.433100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2a561-40b4-42cb-9be9-69810b30d71e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.433433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2a561-40b4-42cb-9be9-69810b30d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.433806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2a561-40b4-42cb-9be9-69810b30d71e admin admin] PATH_INFO: `/v3/projects/3a28ae320ed0403c98477279023398bd/users/c7ff8a626cd5460295136c2de34f841c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.481916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2a561-40b4-42cb-9be9-69810b30d71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a28ae320ed0403c98477279023398bd, user_id=c7ff8a626cd5460295136c2de34f841c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:46.489326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2a561-40b4-42cb-9be9-69810b30d71e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:46.497732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:46.533330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 924/1829] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:46 2022] PUT /identity/v3/projects/3a28ae320ed0403c98477279023398bd/users/c7ff8a626cd5460295136c2de34f841c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:46.536125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bd39a91cb44b129e1179aea3aa4905', 'request_id': 'req-45b052ec-792b-42dd-aae0-f1896c0f7335'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:46.537880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.540236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.540814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.546415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bd39a91cb44b129e1179aea3aa4905', 'request_id': 'req-45b052ec-792b-42dd-aae0-f1896c0f7335'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:46.559893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bd39a91cb44b129e1179aea3aa4905', 'request_id': 'req-45b052ec-792b-42dd-aae0-f1896c0f7335'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:46.573055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] The token cache is being invalidate because project 20bd39a91cb44b129e1179aea3aa4905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:46.573346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:46.574025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45b052ec-792b-42dd-aae0-f1896c0f7335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:46.584159 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 906/1830] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:46 2022] DELETE /identity/v3/projects/20bd39a91cb44b129e1179aea3aa4905 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:46.586253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9b12-6d36-4234-8f44-1521716689e7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.586536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9b12-6d36-4234-8f44-1521716689e7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.586822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9b12-6d36-4234-8f44-1521716689e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.622498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] MFA Rules not processed for user `c7ff8a626cd5460295136c2de34f841c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:46.623825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:46.626383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e00a9b12-6d36-4234-8f44-1521716689e7 None None] MFA Rules not processed for user `862f94043f044a25b740db652bf3dc02`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:46.632350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e00a9b12-6d36-4234-8f44-1521716689e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:46.822351 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 907/1831] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:46.824953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dee595-535c-4297-a83a-8141d24973b0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.825502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dee595-535c-4297-a83a-8141d24973b0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.826070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dee595-535c-4297-a83a-8141d24973b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.839861 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 925/1832] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:46.840913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:46.846426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:46.858492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:46.897899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a6dee595-535c-4297-a83a-8141d24973b0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:46.902883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a6dee595-535c-4297-a83a-8141d24973b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:46.958670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd6da-f497-468e-aa7e-7a28621f6f3d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:46.964959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:46.965907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:46.966332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:46.966871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:46.968450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:46.970770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.049479 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 926/1833] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:46 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:47.051534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.069728 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 908/1834] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:47.080047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae6962-20c8-42a8-ae55-238cdb1cb304 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.082359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae6962-20c8-42a8-ae55-238cdb1cb304 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.082851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae6962-20c8-42a8-ae55-238cdb1cb304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.166029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3bae6962-20c8-42a8-ae55-238cdb1cb304 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:47.168586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3bae6962-20c8-42a8-ae55-238cdb1cb304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:47.245022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d795b-d794-4e7c-9934-50d18077810d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.268138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.269002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.269360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.269750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.270741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.277913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.346005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 927/1835] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:47 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:47.347609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.373937 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 909/1836] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:47.375157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae6962-20c8-42a8-ae55-238cdb1cb304 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.377854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385646b-cdfd-4c43-8014-cd4a5047c426 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.397413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.399027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.400530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.401161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.402204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.403944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae6962-20c8-42a8-ae55-238cdb1cb304 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.408341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.414819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.416322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.416630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.417012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.420793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:47.421235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.424378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.448732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:47.450667 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 928/1837] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:47.453044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.473455 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 910/1838] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:47.476515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.488240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f0815-507a-46b7-9150-b39debb36395 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.505327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.506676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.508485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.509085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] PATH_INFO: `/v3/projects/db2605a6df2e44a5ab5c838febd8df19/users/83828eb232f74e0092746711eb78977b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.518750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a53ec-70cb-422c-ad7c-47ca32b0d14e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.540249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.541376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.541376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.541615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.542241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.545646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.574924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2605a6df2e44a5ab5c838febd8df19, user_id=83828eb232f74e0092746711eb78977b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.583545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.634195 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 929/1839] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:47 2022] PUT /identity/v3/projects/db2605a6df2e44a5ab5c838febd8df19/users/83828eb232f74e0092746711eb78977b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:47.635808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.666061 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 911/1840] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:47 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:47.668674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.734426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:47.755556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32108309-df33-4ad3-8d29-32d5c42a4ab5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.853474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8092c8-aca6-4052-a580-59c4be83e97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.855979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8092c8-aca6-4052-a580-59c4be83e97a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.856472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8092c8-aca6-4052-a580-59c4be83e97a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.857014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8092c8-aca6-4052-a580-59c4be83e97a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.858239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8092c8-aca6-4052-a580-59c4be83e97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.861289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8092c8-aca6-4052-a580-59c4be83e97a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.878188 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 930/1841] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:47.883429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc97305-94f5-4d2c-a633-fa1cb4c02774 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.883640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f390f-0647-458d-b1a3-4125c60d86a0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.884133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc97305-94f5-4d2c-a633-fa1cb4c02774 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.884133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc97305-94f5-4d2c-a633-fa1cb4c02774 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.886088 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 931/1842] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:47.887207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc97305-94f5-4d2c-a633-fa1cb4c02774 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:47.889991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e283b14-f4e1-4ea1-b8bb-d63ec450f161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.890785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e283b14-f4e1-4ea1-b8bb-d63ec450f161 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.891135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e283b14-f4e1-4ea1-b8bb-d63ec450f161 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.891749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e283b14-f4e1-4ea1-b8bb-d63ec450f161 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.892560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e283b14-f4e1-4ea1-b8bb-d63ec450f161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.896622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e283b14-f4e1-4ea1-b8bb-d63ec450f161 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.908894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc97305-94f5-4d2c-a633-fa1cb4c02774 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:47.916388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:47.917295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.917640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.917976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] PATH_INFO: `/v3/projects/db2605a6df2e44a5ab5c838febd8df19/users/83828eb232f74e0092746711eb78977b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:47.921684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2605a6df2e44a5ab5c838febd8df19, user_id=83828eb232f74e0092746711eb78977b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:47.925817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:47.953953 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 932/1843] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:47 2022] PUT /identity/v3/projects/db2605a6df2e44a5ab5c838febd8df19/users/83828eb232f74e0092746711eb78977b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:47.957591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:56:47.959738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:47.982084 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 912/1844] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:47 2022] POST /identity/v3/users => generated 294 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:47.984076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a546d3-2a52-45f0-9022-5d7d40d63ae7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:47.984394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a546d3-2a52-45f0-9022-5d7d40d63ae7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:47.984845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a546d3-2a52-45f0-9022-5d7d40d63ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.084291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-58a546d3-2a52-45f0-9022-5d7d40d63ae7 None None] MFA Rules not processed for user `83828eb232f74e0092746711eb78977b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:48.085967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-58a546d3-2a52-45f0-9022-5d7d40d63ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:48.195999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:48.203799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:48.234840 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 913/1845] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:48.237675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a546d3-2a52-45f0-9022-5d7d40d63ae7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:48.339658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:48.362786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a546d3-2a52-45f0-9022-5d7d40d63ae7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:48.393724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:48.394448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:48.394644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:48.394997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.395553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:48.397843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:48.410980 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 914/1846] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:48.411890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:48.414813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:48.433359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57936ddc-dd64-487b-8d4f-f1e718b730c8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:48.435942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97116616-b982-41b9-ab41-5715207b8636 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:48.443255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:48.444021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:48.444192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:48.444667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.445428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db4b607d5bc940d28d3c3b9edb6ef4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6877dc065e44ab4a5ff07428f46867f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:48.447477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:48.448868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:48.449950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.452960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorizing `identity:get_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:48.456887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:48.459016 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 933/1847] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:56:47 2022] GET /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 383 bytes in 504 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:48.459914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:48.483208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66322c5-57c5-49dc-a642-016264a87fcc tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:48.489029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:48.490942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:48.498881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:48.501192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:48.501852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:48.502234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.504732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:48.506276 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 915/1848] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:48.520472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:48.581790 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 934/1849] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:48.589391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:48.628398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180746-48d2-422c-9310-89c748fddac7 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:48.657455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e5671-0936-4f16-9b7d-827a9fea3a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:48.660292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e5671-0936-4f16-9b7d-827a9fea3a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:48.661841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e5671-0936-4f16-9b7d-827a9fea3a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:48.664885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e5671-0936-4f16-9b7d-827a9fea3a56 admin admin] PATH_INFO: `/v3/projects/56f793358f224ff98e1f6bc0e95309c5/users/13c5c05dd6b244dd8ff15ca2c1ae298a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.749176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e5671-0936-4f16-9b7d-827a9fea3a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f793358f224ff98e1f6bc0e95309c5, user_id=13c5c05dd6b244dd8ff15ca2c1ae298a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:48.753184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:48.755683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:48.761623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e5671-0936-4f16-9b7d-827a9fea3a56 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:48.778309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:48.783240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04262623-2696-4158-b3ed-7d575324c3df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:48.797459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:48.798476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:48.798815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:48.799160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] PATH_INFO: `/v3/users/862f94043f044a25b740db652bf3dc02` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.807661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=862f94043f044a25b740db652bf3dc02)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:48.809537 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 916/1850] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:48 2022] PUT /identity/v3/projects/56f793358f224ff98e1f6bc0e95309c5/users/13c5c05dd6b244dd8ff15ca2c1ae298a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:48.812424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:48.823779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:48.824412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:48.825396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:48.927149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '862f94043f044a25b740db652bf3dc02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:48.928286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '862f94043f044a25b740db652bf3dc02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:48.950077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '862f94043f044a25b740db652bf3dc02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:48.965385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '862f94043f044a25b740db652bf3dc02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:48.966567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c203dc77-91a3-4739-9b86-ca0b98f55a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '862f94043f044a25b740db652bf3dc02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:48.973136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] MFA Rules not processed for user `13c5c05dd6b244dd8ff15ca2c1ae298a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:48.974648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.010148 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 935/1851] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:48 2022] DELETE /identity/v3/users/862f94043f044a25b740db652bf3dc02 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:49.017741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.018237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.018523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.053922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:49.060421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.161385 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 917/1852] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:49.166593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.170787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.199075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 936/1853] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:49.217231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.219491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.295011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b84e2-a78f-48ac-904c-38e729438d97 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:49.300351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf55fd1-eb47-4ad6-a04f-46e5e5d29db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:49.301151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55fd1-eb47-4ad6-a04f-46e5e5d29db8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.301537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55fd1-eb47-4ad6-a04f-46e5e5d29db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.302513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55fd1-eb47-4ad6-a04f-46e5e5d29db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.303020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf55fd1-eb47-4ad6-a04f-46e5e5d29db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:49.305656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf55fd1-eb47-4ad6-a04f-46e5e5d29db8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:49.340513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8c9a-c6c6-4e30-85e3-e07c23ede56e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:49.345723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:49.346676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.347118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.347801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.352884 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 918/1854] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:49 2022] POST /identity/v3/projects => generated 389 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:49.354663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a40786-bae2-48fa-92fd-8d3586b2a2eb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.354911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a40786-bae2-48fa-92fd-8d3586b2a2eb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.355262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a40786-bae2-48fa-92fd-8d3586b2a2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.366842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:49.368582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:49.387893 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 937/1855] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:49.389777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.392172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.422664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c29b9d-7c11-4223-9b97-a442eb752f41 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:49.425497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-40a40786-bae2-48fa-92fd-8d3586b2a2eb None None] MFA Rules not processed for user `5cc03c5c36f14c389cb76fa0b4dc78e0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:49.427159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-40a40786-bae2-48fa-92fd-8d3586b2a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.451625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:49.456379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:49.553202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:49.555595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.556305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.556935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] PATH_INFO: `/v3/users/c7ff8a626cd5460295136c2de34f841c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.559083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ff8a626cd5460295136c2de34f841c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:49.563008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:49.591886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 919/1856] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:49 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:49.592646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a40786-bae2-48fa-92fd-8d3586b2a2eb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.615234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a40786-bae2-48fa-92fd-8d3586b2a2eb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:49.623747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:49.624617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.625017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.625280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.625929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:49.628929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:49.653045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ff8a626cd5460295136c2de34f841c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:49.653838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ff8a626cd5460295136c2de34f841c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:49.661178 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 920/1857] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:49 2022] POST /identity/v3/users => generated 301 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:49.664410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ff8a626cd5460295136c2de34f841c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:49.665155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.668060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.676028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ff8a626cd5460295136c2de34f841c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:49.678122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ff8a626cd5460295136c2de34f841c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:49.713113 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 938/1858] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:49 2022] DELETE /identity/v3/users/c7ff8a626cd5460295136c2de34f841c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:49.714815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.716967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.812326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785e31-7c61-401c-a9b3-f8a9f15ce398 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:49.823660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bdf3-e214-40c7-b481-66915233cceb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:49.830294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bf5ef-ea10-489a-b079-b09a88b117d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:49.831246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bf5ef-ea10-489a-b079-b09a88b117d4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.831734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bf5ef-ea10-489a-b079-b09a88b117d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.832009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bf5ef-ea10-489a-b079-b09a88b117d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.833000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bf5ef-ea10-489a-b079-b09a88b117d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:49.835327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bf5ef-ea10-489a-b079-b09a88b117d4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:49.860742 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 939/1859] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:49.862368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd82c3-9ef2-4291-b05d-ed5a4762b031 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.862662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd82c3-9ef2-4291-b05d-ed5a4762b031 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.862940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd82c3-9ef2-4291-b05d-ed5a4762b031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.894899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1ccd82c3-9ef2-4291-b05d-ed5a4762b031 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:49.896293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ccd82c3-9ef2-4291-b05d-ed5a4762b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.908475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:49.908820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 940/1860] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:49.909501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.910285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd82c3-9ef2-4291-b05d-ed5a4762b031 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.917993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.918278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.921041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.933054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd82c3-9ef2-4291-b05d-ed5a4762b031 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:49.938525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:49.939212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:49.939491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:49.939758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] PATH_INFO: `/v3/projects/8ecef2fe027b4a95a25f3e5aa6360bc2/users/2a56e8def15742d8a8a8a210ca57d8fe/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:49.957623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:49.960309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ecef2fe027b4a95a25f3e5aa6360bc2, user_id=2a56e8def15742d8a8a8a210ca57d8fe, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:49.963811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:49.982916 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 941/1861] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:49 2022] PUT /identity/v3/projects/8ecef2fe027b4a95a25f3e5aa6360bc2/users/2a56e8def15742d8a8a8a210ca57d8fe/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:49.985076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:49.987501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:49.994149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:50.010033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.021013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.030672 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 921/1862] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:49 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:50.031806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:50.036138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:50.107528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a6fd3-6ff4-4af5-9af6-cf525ea5f78f service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:50.120346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652de474-a687-44b7-8034-f9e257f9c3f9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:50.126219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:50.127043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.127401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.127761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] PATH_INFO: `/v3/projects/e81a1b6d13954194a01cc43062b0038d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.129187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81a1b6d13954194a01cc43062b0038d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.131216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.175026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:50.177239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.177239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.177239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.187572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.189159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.198414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81a1b6d13954194a01cc43062b0038d', 'request_id': 'req-9d3b925c-d445-4c49-8529-be05925cea33'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.201788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 922/1863] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:50.202735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:50.210789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81a1b6d13954194a01cc43062b0038d', 'request_id': 'req-9d3b925c-d445-4c49-8529-be05925cea33'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.217058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ab8bf-f050-4e53-9c53-b246efb31b2b service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:50.225015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81a1b6d13954194a01cc43062b0038d', 'request_id': 'req-9d3b925c-d445-4c49-8529-be05925cea33'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.243798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] The token cache is being invalidate because project e81a1b6d13954194a01cc43062b0038d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:50.244645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.245689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.250224 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 942/1864] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:49 2022] DELETE /identity/v3/projects/e81a1b6d13954194a01cc43062b0038d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:50.252352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:50.255169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:50.303884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:50.305726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.306070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.307030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.307776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.313320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.332841 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 923/1865] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:50.333925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:50.337230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:50.364550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47a4fc-9472-4490-a1cf-80e9ea58d2f0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:50.374115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:50.374941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.375333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.375687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] PATH_INFO: `/v3/users/db4b607d5bc940d28d3c3b9edb6ef4a2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.377395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4b607d5bc940d28d3c3b9edb6ef4a2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.379418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.381912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b925c-d445-4c49-8529-be05925cea33 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:50.390795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:50.392131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.392579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.393230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.395785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:50.405222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:50.447007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:50.457108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4b607d5bc940d28d3c3b9edb6ef4a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.459237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4b607d5bc940d28d3c3b9edb6ef4a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.469864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4b607d5bc940d28d3c3b9edb6ef4a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.483729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4b607d5bc940d28d3c3b9edb6ef4a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.504034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4b607d5bc940d28d3c3b9edb6ef4a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.505527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.508327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.514137 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 924/1866] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:50 2022] DELETE /identity/v3/users/db4b607d5bc940d28d3c3b9edb6ef4a2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:50.515750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:50.522677 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 943/1867] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:50 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:50.530293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:50.627876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:50.630921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:50.657323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ecb66-a675-468d-9c6c-852a0689d7c6 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:50.707695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af53ef0-44fd-47cc-925f-ce625d7959fc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:50.720229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4c037-410f-40a7-a45f-cdefc3bf6064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:50.721108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c037-410f-40a7-a45f-cdefc3bf6064 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.721551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c037-410f-40a7-a45f-cdefc3bf6064 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.723770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c037-410f-40a7-a45f-cdefc3bf6064 admin admin] PATH_INFO: `/v3/projects/8ecef2fe027b4a95a25f3e5aa6360bc2/users/2a56e8def15742d8a8a8a210ca57d8fe/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.727710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4c037-410f-40a7-a45f-cdefc3bf6064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ecef2fe027b4a95a25f3e5aa6360bc2, user_id=2a56e8def15742d8a8a8a210ca57d8fe, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.732535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4c037-410f-40a7-a45f-cdefc3bf6064 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.755284 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 925/1868] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:50 2022] PUT /identity/v3/projects/8ecef2fe027b4a95a25f3e5aa6360bc2/users/2a56e8def15742d8a8a8a210ca57d8fe/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:50.769828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.770091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.770370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.805420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:50.806473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:50.806764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:50.807054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] PATH_INFO: `/v3/projects/3a28ae320ed0403c98477279023398bd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:50.808792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a28ae320ed0403c98477279023398bd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:50.811780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:50.852001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] MFA Rules not processed for user `2a56e8def15742d8a8a8a210ca57d8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:50.854197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:50.923225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a28ae320ed0403c98477279023398bd', 'request_id': 'req-4132f45b-0c82-4d3b-aff4-9af0042a1d83'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.934298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a28ae320ed0403c98477279023398bd', 'request_id': 'req-4132f45b-0c82-4d3b-aff4-9af0042a1d83'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.942445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a28ae320ed0403c98477279023398bd', 'request_id': 'req-4132f45b-0c82-4d3b-aff4-9af0042a1d83'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.985985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] The token cache is being invalidate because project 3a28ae320ed0403c98477279023398bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:50.986298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.987020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:50.988902 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 944/1869] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:50 2022] DELETE /identity/v3/projects/3a28ae320ed0403c98477279023398bd => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:56:50.993496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:50.994908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:51.033212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 926/1870] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:50 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:51.048793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:51.052393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:51.075115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:51.182210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:51.194869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132f45b-0c82-4d3b-aff4-9af0042a1d83 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:51.215910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:51.218720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:51.219298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:51.220293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] PATH_INFO: `/v3/users/83828eb232f74e0092746711eb78977b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:51.225431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83828eb232f74e0092746711eb78977b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:51.230744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:51.252769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8272b-9d0a-4670-8536-c752b991245a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:51.309838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83828eb232f74e0092746711eb78977b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.311502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83828eb232f74e0092746711eb78977b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.325028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83828eb232f74e0092746711eb78977b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.338794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83828eb232f74e0092746711eb78977b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.340214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83828eb232f74e0092746711eb78977b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.369716 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 945/1871] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:50 2022] DELETE /identity/v3/users/83828eb232f74e0092746711eb78977b => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:51.369716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:51.391398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:51.407454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:51.419345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:51.420806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:51.421379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:51.427690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:51.542611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f3e8-ad23-455f-b8fb-c25a268e3388 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:51.548112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:51.554915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:51.555042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:51.555171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:51.555171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] PATH_INFO: `/v3/projects/c6877dc065e44ab4a5ff07428f46867f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:51.556081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:51.556736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6877dc065e44ab4a5ff07428f46867f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:51.559608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:51.574287 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 927/1872] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:51 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:51.733474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6877dc065e44ab4a5ff07428f46867f', 'request_id': 'req-b279d6ea-a807-4987-beef-4e706e6e1bae'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.741806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:51.744058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:51.745952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6877dc065e44ab4a5ff07428f46867f', 'request_id': 'req-b279d6ea-a807-4987-beef-4e706e6e1bae'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.753841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6877dc065e44ab4a5ff07428f46867f', 'request_id': 'req-b279d6ea-a807-4987-beef-4e706e6e1bae'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.771477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] The token cache is being invalidate because project c6877dc065e44ab4a5ff07428f46867f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:51.771712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992708f4-6c08-4fc7-afd8-4c4ee0774523 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:51.771805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.772772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b279d6ea-a807-4987-beef-4e706e6e1bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:51.777116 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 946/1873] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:51 2022] DELETE /identity/v3/projects/c6877dc065e44ab4a5ff07428f46867f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:51.781094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67462d0c-24f8-4c5f-9f6a-d29d943081c1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:51.781417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67462d0c-24f8-4c5f-9f6a-d29d943081c1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:51.781712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67462d0c-24f8-4c5f-9f6a-d29d943081c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:51.796399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:51.838704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-67462d0c-24f8-4c5f-9f6a-d29d943081c1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:51.847634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-67462d0c-24f8-4c5f-9f6a-d29d943081c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:51.898985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:51.947938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:51.948622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:51.949202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:51.949534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:51.954840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:51.963876 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 947/1874] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:51.974459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd63069-c3b1-48e2-90af-9ddc05a4af10 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:51.974772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd63069-c3b1-48e2-90af-9ddc05a4af10 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:51.975015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd63069-c3b1-48e2-90af-9ddc05a4af10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.005931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-acd63069-c3b1-48e2-90af-9ddc05a4af10 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:52.007189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-acd63069-c3b1-48e2-90af-9ddc05a4af10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:52.024240 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 948/1875] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:52.029693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd63069-c3b1-48e2-90af-9ddc05a4af10 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.047796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd63069-c3b1-48e2-90af-9ddc05a4af10 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.062050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.062927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.062927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.063218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.064001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:52.064775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.065973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.080822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.085260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.100039 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 928/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:52.101321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 949/1877] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:56:52.105805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.125233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68e5b5-435a-4094-8498-655aca535b0a service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.140221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.142557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.142977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.143297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.144182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.146193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.198139 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 929/1878] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:52 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:52.204006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.223141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81297a-58f4-49e1-96e1-21880b12c549 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.277157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.279915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:52.374797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.375428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.375932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.376197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.377580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.381694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.424772 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 950/1879] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:52 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:52.429919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.443890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d128-adb8-42a7-acab-9e3bc4c46e9c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.458307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1ff19-7cfa-43da-bb25-9030bc7532d1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.458629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.459971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.460489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.461203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] PATH_INFO: `/v3/projects/db2605a6df2e44a5ab5c838febd8df19` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.462990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2605a6df2e44a5ab5c838febd8df19)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.470325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.470849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.471723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.472070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.472404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.473245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.475297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.496997 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 951/1880] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:52.504846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.516183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ede8b-7cca-4b93-8771-654195f1f48a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.525177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.525177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.525177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.525177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.525513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.527061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.543941 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 952/1881] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:52.557605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.558603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2605a6df2e44a5ab5c838febd8df19', 'request_id': 'req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.572692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffbf5b-720f-47ed-89b1-2989093563d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.573939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2605a6df2e44a5ab5c838febd8df19', 'request_id': 'req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.585879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2605a6df2e44a5ab5c838febd8df19', 'request_id': 'req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.587832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:52.642357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] The token cache is being invalidate because project db2605a6df2e44a5ab5c838febd8df19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:52.642520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.644034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.652379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 930/1882] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:52 2022] DELETE /identity/v3/projects/db2605a6df2e44a5ab5c838febd8df19 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:56:52.652379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:52.652379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:52.696499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:52.698424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d858d57-bbb6-4d67-beb1-52d45f5d0c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.699715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d858d57-bbb6-4d67-beb1-52d45f5d0c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.700028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d858d57-bbb6-4d67-beb1-52d45f5d0c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.700549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d858d57-bbb6-4d67-beb1-52d45f5d0c73 admin admin] PATH_INFO: `/v3/projects/b33d41b411144dbbbdb012e694857d27/users/d7d3aef1d4da47b2ab5454c3eb0346a6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.730885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d858d57-bbb6-4d67-beb1-52d45f5d0c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33d41b411144dbbbdb012e694857d27, user_id=d7d3aef1d4da47b2ab5454c3eb0346a6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.735854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d858d57-bbb6-4d67-beb1-52d45f5d0c73 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.755451 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 953/1883] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:52 2022] PUT /identity/v3/projects/b33d41b411144dbbbdb012e694857d27/users/d7d3aef1d4da47b2ab5454c3eb0346a6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:52.758785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6e080-8c25-4aa3-9321-e533ebbf1295 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.759162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6e080-8c25-4aa3-9321-e533ebbf1295 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.759512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6e080-8c25-4aa3-9321-e533ebbf1295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.763162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602a378-d05f-4d12-9faa-78e9cc5d9d3e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:52.784679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-63b6e080-8c25-4aa3-9321-e533ebbf1295 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:52.786127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63b6e080-8c25-4aa3-9321-e533ebbf1295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:52.877244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:52.878250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.878865 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 954/1884] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:52.880178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbc0a-3ab1-4630-bfbd-dc67cdeb5f38 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:52.880531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbc0a-3ab1-4630-bfbd-dc67cdeb5f38 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.881040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbc0a-3ab1-4630-bfbd-dc67cdeb5f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.887229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:52.888067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] PATH_INFO: `/v3/users/13c5c05dd6b244dd8ff15ca2c1ae298a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:52.889438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c5c05dd6b244dd8ff15ca2c1ae298a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:52.892044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:52.920571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-720cbc0a-3ab1-4630-bfbd-dc67cdeb5f38 None None] MFA Rules not processed for user `d7d3aef1d4da47b2ab5454c3eb0346a6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:52.922674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-720cbc0a-3ab1-4630-bfbd-dc67cdeb5f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:52.963328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c5c05dd6b244dd8ff15ca2c1ae298a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.964153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c5c05dd6b244dd8ff15ca2c1ae298a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.978628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c5c05dd6b244dd8ff15ca2c1ae298a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:52.990157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c5c05dd6b244dd8ff15ca2c1ae298a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:53.011399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15acc8df-0a0f-4619-8506-fbbd73f19991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c5c05dd6b244dd8ff15ca2c1ae298a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:53.022687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 931/1885] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:52 2022] DELETE /identity/v3/users/13c5c05dd6b244dd8ff15ca2c1ae298a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:53.024235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.024520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.024980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.054658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:53.062336 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 955/1886] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:53.064183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:53.064823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b3bda-3114-4ad6-8c8f-2f792f73735c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.065207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b3bda-3114-4ad6-8c8f-2f792f73735c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.065578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b3bda-3114-4ad6-8c8f-2f792f73735c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.086432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:53.113811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a66b3bda-3114-4ad6-8c8f-2f792f73735c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:53.115850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a66b3bda-3114-4ad6-8c8f-2f792f73735c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:53.179429 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 932/1887] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:53.180481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.182247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:53.202199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bdb95-815d-44c7-a752-8626ecfa5fe6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.211098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.212144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.212638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.212997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.214306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.216922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.245328 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 956/1888] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:53.254627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b3bda-3114-4ad6-8c8f-2f792f73735c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.281297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b3bda-3114-4ad6-8c8f-2f792f73735c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.290925 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 933/1889] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:53 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:53.291683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.294559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfc270-628d-4dd8-be0c-59ae05234be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.295112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:53.296211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfc270-628d-4dd8-be0c-59ae05234be1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.296608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfc270-628d-4dd8-be0c-59ae05234be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.296955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfc270-628d-4dd8-be0c-59ae05234be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.297641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfc270-628d-4dd8-be0c-59ae05234be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:53.298143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfc270-628d-4dd8-be0c-59ae05234be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.304834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfc270-628d-4dd8-be0c-59ae05234be1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.307422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:53.324510 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 957/1890] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:53.325849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08755a-c263-4095-b93d-99ada3ff194c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.326146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08755a-c263-4095-b93d-99ada3ff194c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.326423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08755a-c263-4095-b93d-99ada3ff194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.349348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ad08755a-c263-4095-b93d-99ada3ff194c None None] MFA Rules not processed for user `b4cebc85e62046d28fab159c6bb13e15`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:53.350642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ad08755a-c263-4095-b93d-99ada3ff194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:53.369582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40b42c-5247-4c85-ab9a-18223e08b838 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.373457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.374565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.374840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.375112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.413615 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 958/1891] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:53.414470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08755a-c263-4095-b93d-99ada3ff194c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.444982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.446512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.458522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 934/1892] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:53.459641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.470997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069a3b4-2d25-49e5-b232-e796dd9c8c22 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.485410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.486034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.486313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.486584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.487222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.489160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.504092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08755a-c263-4095-b93d-99ada3ff194c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.511897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.512588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.516505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.516834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.517899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.520521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.529177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 935/1893] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:53 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:53.532351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.550918 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 959/1894] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:53 2022] POST /identity/v3/users => generated 302 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:53.557258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.569671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c79e46-996e-45a4-a514-913846f94db9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.574732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:53.640432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53c3e1-faf8-4536-8d3c-6c0d722ebae8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.649622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.650919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.651213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.651492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.652164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.655274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.672261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.674240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.674599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.674863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.675532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.677618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.691205 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 936/1895] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:53 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:53.692458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.698425 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 960/1896] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:53.702663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.716339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574db42-6c8d-4598-a0f4-6a488d41e28c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.723259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.723902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.724183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.724454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.725315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.727306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.728508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e98d1-f155-400f-b8fe-9b38a868a4ca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.734377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.735054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.735328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.735597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] PATH_INFO: `/v3/projects/06e7347f7bbc45ce8de8e29de5255fa9/users/f1c3ed62097a4e89bf360a966c764762/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.756735 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 937/1897] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:53.758618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.759676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e7347f7bbc45ce8de8e29de5255fa9, user_id=f1c3ed62097a4e89bf360a966c764762, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.762883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.778316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301f300-268d-4244-8084-3b60e65b734b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.779042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:53.788908 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 961/1898] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:53 2022] PUT /identity/v3/projects/06e7347f7bbc45ce8de8e29de5255fa9/users/f1c3ed62097a4e89bf360a966c764762/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:56:53.802650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.893618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.894825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.895346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.895723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.896764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.900414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.914549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830a11e-7fdd-4a1f-bb57-794a086e548b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.915204 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 938/1899] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:53.919679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.925126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348116a6-5cee-4cdb-9943-f4a2847b5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.926072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348116a6-5cee-4cdb-9943-f4a2847b5df9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.926498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348116a6-5cee-4cdb-9943-f4a2847b5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.927401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348116a6-5cee-4cdb-9943-f4a2847b5df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.928352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348116a6-5cee-4cdb-9943-f4a2847b5df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.930481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348116a6-5cee-4cdb-9943-f4a2847b5df9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.934428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cfb0e-4ef8-462c-a0a8-eba648a9fa8d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.941519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8996db-d259-40cd-ba03-5c153250af0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.942537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8996db-d259-40cd-ba03-5c153250af0e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.942903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8996db-d259-40cd-ba03-5c153250af0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.943389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8996db-d259-40cd-ba03-5c153250af0e admin admin] PATH_INFO: `/v3/projects/ec89ada1460d4275812339f2d6a93042/users/b3dd80698e4c471abe7cd81809fa206c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.948992 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 962/1900] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:53.950695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c5be-ae88-4842-bad1-ed9d49bacf8f None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.950978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c5be-ae88-4842-bad1-ed9d49bacf8f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.951249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c5be-ae88-4842-bad1-ed9d49bacf8f None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.952760 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 963/1901] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:53.958136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7c5be-ae88-4842-bad1-ed9d49bacf8f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:53.974477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7c5be-ae88-4842-bad1-ed9d49bacf8f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:53.977028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8996db-d259-40cd-ba03-5c153250af0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec89ada1460d4275812339f2d6a93042, user_id=b3dd80698e4c471abe7cd81809fa206c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.979462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:53.980142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:53.980450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:53.981074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8996db-d259-40cd-ba03-5c153250af0e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:53.984824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] PATH_INFO: `/v3/projects/06e7347f7bbc45ce8de8e29de5255fa9/users/f1c3ed62097a4e89bf360a966c764762/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:53.988403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e7347f7bbc45ce8de8e29de5255fa9, user_id=f1c3ed62097a4e89bf360a966c764762, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:53.991875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.005624 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 939/1902] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:53 2022] PUT /identity/v3/projects/ec89ada1460d4275812339f2d6a93042/users/b3dd80698e4c471abe7cd81809fa206c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:54.006931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339d73-8fb1-4f55-8e40-074b105b3715 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.007225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339d73-8fb1-4f55-8e40-074b105b3715 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.007492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339d73-8fb1-4f55-8e40-074b105b3715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.013788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 964/1903] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:53 2022] PUT /identity/v3/projects/06e7347f7bbc45ce8de8e29de5255fa9/users/f1c3ed62097a4e89bf360a966c764762/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:54.014981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.016704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.035883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cc339d73-8fb1-4f55-8e40-074b105b3715 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:54.037200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cc339d73-8fb1-4f55-8e40-074b105b3715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.088058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d5af-5a55-4dfe-b17f-1835cf7d46fd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.092633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.093336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.093611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.093851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] PATH_INFO: `/v3/projects/56f793358f224ff98e1f6bc0e95309c5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.095184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f793358f224ff98e1f6bc0e95309c5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.096921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.103444 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 940/1904] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:56:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:54.104929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.105230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.105639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.142804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] MFA Rules not processed for user `b3dd80698e4c471abe7cd81809fa206c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:54.144030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.155011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:54.161561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f793358f224ff98e1f6bc0e95309c5', 'request_id': 'req-1ebf7e48-6d0f-425b-908c-32d652c86373'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.173057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f793358f224ff98e1f6bc0e95309c5', 'request_id': 'req-1ebf7e48-6d0f-425b-908c-32d652c86373'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.181413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f793358f224ff98e1f6bc0e95309c5', 'request_id': 'req-1ebf7e48-6d0f-425b-908c-32d652c86373'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.193211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] The token cache is being invalidate because project 56f793358f224ff98e1f6bc0e95309c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:54.193519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.195246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ebf7e48-6d0f-425b-908c-32d652c86373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.197038 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 965/1905] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:54 2022] DELETE /identity/v3/projects/56f793358f224ff98e1f6bc0e95309c5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:54.198207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.198489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.198757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.231498 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 941/1906] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:54.232238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.235028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:54.236229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.238563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] MFA Rules not processed for user `f1c3ed62097a4e89bf360a966c764762`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:54.240386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.313079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c8d9c-baf8-4d8f-afd0-a829ce8c2b12 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.316829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.317917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.317917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.318149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.320780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.323428 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 966/1907] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:54.324136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.325852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.336848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.338351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.347397 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 942/1908] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:54.348112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.349693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.414520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6d686-2b62-4d51-a667-b3c470d39f64 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.424039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.424720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.424995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.425664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] PATH_INFO: `/v3/users/2a56e8def15742d8a8a8a210ca57d8fe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.427171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a56e8def15742d8a8a8a210ca57d8fe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.429301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.432690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d998b8-6b09-4b68-9f40-bb798ff0fb12 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.440061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7b429-c46d-4448-b7a2-404d1fe274fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.441593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7b429-c46d-4448-b7a2-404d1fe274fe admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.441872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7b429-c46d-4448-b7a2-404d1fe274fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.442343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7b429-c46d-4448-b7a2-404d1fe274fe admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.442963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7b429-c46d-4448-b7a2-404d1fe274fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.445311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7b429-c46d-4448-b7a2-404d1fe274fe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.477158 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 943/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:54 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:54.478431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.478747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.478995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.489089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a56e8def15742d8a8a8a210ca57d8fe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.489735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a56e8def15742d8a8a8a210ca57d8fe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.500199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] MFA Rules not processed for user `d7d3aef1d4da47b2ab5454c3eb0346a6`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:54.501268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a56e8def15742d8a8a8a210ca57d8fe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.501699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.512650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a56e8def15742d8a8a8a210ca57d8fe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.513810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a56e8def15742d8a8a8a210ca57d8fe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:54.536057 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 967/1910] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:54 2022] DELETE /identity/v3/users/2a56e8def15742d8a8a8a210ca57d8fe => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:54.536993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.538345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.547567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:54.561352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:54.587393 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 944/1911] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:54.601102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.602665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.647790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd2565-8f54-44a7-af92-8e94ec327b1d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.676061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.677361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.677361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.677593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.678234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.680255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.707175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc8b84-0f1a-47ef-b147-e0eee091e290 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.721795 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 968/1912] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:54 2022] POST /identity/v3/users => generated 303 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:54.722752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.734427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4939d-1e14-4085-b62c-1deec99629a5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.743037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.743670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.743987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.744294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.750194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.767873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.803725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 969/1913] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:56:54.805023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.818946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445520-1011-47b0-84e6-bed47ad5a66b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:54.838000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.839152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.839797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:54.841378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.842141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.842988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:54.844551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:54.846159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] PATH_INFO: `/v3/projects/02e1a07303744cf099095529bf775200/users/4feaaf44afea40d392a5eb503c315413/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:54.847317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:54.889625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e1a07303744cf099095529bf775200, user_id=4feaaf44afea40d392a5eb503c315413, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.892382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.928041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:54.929910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:54.938585 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 970/1914] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:54 2022] PUT /identity/v3/projects/02e1a07303744cf099095529bf775200/users/4feaaf44afea40d392a5eb503c315413/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:54.940045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:54.956843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 945/1915] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:56:55.081050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8604c-b6c8-4f43-8edc-c267552a4836 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.086389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:55.088503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:55.088808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.089454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.089732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.090147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.090814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.092626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:55.108594 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 971/1916] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:55.110274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:55.120711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3b9b4-f048-40fc-ac6d-000326747ffe service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.125776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eceeea-e93e-48f7-8aea-e3758ec20946 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.129210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.130366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.130821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.131239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] PATH_INFO: `/v3/projects/8ecef2fe027b4a95a25f3e5aa6360bc2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.133151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ecef2fe027b4a95a25f3e5aa6360bc2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.133319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02453130-ac58-4645-b8be-4e689855cbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.134076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02453130-ac58-4645-b8be-4e689855cbf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.134459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02453130-ac58-4645-b8be-4e689855cbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.135439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02453130-ac58-4645-b8be-4e689855cbf8 admin admin] PATH_INFO: `/v3/projects/02e1a07303744cf099095529bf775200/users/4feaaf44afea40d392a5eb503c315413/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.138247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02453130-ac58-4645-b8be-4e689855cbf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e1a07303744cf099095529bf775200, user_id=4feaaf44afea40d392a5eb503c315413, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.139159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:55.141541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02453130-ac58-4645-b8be-4e689855cbf8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:55.160766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 972/1917] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:55 2022] PUT /identity/v3/projects/02e1a07303744cf099095529bf775200/users/4feaaf44afea40d392a5eb503c315413/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:55.172586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99db0-6691-4dcc-a412-bd15798d2f3d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.173423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99db0-6691-4dcc-a412-bd15798d2f3d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.173939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99db0-6691-4dcc-a412-bd15798d2f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.208068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:55.225929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-78d99db0-6691-4dcc-a412-bd15798d2f3d None None] MFA Rules not processed for user `4feaaf44afea40d392a5eb503c315413`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:55.227201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78d99db0-6691-4dcc-a412-bd15798d2f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:55.261629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ecef2fe027b4a95a25f3e5aa6360bc2', 'request_id': 'req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:55.272421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ecef2fe027b4a95a25f3e5aa6360bc2', 'request_id': 'req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:55.287461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:55.289519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ecef2fe027b4a95a25f3e5aa6360bc2', 'request_id': 'req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:55.315653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] The token cache is being invalidate because project 8ecef2fe027b4a95a25f3e5aa6360bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:55.315941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:55.318404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bbfc81ac-f954-4d4f-a4db-e8dda48add9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:55.322041 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 946/1918] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:55 2022] DELETE /identity/v3/projects/8ecef2fe027b4a95a25f3e5aa6360bc2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:55.327586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf924-2f0d-489c-b651-4755324a2bcb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.327882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf924-2f0d-489c-b651-4755324a2bcb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.328529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf924-2f0d-489c-b651-4755324a2bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.338295 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 973/1919] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:55.349466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.351578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.351578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.387469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] MFA Rules not processed for user `b3dd80698e4c471abe7cd81809fa206c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:55.388825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:55.391085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d2acf924-2f0d-489c-b651-4755324a2bcb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:55.395245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d2acf924-2f0d-489c-b651-4755324a2bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:55.481711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 974/1920] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:56:55.492852 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 947/1921] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:55.493038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:55.493038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:55.497079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683e0bd-e78f-44d3-b2ed-10d438f78273 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.497250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683e0bd-e78f-44d3-b2ed-10d438f78273 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.497543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683e0bd-e78f-44d3-b2ed-10d438f78273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.544509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e683e0bd-e78f-44d3-b2ed-10d438f78273 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:55.547179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e683e0bd-e78f-44d3-b2ed-10d438f78273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:55.581963 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 948/1922] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:55.587125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683e0bd-e78f-44d3-b2ed-10d438f78273 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:55.621164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683e0bd-e78f-44d3-b2ed-10d438f78273 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.635519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.636444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.636783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.637170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.638430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:55.638856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.641558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:55.658809 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 949/1923] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:55.667141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:55.676417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a33955-fadc-4325-9b3a-f15d87e80f2c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.689453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.690073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.690352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.690623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.692869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae59fc-416d-4858-b548-f8b27de0159a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.693547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:55.704482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.706025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.707974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.708369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.708804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.709626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.710378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:55.715865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:55.736332 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 975/1924] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:55.770884 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 950/1925] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:55 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:55.775934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:55.793856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ebfae-fa69-4efe-8aee-bfc6d5021d0a service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.898429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.899063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.899478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.900163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.901100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.903281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:55.931290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:55.938082 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 976/1926] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:55 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:55.943336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:55.962308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afc932-3f73-4ec4-8e5f-ce309022fffa admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:55.969945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:55.970704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:55.971194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:55.971845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:55.972626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:55.975297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.016190 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 951/1927] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:56.016638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:56.039665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798dded-65a9-4a9a-b3c9-1c58da4f3059 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:56.045532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b2f42-9548-4ee5-8548-21072a47d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:56.046162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b2f42-9548-4ee5-8548-21072a47d3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.046590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b2f42-9548-4ee5-8548-21072a47d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.046884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b2f42-9548-4ee5-8548-21072a47d3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.047719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b2f42-9548-4ee5-8548-21072a47d3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:56.049536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b2f42-9548-4ee5-8548-21072a47d3a8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.067435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 977/1928] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:56:56.073321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:56.094889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ba1e8-7312-4159-8933-6b63ea8675cc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:56.110871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:56.111768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.112190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.112784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] PATH_INFO: `/v3/projects/412dfc5cc92a477292bd34bfd94509d4/users/f27cbc96131349e88289ebaf70702633/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.156025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412dfc5cc92a477292bd34bfd94509d4, user_id=f27cbc96131349e88289ebaf70702633, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:56.160980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.197725 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 952/1929] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:56 2022] PUT /identity/v3/projects/412dfc5cc92a477292bd34bfd94509d4/users/f27cbc96131349e88289ebaf70702633/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:56.202141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f1dc-cf3a-4e0a-86c7-a871d718910d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.202557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f1dc-cf3a-4e0a-86c7-a871d718910d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.202841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f1dc-cf3a-4e0a-86c7-a871d718910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.239649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ee63f1dc-cf3a-4e0a-86c7-a871d718910d None None] MFA Rules not processed for user `f27cbc96131349e88289ebaf70702633`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:56.241493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee63f1dc-cf3a-4e0a-86c7-a871d718910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:56.358606 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 978/1930] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:56.363128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:56.460964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:56.496855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3642ce1-9834-4f11-bbe6-10bdfa0090de None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.497410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3642ce1-9834-4f11-bbe6-10bdfa0090de None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.498020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3642ce1-9834-4f11-bbe6-10bdfa0090de None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.501210 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 979/1931] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:56:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:56:56.524603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.524927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.525200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.529327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723f0f9-f93c-467d-950b-4fe01a3b6377 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:56.536628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:56.537552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.539814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.540099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.540903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:56.542797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.586396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:56.587640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:56.591919 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 953/1932] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:56 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:56.596309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:56.711454 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 980/1933] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:56:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:56.713047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5881a9-7256-4338-86be-b2b48e13fcdb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:56.718902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:56.719576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.719884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.720156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.720927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:56.726270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.728361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:56.729948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:56.746483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3470b-0552-47aa-be39-e9e9bff0617e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:56.766804 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 954/1934] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:56 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:56.771269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:56.786076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935887a-c981-4896-ba8c-16165f8c86af admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:56.799400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:56.800110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.800386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.800672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.801356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:56.803340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.820058 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 955/1935] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:56.833321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:56.849923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d12b-1af9-4a69-bef4-44fb7d3c9481 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:56.871794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:56.872684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.873048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.873445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] PATH_INFO: `/v3/projects/c6fb631723a844c9a7d39809e4dffc41/users/90aea0aa5a2c4cf2ad27dd7ee9db4824/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.898470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fb631723a844c9a7d39809e4dffc41, user_id=90aea0aa5a2c4cf2ad27dd7ee9db4824, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:56.901793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:56.903935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.904773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:56.905124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:56.905457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:56.913991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:56.919251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:56.921044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:56.932295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 956/1936] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:56 2022] PUT /identity/v3/projects/c6fb631723a844c9a7d39809e4dffc41/users/90aea0aa5a2c4cf2ad27dd7ee9db4824/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:56.934558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 981/1937] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:56.948747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:57.050696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff7c50-2e85-4b1f-ab28-61e8a8337528 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:57.057912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b09ec-1d9d-4a5c-817f-e249e6934abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:57.058833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b09ec-1d9d-4a5c-817f-e249e6934abb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:57.059251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b09ec-1d9d-4a5c-817f-e249e6934abb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:57.059663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b09ec-1d9d-4a5c-817f-e249e6934abb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:57.060489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b09ec-1d9d-4a5c-817f-e249e6934abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:57.062741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b09ec-1d9d-4a5c-817f-e249e6934abb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:57.076020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 957/1938] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:57.080560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:57.094817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab1da9-3719-4aac-ab23-d208ee1e07b1 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:57.100870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:57.101865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:57.101959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:57.102177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] PATH_INFO: `/v3/projects/c6fb631723a844c9a7d39809e4dffc41/users/90aea0aa5a2c4cf2ad27dd7ee9db4824/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:57.104732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fb631723a844c9a7d39809e4dffc41, user_id=90aea0aa5a2c4cf2ad27dd7ee9db4824, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:57.107447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:57.148965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6be57-3138-4b34-92e4-efbad9cb1e31 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:57.149490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6be57-3138-4b34-92e4-efbad9cb1e31 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:57.149944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6be57-3138-4b34-92e4-efbad9cb1e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:57.153649 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 982/1939] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:57 2022] PUT /identity/v3/projects/c6fb631723a844c9a7d39809e4dffc41/users/90aea0aa5a2c4cf2ad27dd7ee9db4824/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:57.163834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:57.215934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-91a6be57-3138-4b34-92e4-efbad9cb1e31 None None] MFA Rules not processed for user `90aea0aa5a2c4cf2ad27dd7ee9db4824`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:57.218088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-91a6be57-3138-4b34-92e4-efbad9cb1e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:57.318125 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 958/1940] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:57.330911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:57.431734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ad8b7-0e5c-437d-b2ce-d5e7402baf09 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:57.447191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e629f4a-ff24-405c-a39f-2a857f8f2093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:57.447834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629f4a-ff24-405c-a39f-2a857f8f2093 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:57.448569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629f4a-ff24-405c-a39f-2a857f8f2093 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:57.448818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629f4a-ff24-405c-a39f-2a857f8f2093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:57.460021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e629f4a-ff24-405c-a39f-2a857f8f2093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:57.462318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e629f4a-ff24-405c-a39f-2a857f8f2093 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:57.474380 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 983/1941] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:57.570313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6be57-3138-4b34-92e4-efbad9cb1e31 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:57.639372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6be57-3138-4b34-92e4-efbad9cb1e31 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:57.644440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:57.645071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:57.645374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:57.645628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] PATH_INFO: `/v3/users/f27cbc96131349e88289ebaf70702633` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:57.646758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f27cbc96131349e88289ebaf70702633)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:57.648753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:57.708144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f27cbc96131349e88289ebaf70702633'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:57.708937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f27cbc96131349e88289ebaf70702633'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:57.721318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f27cbc96131349e88289ebaf70702633'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:57.732652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f27cbc96131349e88289ebaf70702633'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:57.756185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f27cbc96131349e88289ebaf70702633'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:57.768020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 959/1942] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:57 2022] DELETE /identity/v3/users/f27cbc96131349e88289ebaf70702633 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:57.779615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:57.780272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:57.780272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:57.806216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:57.807748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:57.892688 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 984/1943] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:57.903837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:57.905735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:57.924803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:57.990683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9036a-3eb3-4b52-a5d0-0ff4760813a9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:57.994897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:57.995517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:57.995792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:57.996059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:58.008526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:58.015515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:58.022769 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 960/1944] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:58.365339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:58.368831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:58.392133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc9a0f-fead-44c8-a029-44499b7a2991 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:58.397167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:58.397898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:58.398324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:58.398600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] PATH_INFO: `/v3/projects/412dfc5cc92a477292bd34bfd94509d4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:58.403441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] RBAC: Authorizing `identity:delete_project(project_id=412dfc5cc92a477292bd34bfd94509d4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:58.408692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:58.466473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412dfc5cc92a477292bd34bfd94509d4', 'request_id': 'req-4c75fae1-0c8f-4945-8095-4cd6a16f699a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.476026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412dfc5cc92a477292bd34bfd94509d4', 'request_id': 'req-4c75fae1-0c8f-4945-8095-4cd6a16f699a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.482159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:58.483435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412dfc5cc92a477292bd34bfd94509d4', 'request_id': 'req-4c75fae1-0c8f-4945-8095-4cd6a16f699a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.498059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] The token cache is being invalidate because project 412dfc5cc92a477292bd34bfd94509d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:58.498399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.499366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.501469 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 985/1945] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:58 2022] DELETE /identity/v3/projects/412dfc5cc92a477292bd34bfd94509d4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:56:58.507535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:58.520557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086b427-8df4-4ece-9441-08ff45fbd4ef service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:58.591876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:58.592525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:58.592843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:58.593112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] PATH_INFO: `/v3/users/90aea0aa5a2c4cf2ad27dd7ee9db4824` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:58.594328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90aea0aa5a2c4cf2ad27dd7ee9db4824)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:58.596284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:58.655120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90aea0aa5a2c4cf2ad27dd7ee9db4824'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.655831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90aea0aa5a2c4cf2ad27dd7ee9db4824'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.665117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90aea0aa5a2c4cf2ad27dd7ee9db4824'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.674936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90aea0aa5a2c4cf2ad27dd7ee9db4824'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.689974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f8e8375-cbc3-4653-bc59-b963d9624af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90aea0aa5a2c4cf2ad27dd7ee9db4824'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:58.696610 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 961/1946] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:56:58 2022] DELETE /identity/v3/users/90aea0aa5a2c4cf2ad27dd7ee9db4824 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:59.005859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.007064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:59.029887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75fae1-0c8f-4945-8095-4cd6a16f699a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.097139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.097941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.098322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.098690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] PATH_INFO: `/v3/projects/c6fb631723a844c9a7d39809e4dffc41` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.099913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6fb631723a844c9a7d39809e4dffc41)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.102239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.172979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6fb631723a844c9a7d39809e4dffc41', 'request_id': 'req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:59.185185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6fb631723a844c9a7d39809e4dffc41', 'request_id': 'req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:59.191054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6fb631723a844c9a7d39809e4dffc41', 'request_id': 'req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:59.205701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] The token cache is being invalidate because project c6fb631723a844c9a7d39809e4dffc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:56:59.206080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:59.206845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbcf517-3c66-418a-b9ec-5c05dd1ecf7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:56:59.208960 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 986/1947] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:56:59 2022] DELETE /identity/v3/projects/c6fb631723a844c9a7d39809e4dffc41 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:59.217728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b1d6b-5116-4d06-84b2-52b77efbf6dd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.218131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b1d6b-5116-4d06-84b2-52b77efbf6dd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.218286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b1d6b-5116-4d06-84b2-52b77efbf6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.239962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d59b1d6b-5116-4d06-84b2-52b77efbf6dd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:59.241358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d59b1d6b-5116-4d06-84b2-52b77efbf6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:59.269614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:59.316406 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 962/1948] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:59.324983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49f12c-4d73-4493-a0f8-06242f1c9bf8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.325405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49f12c-4d73-4493-a0f8-06242f1c9bf8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.325605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49f12c-4d73-4493-a0f8-06242f1c9bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.348406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5e49f12c-4d73-4493-a0f8-06242f1c9bf8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:59.350002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5e49f12c-4d73-4493-a0f8-06242f1c9bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:59.359541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 987/1949] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:59.364783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b1d6b-5116-4d06-84b2-52b77efbf6dd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.377508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b1d6b-5116-4d06-84b2-52b77efbf6dd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.387634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.388374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.388750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.389033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.389748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:56:59.390050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.391699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.406474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 963/1950] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:56:59.411172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49f12c-4d73-4493-a0f8-06242f1c9bf8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.423067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49f12c-4d73-4493-a0f8-06242f1c9bf8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.428754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.429527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.429775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.430217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.430775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.432720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.476392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 988/1951] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:59 2022] POST /identity/v3/projects => generated 407 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:59.478328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.492985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53afc9-a51f-4c9c-a347-2a65c23c5145 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.570186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.571305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.571654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.571963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.572619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.574625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.608377 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 964/1952] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:59 2022] POST /identity/v3/users => generated 304 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:59.612389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.623340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357a054-35f3-4f67-b25c-b7a777d187d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.628025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.628670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.628963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.629233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.629909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.631513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.642619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 989/1953] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:59.647042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.658235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e5962-795a-4bd0-9b05-e2e63d924826 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.663289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da84261-5d0c-4464-b8c3-e42b18a3e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.663925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da84261-5d0c-4464-b8c3-e42b18a3e198 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.664203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da84261-5d0c-4464-b8c3-e42b18a3e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.664521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da84261-5d0c-4464-b8c3-e42b18a3e198 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.665251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da84261-5d0c-4464-b8c3-e42b18a3e198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.666873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da84261-5d0c-4464-b8c3-e42b18a3e198 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.679658 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 965/1954] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:56:59.684033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.694864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf3e84-4a89-4e5b-8e9a-c22ad0f72f07 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.702328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.702474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.702673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.702949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] PATH_INFO: `/v3/projects/e226365285a64117b2264a08b1426d60/users/0e489c45671e4250acdeeb15d296e702/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.721385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e226365285a64117b2264a08b1426d60, user_id=0e489c45671e4250acdeeb15d296e702, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.724410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.740773 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 990/1955] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:56:59 2022] PUT /identity/v3/projects/e226365285a64117b2264a08b1426d60/users/0e489c45671e4250acdeeb15d296e702/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:56:59.752617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525efc8-4485-4d2f-a39c-dc3b21e74bec None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.754004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525efc8-4485-4d2f-a39c-dc3b21e74bec None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.754127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525efc8-4485-4d2f-a39c-dc3b21e74bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.783809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7525efc8-4485-4d2f-a39c-dc3b21e74bec None None] MFA Rules not processed for user `0e489c45671e4250acdeeb15d296e702`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:56:59.785039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7525efc8-4485-4d2f-a39c-dc3b21e74bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:56:59.864281 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 966/1956] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:56:59 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:56:59.869421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.891412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:59.938283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853be588-fda0-49bc-bcb1-eebd2de6e02e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:56:59.944254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:56:59.944948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:56:59.945221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:56:59.945485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:56:59.946330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:56:59.948233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:56:59.978397 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 991/1957] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:56:59 2022] POST /identity/v3/projects => generated 407 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:56:59.983339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525efc8-4485-4d2f-a39c-dc3b21e74bec None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:56:59.995034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525efc8-4485-4d2f-a39c-dc3b21e74bec None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:00.026549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:00.071772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:00.072394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.072681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.072972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.073607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:00.075629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:00.101138 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 967/1958] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:56:59 2022] POST /identity/v3/users => generated 310 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:00.105674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:00.117166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbced5-0a8d-40ad-8eff-1e438d9fa995 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:00.123045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:00.123835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.124138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.124837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.125596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:00.127278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:00.140225 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 992/1959] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:00.144837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:00.156869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458d92-55bf-4649-b0f4-b4503d9e10ec admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:00.162687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:00.163629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.163629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.163911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] PATH_INFO: `/v3/projects/ee9efcd705da4706a27a72d888694dc5/users/c712aca72281469ea8acc5354b3604c9/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.184171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9efcd705da4706a27a72d888694dc5, user_id=c712aca72281469ea8acc5354b3604c9, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:00.186908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:00.208935 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 968/1960] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:00 2022] PUT /identity/v3/projects/ee9efcd705da4706a27a72d888694dc5/users/c712aca72281469ea8acc5354b3604c9/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:00.210250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:00.225440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb5c17-e935-44fd-91f6-99a084b68ed6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:00.305299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046ca61-d6e8-4f68-b44a-a7f292711992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:00.306030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046ca61-d6e8-4f68-b44a-a7f292711992 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.306309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046ca61-d6e8-4f68-b44a-a7f292711992 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.306584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046ca61-d6e8-4f68-b44a-a7f292711992 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.307225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046ca61-d6e8-4f68-b44a-a7f292711992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:00.309098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046ca61-d6e8-4f68-b44a-a7f292711992 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:00.321906 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 993/1961] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:00.326273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:00.337773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c2069-8f8c-466b-96de-4de3cf9e8279 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:00.342565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:00.343884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.343884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.343884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] PATH_INFO: `/v3/projects/ee9efcd705da4706a27a72d888694dc5/users/c712aca72281469ea8acc5354b3604c9/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.345955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9efcd705da4706a27a72d888694dc5, user_id=c712aca72281469ea8acc5354b3604c9, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:00.348578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:00.362982 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 969/1962] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:00 2022] PUT /identity/v3/projects/ee9efcd705da4706a27a72d888694dc5/users/c712aca72281469ea8acc5354b3604c9/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:00.373917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c62ad8-612b-410d-91a0-f638230436ce None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.374102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c62ad8-612b-410d-91a0-f638230436ce None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.374370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c62ad8-612b-410d-91a0-f638230436ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.404677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-15c62ad8-612b-410d-91a0-f638230436ce None None] MFA Rules not processed for user `c712aca72281469ea8acc5354b3604c9`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:00.405884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15c62ad8-612b-410d-91a0-f638230436ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:00.492217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 994/1963] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:00.501032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:00.585509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9763ce-9913-43ed-9ad7-cc4c30ce4e3d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:00.591573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:00.592237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.592718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.593006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] PATH_INFO: `/v3/users/0e489c45671e4250acdeeb15d296e702` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.594232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e489c45671e4250acdeeb15d296e702)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:00.596225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:00.666722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e489c45671e4250acdeeb15d296e702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:00.667511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e489c45671e4250acdeeb15d296e702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:00.678701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e489c45671e4250acdeeb15d296e702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:00.697060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e489c45671e4250acdeeb15d296e702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:00.715026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:00.716801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e489c45671e4250acdeeb15d296e702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:00.726088 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 970/1964] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:00 2022] DELETE /identity/v3/users/0e489c45671e4250acdeeb15d296e702 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:00.731254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.731650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.731862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.762381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:00.763602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:00.779067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:00.840126 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 995/1965] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:00.853437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:00.854870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:00.936709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55d36-8c48-465d-af27-079e71dd296e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:00.941172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:00.943116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:00.943116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:00.943421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:00.954349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:00.955869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:00.967681 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 971/1966] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:01.525723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:01.527428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:01.540486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa7f1-45d8-4dc7-b060-0b54e6cb046c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:01.546513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:01.547178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:01.547464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:01.547736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] PATH_INFO: `/v3/projects/e226365285a64117b2264a08b1426d60` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:01.549105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e226365285a64117b2264a08b1426d60)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:01.551407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:01.609898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e226365285a64117b2264a08b1426d60', 'request_id': 'req-12c6dc43-f7b4-454d-91de-ec9a06798a96'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.621525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e226365285a64117b2264a08b1426d60', 'request_id': 'req-12c6dc43-f7b4-454d-91de-ec9a06798a96'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.628362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e226365285a64117b2264a08b1426d60', 'request_id': 'req-12c6dc43-f7b4-454d-91de-ec9a06798a96'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.640802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] The token cache is being invalidate because project e226365285a64117b2264a08b1426d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:01.641101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.641787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.643337 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 996/1967] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:01 2022] DELETE /identity/v3/projects/e226365285a64117b2264a08b1426d60 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:01.647570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:01.658115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9560f-6122-4852-ba97-db50fb40f3bf service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:01.731977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:01.733181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:01.733464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:01.733737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] PATH_INFO: `/v3/users/c712aca72281469ea8acc5354b3604c9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:01.735340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c712aca72281469ea8acc5354b3604c9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:01.737548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:01.802945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c712aca72281469ea8acc5354b3604c9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.804586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c712aca72281469ea8acc5354b3604c9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.812444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c712aca72281469ea8acc5354b3604c9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.828771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c712aca72281469ea8acc5354b3604c9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.843593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac5afdf1-7bd8-41bd-9d50-22bf55ed41ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c712aca72281469ea8acc5354b3604c9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:01.850931 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 972/1968] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:01 2022] DELETE /identity/v3/users/c712aca72281469ea8acc5354b3604c9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:02.175789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:02.178565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:02.193563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6dc43-f7b4-454d-91de-ec9a06798a96 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:02.256384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:02.257052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.257325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.257593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] PATH_INFO: `/v3/projects/ee9efcd705da4706a27a72d888694dc5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.258779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9efcd705da4706a27a72d888694dc5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:02.260708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:02.306313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:02.315368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9efcd705da4706a27a72d888694dc5', 'request_id': 'req-62ef997c-44d3-4f89-afb6-6d959dde8344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:02.325666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9efcd705da4706a27a72d888694dc5', 'request_id': 'req-62ef997c-44d3-4f89-afb6-6d959dde8344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:02.331673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9efcd705da4706a27a72d888694dc5', 'request_id': 'req-62ef997c-44d3-4f89-afb6-6d959dde8344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:02.343879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] The token cache is being invalidate because project ee9efcd705da4706a27a72d888694dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:02.344172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:02.344939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-62ef997c-44d3-4f89-afb6-6d959dde8344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:02.346625 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 997/1969] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:02 2022] DELETE /identity/v3/projects/ee9efcd705da4706a27a72d888694dc5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:02.353322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e323a4-601f-48b8-b8f3-7eba3b21151c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.353603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e323a4-601f-48b8-b8f3-7eba3b21151c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.353904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e323a4-601f-48b8-b8f3-7eba3b21151c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.375364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-17e323a4-601f-48b8-b8f3-7eba3b21151c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:02.376586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-17e323a4-601f-48b8-b8f3-7eba3b21151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:02.390872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:02.448557 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 973/1970] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:02.458309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b805-48fa-4a7e-a2be-328196736fb3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.458639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b805-48fa-4a7e-a2be-328196736fb3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.458904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b805-48fa-4a7e-a2be-328196736fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.482683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c3d5b805-48fa-4a7e-a2be-328196736fb3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:02.483907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3d5b805-48fa-4a7e-a2be-328196736fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:02.493724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 998/1971] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:02.498083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e323a4-601f-48b8-b8f3-7eba3b21151c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:02.508913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e323a4-601f-48b8-b8f3-7eba3b21151c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:02.515715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:02.516364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.516643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.516950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.517591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:02.517902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:02.519531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:02.533375 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 974/1972] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:02.537786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b805-48fa-4a7e-a2be-328196736fb3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:02.560682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b805-48fa-4a7e-a2be-328196736fb3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:02.568361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:02.569021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.569298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.569568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.570377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:02.573169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:02.620738 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 999/1973] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:02 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:02.622420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:02.635205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe9b6d-dde3-4f6d-90b9-d096320b7f9f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:02.704076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:02.704814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.705157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.705497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.706201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:02.708190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:02.733267 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 975/1974] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:02 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:02.737687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:02.749877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950d7c-2a8f-415a-8fc2-f287cff0fd42 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:02.754501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:02.755107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.755399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.755650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.756322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:02.757952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:02.769464 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1000/1975] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:02.773506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:02.785014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4244421-0e4d-4436-8fd1-172145dabdc0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:02.791216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a5567-1900-4515-817d-a6317b2bc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:02.791841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a5567-1900-4515-817d-a6317b2bc34c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.792121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a5567-1900-4515-817d-a6317b2bc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.792413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a5567-1900-4515-817d-a6317b2bc34c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.793107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a5567-1900-4515-817d-a6317b2bc34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:02.794713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a5567-1900-4515-817d-a6317b2bc34c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:02.807146 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 976/1976] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:02.811315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:02.822357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128b93b-ced9-4e1f-a692-2d4d23644d46 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:02.827855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:02.828589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.828894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.829206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] PATH_INFO: `/v3/projects/c1ac7b64c68e4896b2c631baacdbe2be/users/00423eb1e4c24763ae0a546225ac1dc7/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.845056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ac7b64c68e4896b2c631baacdbe2be, user_id=00423eb1e4c24763ae0a546225ac1dc7, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:02.848468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:02.865388 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1001/1977] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:02 2022] PUT /identity/v3/projects/c1ac7b64c68e4896b2c631baacdbe2be/users/00423eb1e4c24763ae0a546225ac1dc7/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:02.878108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47053ad-7db7-4ace-8647-04893e667ac8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:02.878467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47053ad-7db7-4ace-8647-04893e667ac8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:02.878832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47053ad-7db7-4ace-8647-04893e667ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:02.909374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f47053ad-7db7-4ace-8647-04893e667ac8 None None] MFA Rules not processed for user `00423eb1e4c24763ae0a546225ac1dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:02.910589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f47053ad-7db7-4ace-8647-04893e667ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:02.982816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 977/1978] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:02.983974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:03.047393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a160227-70b2-44e5-873b-e2b708368e0a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:03.052520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:03.053248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.053519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.053784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.054410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:03.056367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:03.085689 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1002/1979] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:02 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:03.090949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47053ad-7db7-4ace-8647-04893e667ac8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:03.106267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47053ad-7db7-4ace-8647-04893e667ac8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:03.122059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:03.184124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:03.184739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.185075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.185275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.186085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:03.187836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:03.216217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 978/1980] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:03 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:03.220278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:03.231520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1fbdd-8a2a-428a-8e9c-ba082576b58b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:03.236430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:03.237087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.237364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.237635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.238419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:03.240014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:03.251856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1003/1981] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:03.256054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:03.267110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7177ce-cf9d-4f1f-9451-2be24667fa15 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:03.271904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:03.272566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.272882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.273157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] PATH_INFO: `/v3/projects/752a253dce3f4a0db3010c597e5533d7/users/82777c78616b4bd6b205a12c2416c420/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.288944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=752a253dce3f4a0db3010c597e5533d7, user_id=82777c78616b4bd6b205a12c2416c420, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:03.291907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:03.309031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 979/1982] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:03 2022] PUT /identity/v3/projects/752a253dce3f4a0db3010c597e5533d7/users/82777c78616b4bd6b205a12c2416c420/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:03.313259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:03.325269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561389a-5fc6-4d5e-801e-e43f64bea902 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:03.367313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:03.397223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e5712-aa0d-4088-80ae-2114c59a969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:03.397848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e5712-aa0d-4088-80ae-2114c59a969c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.398119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e5712-aa0d-4088-80ae-2114c59a969c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.398389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e5712-aa0d-4088-80ae-2114c59a969c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.399378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e5712-aa0d-4088-80ae-2114c59a969c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:03.401118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e5712-aa0d-4088-80ae-2114c59a969c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:03.412364 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1004/1983] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:03.417150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:03.430189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cc75e-de96-497d-b9a7-0a943f17ebdf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:03.435981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46104acf-86dc-4c81-9d8c-fe6e173c75fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:03.436795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46104acf-86dc-4c81-9d8c-fe6e173c75fb admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.437142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46104acf-86dc-4c81-9d8c-fe6e173c75fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.437475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46104acf-86dc-4c81-9d8c-fe6e173c75fb admin admin] PATH_INFO: `/v3/projects/752a253dce3f4a0db3010c597e5533d7/users/82777c78616b4bd6b205a12c2416c420/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.440001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46104acf-86dc-4c81-9d8c-fe6e173c75fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=752a253dce3f4a0db3010c597e5533d7, user_id=82777c78616b4bd6b205a12c2416c420, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:03.442979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46104acf-86dc-4c81-9d8c-fe6e173c75fb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:03.455788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 980/1984] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:03 2022] PUT /identity/v3/projects/752a253dce3f4a0db3010c597e5533d7/users/82777c78616b4bd6b205a12c2416c420/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:03.466482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.466831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.467164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.496625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] MFA Rules not processed for user `82777c78616b4bd6b205a12c2416c420`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:03.498066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:03.592258 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1005/1985] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:03.604628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da17bc-8146-4d16-834e-75dea7bad560 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.604628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da17bc-8146-4d16-834e-75dea7bad560 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.604917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da17bc-8146-4d16-834e-75dea7bad560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.643181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-64da17bc-8146-4d16-834e-75dea7bad560 None None] MFA Rules not processed for user `00423eb1e4c24763ae0a546225ac1dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:03.645465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-64da17bc-8146-4d16-834e-75dea7bad560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:03.740821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 981/1986] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:03.763254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:03.764891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:03.875574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a5f8f-91d7-42a2-bd21-b731ce39f4a8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:03.883163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:03.883794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:03.884125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:03.884332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:03.893602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:03.895213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:03.906068 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1006/1987] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:06.412682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:06.413137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:06.413351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:06.467477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:06.468790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:06.580586 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 982/1988] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Sep 27 09:57:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:06.594364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:06.596539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:06.638076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354851a-c86c-4651-b576-b19d88d8c443 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:06.645238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbb8f9-666e-401f-807c-3fa00d8423c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:06.645699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbb8f9-666e-401f-807c-3fa00d8423c7 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:06.646499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbb8f9-666e-401f-807c-3fa00d8423c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:06.647173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbb8f9-666e-401f-807c-3fa00d8423c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:06.657747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbb8f9-666e-401f-807c-3fa00d8423c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:06.659557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbb8f9-666e-401f-807c-3fa00d8423c7 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:06.670186 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1007/1989] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:07.347445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:07.349355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:07.407372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:07.532793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209429b2-8640-4a13-bbd9-0f6f282b87e5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:07.540528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:07.541936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:07.542481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:07.543215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] PATH_INFO: `/v3/users/abb11855f4bd4764b26791b9cab9b043` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:07.545110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb11855f4bd4764b26791b9cab9b043)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:07.547666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:07.645488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb11855f4bd4764b26791b9cab9b043'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:07.646264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb11855f4bd4764b26791b9cab9b043'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:07.655055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb11855f4bd4764b26791b9cab9b043'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:07.667249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb11855f4bd4764b26791b9cab9b043'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:07.687243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb11855f4bd4764b26791b9cab9b043'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:07.697601 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 983/1990] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:07 2022] DELETE /identity/v3/users/abb11855f4bd4764b26791b9cab9b043 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:07.702914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:07.703351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:07.703351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:07.759563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:07.765310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:07.821742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:07.903769 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1008/1991] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:07.917465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:07.918922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:07.942659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3e957-9363-4cf7-a36e-07f17eb9cfd1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:07.950539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:07.951210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:07.951598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:07.951780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] PATH_INFO: `/v3/users/3cde508c0fb74264acc5faa0acbbf35f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:07.953461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cde508c0fb74264acc5faa0acbbf35f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:07.955639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:07.961037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:07.963590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:08.093454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cde508c0fb74264acc5faa0acbbf35f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:08.095147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cde508c0fb74264acc5faa0acbbf35f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:08.117940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cde508c0fb74264acc5faa0acbbf35f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:08.144549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cde508c0fb74264acc5faa0acbbf35f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:08.169262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bf81e-6550-46ba-821a-48f92c7ad12a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:08.175591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:08.176597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:08.177431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:08.177875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:08.179778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36e55c4f-9894-4cbd-ad89-fb7bc9fcff65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cde508c0fb74264acc5faa0acbbf35f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:08.183484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:08.190735 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 984/1992] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:07 2022] DELETE /identity/v3/users/3cde508c0fb74264acc5faa0acbbf35f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:08.197655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd6b7d-21a8-4b41-b1d7-8b581e57319c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:08.198220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd6b7d-21a8-4b41-b1d7-8b581e57319c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:08.199196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd6b7d-21a8-4b41-b1d7-8b581e57319c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:08.203370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:08.206823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:08.237128 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1009/1993] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:08.245575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b5bd6b7d-21a8-4b41-b1d7-8b581e57319c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:08.247036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b5bd6b7d-21a8-4b41-b1d7-8b581e57319c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:08.254342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:08.333342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 985/1994] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:08.370370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:08.372761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:08.388727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721500-0fca-4c48-9a28-3a8efe472ae4 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:08.465741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c40401-c7be-42e2-acad-edd5975cd542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:08.466725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c40401-c7be-42e2-acad-edd5975cd542 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:08.466725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c40401-c7be-42e2-acad-edd5975cd542 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:08.467636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c40401-c7be-42e2-acad-edd5975cd542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:08.482863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c40401-c7be-42e2-acad-edd5975cd542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:08.484530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c40401-c7be-42e2-acad-edd5975cd542 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:08.498332 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1010/1995] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:08.588695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19725-9f00-4980-8e06-2c73ff65cc8e None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:08.589124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19725-9f00-4980-8e06-2c73ff65cc8e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:08.589268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19725-9f00-4980-8e06-2c73ff65cc8e None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:08.591067 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 986/1996] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:57:08.680413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:08.681944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:08.682279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:08.719085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:08.723251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:08.873612 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1011/1997] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:57:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:09.007613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19725-9f00-4980-8e06-2c73ff65cc8e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:09.010758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06a19725-9f00-4980-8e06-2c73ff65cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:09.059859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19725-9f00-4980-8e06-2c73ff65cc8e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:09.067160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:09.067798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:09.068070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:09.068341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:09.096033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:09.098023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:09.114031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 987/1998] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:09.516407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:09.517911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:09.545654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087276e9-72ff-4249-8ce4-4dc06f167ac8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:09.560518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:09.561416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:09.561873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:09.562072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] PATH_INFO: `/v3/projects/6c6a155df2a04d928c0e39a0b04ebe26` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:09.563499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6a155df2a04d928c0e39a0b04ebe26)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:09.572443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:09.583031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:09.652424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:09.655752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:09.658941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6a155df2a04d928c0e39a0b04ebe26', 'request_id': 'req-3841b639-049e-458a-b218-99d54551e573'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:09.672143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6a155df2a04d928c0e39a0b04ebe26', 'request_id': 'req-3841b639-049e-458a-b218-99d54551e573'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:09.685713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6a155df2a04d928c0e39a0b04ebe26', 'request_id': 'req-3841b639-049e-458a-b218-99d54551e573'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:09.738220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] The token cache is being invalidate because project 6c6a155df2a04d928c0e39a0b04ebe26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:09.739711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:09.742666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:09.747123 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1012/1999] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:09 2022] DELETE /identity/v3/projects/6c6a155df2a04d928c0e39a0b04ebe26 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:09.751575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:09.752434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c5ff-fe1c-4c39-8221-a4059efcfd6f service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:09.755293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:09.925358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:09.926225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:09.926493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:09.926944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] PATH_INFO: `/v3/projects/f08ab464ec0846d6a50c15d130cf88a2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:09.929562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f08ab464ec0846d6a50c15d130cf88a2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:09.932162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:09.935167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841b639-049e-458a-b218-99d54551e573 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:09.942898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:09.947105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:09.947502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:09.947631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] PATH_INFO: `/v3/users/12d9b17377ac46a2b77eb0957690dbaa` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:09.954263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d9b17377ac46a2b77eb0957690dbaa)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:09.957679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:10.042746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f08ab464ec0846d6a50c15d130cf88a2', 'request_id': 'req-8129a0ca-f51e-41d1-a237-708be18e526f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.043636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d9b17377ac46a2b77eb0957690dbaa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.044422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d9b17377ac46a2b77eb0957690dbaa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.057266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f08ab464ec0846d6a50c15d130cf88a2', 'request_id': 'req-8129a0ca-f51e-41d1-a237-708be18e526f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.071804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d9b17377ac46a2b77eb0957690dbaa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.074296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f08ab464ec0846d6a50c15d130cf88a2', 'request_id': 'req-8129a0ca-f51e-41d1-a237-708be18e526f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.089656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] The token cache is being invalidate because project f08ab464ec0846d6a50c15d130cf88a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:10.090292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d9b17377ac46a2b77eb0957690dbaa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.090582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.091590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.092115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d9b17377ac46a2b77eb0957690dbaa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.097217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 988/2000] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:09 2022] DELETE /identity/v3/projects/f08ab464ec0846d6a50c15d130cf88a2 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:10.098533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:10.100338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:10.138441 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1013/2001] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:09 2022] DELETE /identity/v3/users/12d9b17377ac46a2b77eb0957690dbaa => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:10.146293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:10.209640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a0ca-f51e-41d1-a237-708be18e526f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:10.324545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:10.325334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:10.325614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:10.325905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] PATH_INFO: `/v3/users/c274baa6558447ca810435c59673fbab` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:10.327429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c274baa6558447ca810435c59673fbab)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:10.330034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:10.406654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c274baa6558447ca810435c59673fbab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.408018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c274baa6558447ca810435c59673fbab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.419870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c274baa6558447ca810435c59673fbab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.431440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c274baa6558447ca810435c59673fbab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.455392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c274baa6558447ca810435c59673fbab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:10.464098 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 989/2002] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:10 2022] DELETE /identity/v3/users/c274baa6558447ca810435c59673fbab => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:11.337969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:11.341290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:11.379307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:11.384141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:11.389174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3065a0-014b-4aab-81ce-2a9cb7dead13 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:11.463947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:11.527167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:11.543679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c77fd0-a75c-41a6-9aff-32e588bdd6b5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:11.555449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:11.556352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:11.558433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:11.558940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] PATH_INFO: `/v3/projects/17f3acdcf8a645898bbc1d44b0778be9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:11.561422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f3acdcf8a645898bbc1d44b0778be9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:11.566209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:11.590762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:11.591523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:11.592966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:11.593346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] PATH_INFO: `/v3/users/8ea0f9af73d5435088120d793a61bef3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:11.594968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea0f9af73d5435088120d793a61bef3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:11.613277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:11.716026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f3acdcf8a645898bbc1d44b0778be9', 'request_id': 'req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.726603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f3acdcf8a645898bbc1d44b0778be9', 'request_id': 'req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.742614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f3acdcf8a645898bbc1d44b0778be9', 'request_id': 'req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.768747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea0f9af73d5435088120d793a61bef3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.770786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea0f9af73d5435088120d793a61bef3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.776928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] The token cache is being invalidate because project 17f3acdcf8a645898bbc1d44b0778be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:11.777729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.778809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.782615 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 990/2003] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:11 2022] DELETE /identity/v3/projects/17f3acdcf8a645898bbc1d44b0778be9 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:11.783861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:11.786065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:11.790165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea0f9af73d5435088120d793a61bef3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.811831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea0f9af73d5435088120d793a61bef3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.813116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e5a4dbc-0450-4242-b36e-1e2a4bd66680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea0f9af73d5435088120d793a61bef3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:11.855732 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1014/2004] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:11 2022] DELETE /identity/v3/users/8ea0f9af73d5435088120d793a61bef3 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:11.856796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96988fe8-d40e-431b-ad94-e320a27873a6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:11.857082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96988fe8-d40e-431b-ad94-e320a27873a6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:11.857360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96988fe8-d40e-431b-ad94-e320a27873a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:11.898759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-96988fe8-d40e-431b-ad94-e320a27873a6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:11.900211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-96988fe8-d40e-431b-ad94-e320a27873a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:11.917353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9cbc3-26e7-4068-8566-4f8b2f0c39e6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.027381 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1015/2005] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:12.028753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3fb1-32d3-4e64-96f2-068527b180ef None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.029141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3fb1-32d3-4e64-96f2-068527b180ef None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.029513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3fb1-32d3-4e64-96f2-068527b180ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.048317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.049080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.049355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.049746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] PATH_INFO: `/v3/projects/d23ec99b72334ba4ad76bb15521b7fdb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.051312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23ec99b72334ba4ad76bb15521b7fdb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.054632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.072200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5b6a3fb1-32d3-4e64-96f2-068527b180ef None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:12.073641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5b6a3fb1-32d3-4e64-96f2-068527b180ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.092183 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1016/2006] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:12.095770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.096055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.096390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.142958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:12.144490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.156380 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1017/2007] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:12.157296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.158747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.177634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:12.207062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:12.207278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23ec99b72334ba4ad76bb15521b7fdb', 'request_id': 'req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.219251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23ec99b72334ba4ad76bb15521b7fdb', 'request_id': 'req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.233388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23ec99b72334ba4ad76bb15521b7fdb', 'request_id': 'req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.251409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] The token cache is being invalidate because project d23ec99b72334ba4ad76bb15521b7fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:12.251841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.252901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.260426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 991/2008] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:11 2022] DELETE /identity/v3/projects/d23ec99b72334ba4ad76bb15521b7fdb => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:12.260426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.260426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.295804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf10a1-67d5-4a0e-9f88-53c87c43ced9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.385767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdde43d-c398-48ec-8e7d-5ce3587c4f66 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.389758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932c5f8-29e1-4772-a723-46f1ed2c3c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.390564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5f8-29e1-4772-a723-46f1ed2c3c0e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.390751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5f8-29e1-4772-a723-46f1ed2c3c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.391068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5f8-29e1-4772-a723-46f1ed2c3c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.392914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b932c5f8-29e1-4772-a723-46f1ed2c3c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.397749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.398877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.400336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.400884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.402041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:12.402569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.406365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.407836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932c5f8-29e1-4772-a723-46f1ed2c3c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.409432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932c5f8-29e1-4772-a723-46f1ed2c3c0e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.425199 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1018/2009] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:12.430492 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 992/2010] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:12.434661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.438923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78207c-41d4-49f0-92c4-6dd3b5712458 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.440990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78207c-41d4-49f0-92c4-6dd3b5712458 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.441456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78207c-41d4-49f0-92c4-6dd3b5712458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.463682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c58eb-b9f2-4fe9-ba47-cb3d5267b44e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.469897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.470535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0b78207c-41d4-49f0-92c4-6dd3b5712458 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:12.471339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.471610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.471965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.472962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.474440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0b78207c-41d4-49f0-92c4-6dd3b5712458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.474909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.491830 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1019/2011] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:12.499225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.499528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.499874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.515087 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 993/2012] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:12.519194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.537859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:12.539068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.646686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60a1d5-31a5-421a-ae85-e2f5904952f0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.652796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.653374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.653650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.653917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.654858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.656547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.668539 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1020/2013] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:12.670450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.672632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.687093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 994/2014] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:12.687093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.687430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e28f7-430d-4e29-8d59-0df047523659 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.692530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.693231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.693513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.693787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] PATH_INFO: `/v3/users/023730e34b2d4e2a996bfa068a97413c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.694966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=023730e34b2d4e2a996bfa068a97413c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.697309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.698779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107eb1f-6774-49ec-896c-4b4afca1189c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.704751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.705436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.705917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.706232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.706988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:12.707359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.709493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.724888 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 995/2015] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:12.726072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.737487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dd32b-f531-4d77-85e6-3a329470ca53 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.744308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.744978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.745260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.745535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.748700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.748700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.759018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:12.763928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023730e34b2d4e2a996bfa068a97413c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.764867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023730e34b2d4e2a996bfa068a97413c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.769256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 996/2016] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:12.770844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.774367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.775940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023730e34b2d4e2a996bfa068a97413c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.787727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023730e34b2d4e2a996bfa068a97413c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.788471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023730e34b2d4e2a996bfa068a97413c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:12.791595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:12.792724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7ea38-2aa8-4a8a-b189-dfdee8b2ba70 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.799263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ab443-8f84-4717-a5d3-98efd9bbae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.800066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ab443-8f84-4717-a5d3-98efd9bbae0c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.800467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ab443-8f84-4717-a5d3-98efd9bbae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.800892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ab443-8f84-4717-a5d3-98efd9bbae0c admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.801593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ab443-8f84-4717-a5d3-98efd9bbae0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:12.803989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ab443-8f84-4717-a5d3-98efd9bbae0c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:12.815352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1021/2017] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:12 2022] DELETE /identity/v3/users/023730e34b2d4e2a996bfa068a97413c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:12.817716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.819965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.839913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 997/2018] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/projects => generated 399 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:12.842263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a14ea-fada-40b9-a508-3094b5b65e99 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.842659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a14ea-fada-40b9-a508-3094b5b65e99 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.844343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a14ea-fada-40b9-a508-3094b5b65e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:12.869829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ea1a14ea-fada-40b9-a508-3094b5b65e99 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:12.872016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ea1a14ea-fada-40b9-a508-3094b5b65e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:12.913534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3433e-6ed3-4359-b516-77260dc751fa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.965917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 998/2019] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:12.966915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a14ea-fada-40b9-a508-3094b5b65e99 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:12.989569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a14ea-fada-40b9-a508-3094b5b65e99 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:12.997022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:12.998074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:12.998926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:12.999382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.000526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.004268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.055770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.057002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.057482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.057954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.061082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.065059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.071172 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 999/2020] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:12 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:13.071172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.072346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.094706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1022/2021] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:13.095928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.098128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.100246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae5673-8a8d-48eb-a7ff-42671f063cf4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.107284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.108759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.109250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.109731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] PATH_INFO: `/v3/projects/5a71603028ff42b4847878e7ede6aebe` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.111379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a71603028ff42b4847878e7ede6aebe)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.114327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.195227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffc1c4-4d69-4eda-b7ae-e446d2c4693e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.208449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.210377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.210885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.211672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.228888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.230974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.248560 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1023/2022] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:13.250305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a71603028ff42b4847878e7ede6aebe', 'request_id': 'req-4b72fd72-5b83-4899-87f3-6c97a95ab854'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.251092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.261874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a71603028ff42b4847878e7ede6aebe', 'request_id': 'req-4b72fd72-5b83-4899-87f3-6c97a95ab854'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.262945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078262e-209b-4672-a094-014ab75e58a9 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.275468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a71603028ff42b4847878e7ede6aebe', 'request_id': 'req-4b72fd72-5b83-4899-87f3-6c97a95ab854'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.298049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] The token cache is being invalidate because project 5a71603028ff42b4847878e7ede6aebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:13.298349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.301282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.305197 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1000/2023] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:13 2022] DELETE /identity/v3/projects/5a71603028ff42b4847878e7ede6aebe => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:13.306203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.309101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:13.312304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.363022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.363022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.363022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.363022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.363497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.365418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.389726 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1024/2024] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:13.389726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.392632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.406665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54b84-babb-4370-8e7c-befafb5a3708 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.407139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:13.414005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.415061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.415503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.415872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] PATH_INFO: `/v3/users/49f35168484b42feb22d106d745e6546` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.417352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49f35168484b42feb22d106d745e6546)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.419673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.462790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72fd72-5b83-4899-87f3-6c97a95ab854 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.469710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.470782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.471209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.471717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] PATH_INFO: `/v3/projects/3132070d235941498e3f64d9d7075f6b/users/7ba3c51770094c4ea7fec1e597ae13a0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.494391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3132070d235941498e3f64d9d7075f6b, user_id=7ba3c51770094c4ea7fec1e597ae13a0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.497893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.520516 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1001/2025] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:13 2022] PUT /identity/v3/projects/3132070d235941498e3f64d9d7075f6b/users/7ba3c51770094c4ea7fec1e597ae13a0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:13.521485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.525085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.549946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49f35168484b42feb22d106d745e6546'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.551576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49f35168484b42feb22d106d745e6546'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.565290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49f35168484b42feb22d106d745e6546'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.577759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49f35168484b42feb22d106d745e6546'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.579193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7bdec890-ed89-498c-baf9-2fb5204a9f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49f35168484b42feb22d106d745e6546'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:13.609226 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1025/2026] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:13 2022] DELETE /identity/v3/users/49f35168484b42feb22d106d745e6546 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:13.610668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36292ae5-0104-4702-a35e-d960d299424a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.611060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36292ae5-0104-4702-a35e-d960d299424a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.611337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36292ae5-0104-4702-a35e-d960d299424a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.652318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-36292ae5-0104-4702-a35e-d960d299424a None None] MFA Rules not processed for user `7ba3c51770094c4ea7fec1e597ae13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:13.655823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36292ae5-0104-4702-a35e-d960d299424a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.681469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99ba17-d029-4be2-a3d3-ac93d6dcf01d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.851158 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1026/2027] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:13.852011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.853388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.858932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36292ae5-0104-4702-a35e-d960d299424a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.859800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.860120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.861011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.862931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36292ae5-0104-4702-a35e-d960d299424a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.866648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.895359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36292ae5-0104-4702-a35e-d960d299424a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.897077 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1002/2028] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:13.902037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:13.908930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:13.918498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.923105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.924177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.924622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.925608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.928930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.935877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf608d-a01c-475c-aa9c-fb66c9daefad admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:13.945961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b4d6b-062f-4e9b-8cee-d880acb1ac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:13.946733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b4d6b-062f-4e9b-8cee-d880acb1ac4c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:13.947108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b4d6b-062f-4e9b-8cee-d880acb1ac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:13.947393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b4d6b-062f-4e9b-8cee-d880acb1ac4c admin admin] PATH_INFO: `/v3/projects/b19fa985d89f459ba0e811b1779dc602/users/2e121c7ab3e2401eb5a56962d2815b38/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:13.974768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b4d6b-062f-4e9b-8cee-d880acb1ac4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19fa985d89f459ba0e811b1779dc602, user_id=2e121c7ab3e2401eb5a56962d2815b38, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:13.984493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b4d6b-062f-4e9b-8cee-d880acb1ac4c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:13.998024 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1027/2029] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:13 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:14.005368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.012771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:14.028709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1003/2030] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:13 2022] PUT /identity/v3/projects/b19fa985d89f459ba0e811b1779dc602/users/2e121c7ab3e2401eb5a56962d2815b38/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:14.035543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a32e-3e81-4061-b360-5ef77da545fe None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.037116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a32e-3e81-4061-b360-5ef77da545fe None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.040369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a32e-3e81-4061-b360-5ef77da545fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.123143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fee1a32e-3e81-4061-b360-5ef77da545fe None None] MFA Rules not processed for user `2e121c7ab3e2401eb5a56962d2815b38`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:14.126180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fee1a32e-3e81-4061-b360-5ef77da545fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:14.192417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:14.221812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fe7db-beb1-4287-bd64-636ed41e5367 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.283065 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1004/2031] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:14.289649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1a32e-3e81-4061-b360-5ef77da545fe None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.373041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.373722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.374269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.374835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.376673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.378964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.411639 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1028/2032] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:14 2022] POST /identity/v3/users => generated 301 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:14.417608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.433558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010f78a-9781-44da-944a-94c9ef7ee73c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.447786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.450316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.450694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.451100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.452133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.453621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.474326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1029/2033] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:14.474594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1a32e-3e81-4061-b360-5ef77da545fe None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.477031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.480597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.481886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.482365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.482896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.484442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.488186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.489773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49678201-ff9f-4b39-b26e-01bb52fd3cb8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.495841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.496744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.497008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.497270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] PATH_INFO: `/v3/projects/d0ffaa13af1744fd92951b2207c6bea2/users/324bddae7f37458e9217889fdf43c6eb/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.519234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ffaa13af1744fd92951b2207c6bea2, user_id=324bddae7f37458e9217889fdf43c6eb, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.522407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.549519 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1030/2034] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:14 2022] PUT /identity/v3/projects/d0ffaa13af1744fd92951b2207c6bea2/users/324bddae7f37458e9217889fdf43c6eb/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:14.552420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.568194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1005/2035] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:14 2022] POST /identity/v3/projects => generated 399 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:14.571522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.687281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe75d1-718c-47a6-a42e-5720d28d9e93 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.694875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.696573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.696856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.697299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.697970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.700102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.700934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb433ee5-b5cb-428c-8df4-7a0481f27024 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.708233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.709713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.710805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.711473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.713046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.715925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.741141 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1006/2036] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:14 2022] POST /identity/v3/users => generated 306 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:14.745916 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1031/2037] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:14.749053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.749388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.768387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:14.780550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e7007-ad19-4c77-9bd6-19c6637d5f42 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.781033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236d81-3ef2-43f9-a3b8-225f2737e108 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.806488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.808002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.809195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.809195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.809782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:14.810995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.811556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:14.812070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:14.812534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] PATH_INFO: `/v3/projects/d0ffaa13af1744fd92951b2207c6bea2/users/324bddae7f37458e9217889fdf43c6eb/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:14.815807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.817338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ffaa13af1744fd92951b2207c6bea2, user_id=324bddae7f37458e9217889fdf43c6eb, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:14.820988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:14.849002 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1007/2038] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:14.860707 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1032/2039] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:14 2022] PUT /identity/v3/projects/d0ffaa13af1744fd92951b2207c6bea2/users/324bddae7f37458e9217889fdf43c6eb/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:14.861151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.861318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:14.861416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:14.897447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca453988-08e9-40de-905a-533bd0cb9ebc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:14.992826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:15.025208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d85834-9b02-4fdb-bd55-e8eb932cb210 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:15.030656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec794-8bf6-4053-90fe-45b0b9deab5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:15.031347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec794-8bf6-4053-90fe-45b0b9deab5d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.031759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec794-8bf6-4053-90fe-45b0b9deab5d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.032039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec794-8bf6-4053-90fe-45b0b9deab5d admin admin] PATH_INFO: `/v3/projects/991c193b9ac144c18fc25251ae253764/users/5430c5ee5cbf4702b9f5056a47b02ad2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.041654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:15.042360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.042645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.042964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] PATH_INFO: `/v3/projects/681fc4282a6c4fd8ba4f078705f4ca5c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.044289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=681fc4282a6c4fd8ba4f078705f4ca5c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:15.047464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:15.064353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ec794-8bf6-4053-90fe-45b0b9deab5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=991c193b9ac144c18fc25251ae253764, user_id=5430c5ee5cbf4702b9f5056a47b02ad2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:15.068634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ec794-8bf6-4053-90fe-45b0b9deab5d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:15.110529 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1033/2040] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:14 2022] PUT /identity/v3/projects/991c193b9ac144c18fc25251ae253764/users/5430c5ee5cbf4702b9f5056a47b02ad2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:15.115758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.116333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.117092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.140737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681fc4282a6c4fd8ba4f078705f4ca5c', 'request_id': 'req-863d94ea-0b3f-431b-a44d-2c753bcbe39a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.155352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681fc4282a6c4fd8ba4f078705f4ca5c', 'request_id': 'req-863d94ea-0b3f-431b-a44d-2c753bcbe39a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.161357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] MFA Rules not processed for user `324bddae7f37458e9217889fdf43c6eb`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:15.163309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.176356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681fc4282a6c4fd8ba4f078705f4ca5c', 'request_id': 'req-863d94ea-0b3f-431b-a44d-2c753bcbe39a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.243163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] The token cache is being invalidate because project 681fc4282a6c4fd8ba4f078705f4ca5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:15.243661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.245166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.249282 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1008/2041] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:14 2022] DELETE /identity/v3/projects/681fc4282a6c4fd8ba4f078705f4ca5c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:15.251759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:15.255399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.390663 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1034/2042] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:15.391922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:15.393403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.437130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d94ea-0b3f-431b-a44d-2c753bcbe39a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:15.444040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d05706-881f-4adc-9f5e-e075bdbfe5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:15.445389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05706-881f-4adc-9f5e-e075bdbfe5fe admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.446116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05706-881f-4adc-9f5e-e075bdbfe5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.446656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05706-881f-4adc-9f5e-e075bdbfe5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.447683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d05706-881f-4adc-9f5e-e075bdbfe5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:15.449990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d05706-881f-4adc-9f5e-e075bdbfe5fe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:15.467330 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1009/2043] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:15.469351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a1f1-6edd-49ee-a333-f00ba6bf967c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.470746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a1f1-6edd-49ee-a333-f00ba6bf967c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.471251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a1f1-6edd-49ee-a333-f00ba6bf967c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.499101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1677a1f1-6edd-49ee-a333-f00ba6bf967c None None] MFA Rules not processed for user `7ba3c51770094c4ea7fec1e597ae13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:15.500524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1677a1f1-6edd-49ee-a333-f00ba6bf967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.509421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:15.515808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10dedf-2a46-40be-bbc2-d79bd212f690 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:15.531656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:15.531656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.532217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.533312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] PATH_INFO: `/v3/users/e0808d9ea0734660b695111afab964e3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.535972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0808d9ea0734660b695111afab964e3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:15.537956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:15.598625 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1010/2044] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:15.600366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677a1f1-6edd-49ee-a333-f00ba6bf967c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:15.622365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677a1f1-6edd-49ee-a333-f00ba6bf967c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:15.637530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:15.638298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.638736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.639127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] PATH_INFO: `/v3/projects/991c193b9ac144c18fc25251ae253764/users/5430c5ee5cbf4702b9f5056a47b02ad2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.641479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991c193b9ac144c18fc25251ae253764, user_id=5430c5ee5cbf4702b9f5056a47b02ad2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:15.642300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0808d9ea0734660b695111afab964e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.642997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0808d9ea0734660b695111afab964e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.644493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:15.652293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:15.656500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0808d9ea0734660b695111afab964e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.670281 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1011/2045] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:15 2022] PUT /identity/v3/projects/991c193b9ac144c18fc25251ae253764/users/5430c5ee5cbf4702b9f5056a47b02ad2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:15.670281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:15.670281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.675282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0808d9ea0734660b695111afab964e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.676048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0808d9ea0734660b695111afab964e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.701761 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1035/2046] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:15 2022] DELETE /identity/v3/users/e0808d9ea0734660b695111afab964e3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:15.702841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:15.704250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.751671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed03e19-4530-4b1a-a41f-03e1d4a30140 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:15.810308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a350-5646-4012-b8f2-53fe3c713dc6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:15.818225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609e17f-baff-4929-af09-1e3335d4c4b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:15.818872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e17f-baff-4929-af09-1e3335d4c4b2 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.819187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e17f-baff-4929-af09-1e3335d4c4b2 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.819678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e17f-baff-4929-af09-1e3335d4c4b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.821982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1609e17f-baff-4929-af09-1e3335d4c4b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.824567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:15.825457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.825994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.826382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] PATH_INFO: `/v3/projects/fe46a7ae8eab451e941f83e19be3b539` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.827687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe46a7ae8eab451e941f83e19be3b539)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:15.829770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:15.908841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe46a7ae8eab451e941f83e19be3b539', 'request_id': 'req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.909013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609e17f-baff-4929-af09-1e3335d4c4b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:15.909111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe46a7ae8eab451e941f83e19be3b539', 'request_id': 'req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.909203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609e17f-baff-4929-af09-1e3335d4c4b2 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:15.917607 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1036/2047] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:15.918649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d624c7-3ace-442d-941f-621abd2845ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:15.918903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d624c7-3ace-442d-941f-621abd2845ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:15.919179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d624c7-3ace-442d-941f-621abd2845ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:15.922060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe46a7ae8eab451e941f83e19be3b539', 'request_id': 'req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.953006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:15.957399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e0d624c7-3ace-442d-941f-621abd2845ee None None] MFA Rules not processed for user `5430c5ee5cbf4702b9f5056a47b02ad2`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:15.958310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] The token cache is being invalidate because project fe46a7ae8eab451e941f83e19be3b539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:15.959977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e0d624c7-3ace-442d-941f-621abd2845ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:15.961002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.962037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:15.966478 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1012/2048] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:15 2022] DELETE /identity/v3/projects/fe46a7ae8eab451e941f83e19be3b539 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:15.967686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:15.972829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.040952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:16.077959 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1037/2049] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:15 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:16.081164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.081441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.081715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.090756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c4c3f-48ed-4405-9efc-6b7f3cddf7db admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.095863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.096943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.096943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.097623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] PATH_INFO: `/v3/projects/dc9d0df2395744b0baafe373c8913b91` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.098278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9d0df2395744b0baafe373c8913b91)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.101137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.114220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:16.115800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.142844 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1038/2050] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:16.144248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.145907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.170192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9d0df2395744b0baafe373c8913b91', 'request_id': 'req-39ffa97d-4591-49f0-8956-a39a8fa154ac'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:16.178185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9d0df2395744b0baafe373c8913b91', 'request_id': 'req-39ffa97d-4591-49f0-8956-a39a8fa154ac'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:16.190063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9d0df2395744b0baafe373c8913b91', 'request_id': 'req-39ffa97d-4591-49f0-8956-a39a8fa154ac'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:16.203905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] The token cache is being invalidate because project dc9d0df2395744b0baafe373c8913b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:16.204392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:16.205189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39ffa97d-4591-49f0-8956-a39a8fa154ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:16.207372 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1013/2051] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:15 2022] DELETE /identity/v3/projects/dc9d0df2395744b0baafe373c8913b91 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:16.208922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a95d3f-ffcb-4942-994d-ec77bf705da0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.209471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a95d3f-ffcb-4942-994d-ec77bf705da0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.209845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a95d3f-ffcb-4942-994d-ec77bf705da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.233216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-34a95d3f-ffcb-4942-994d-ec77bf705da0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:16.234498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-34a95d3f-ffcb-4942-994d-ec77bf705da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.235137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ea268-927c-4753-95d3-e403cbe3cb95 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.312003 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1014/2052] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:16.313351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf6ce6-c789-483b-aab5-340b6e32e8a6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.313701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf6ce6-c789-483b-aab5-340b6e32e8a6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.314153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf6ce6-c789-483b-aab5-340b6e32e8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.343397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.344274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.344722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.345138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.347826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.348933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cdbf6ce6-c789-483b-aab5-340b6e32e8a6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:16.350288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cdbf6ce6-c789-483b-aab5-340b6e32e8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.369604 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1015/2053] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:16.369604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf6ce6-c789-483b-aab5-340b6e32e8a6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.384911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf6ce6-c789-483b-aab5-340b6e32e8a6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.390181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7361c-8ff9-4520-b424-8768289b4be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.391402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7361c-8ff9-4520-b424-8768289b4be5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.391775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7361c-8ff9-4520-b424-8768289b4be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.392315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7361c-8ff9-4520-b424-8768289b4be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.393299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7361c-8ff9-4520-b424-8768289b4be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:16.393734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7361c-8ff9-4520-b424-8768289b4be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.395948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7361c-8ff9-4520-b424-8768289b4be5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.397168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:16.412705 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1016/2054] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:16.413420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f976f8-9a48-4994-8a84-7598d4ae5cc2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.413893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f976f8-9a48-4994-8a84-7598d4ae5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.414417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f976f8-9a48-4994-8a84-7598d4ae5cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.447831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.449603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.468649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-02f976f8-9a48-4994-8a84-7598d4ae5cc2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:16.470307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02f976f8-9a48-4994-8a84-7598d4ae5cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.476533 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1039/2055] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:16.478894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.480864 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1017/2056] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:16.485457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f976f8-9a48-4994-8a84-7598d4ae5cc2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.508515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f976f8-9a48-4994-8a84-7598d4ae5cc2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.511874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54243f-4fb3-493a-903f-5e6a868158a9 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.514359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.515075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.515353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.515622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.516360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:16.516733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.518480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.521106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.521751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.522021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.522302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.523039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.526467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.533363 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1018/2057] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:16.538160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.553103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde3833-1926-423a-9842-e355a7998ba4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.561044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.561977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.562280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.562558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.563467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.571169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.591551 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1040/2058] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:16.596473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.623534 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1019/2059] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:16.624998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.639402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:16.767103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988761d5-187b-4098-914d-aa772878ab9d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.813792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4567e6-b35e-456d-b163-3c0c706ecf3d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:16.835857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.839037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.839037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.839472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.840067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.843482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.905066 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1020/2060] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:16.907261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:16.910972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:16.915688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:16.917748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:16.917748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:16.917748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:16.918921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:16.922823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:16.987458 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1041/2061] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:16 2022] POST /identity/v3/users => generated 292 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:16.989239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.016840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02762-904f-424e-a633-a43aa728b657 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.030628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd985be-e34f-453e-ba47-de8123209f0a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.037744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.038485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.038824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.039189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.041973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:17.055214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.056337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.056858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.061380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.064835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.068499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.088958 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1042/2062] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:17.090060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.109727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65426a3-2678-4519-a986-8cc6b9e8e879 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.122410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.123037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.123310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.123585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.124283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.126004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.144279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.146176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.162701 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1043/2063] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:17.165616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.176396 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1021/2064] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:16 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:17.178142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.182713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321fdc8-bb94-41df-bbd2-1d10f21036e2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.188617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.189557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.189841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.190113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.191398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.193205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.209658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f393fa-c43f-437d-ad8e-de759370e6a2 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.213971 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1044/2065] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:17.218499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.224931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.225591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.225905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.226229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.227181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.229642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.243083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ad29-f69d-441b-8af0-1ddbaa0e831b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.245251 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1022/2066] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:17.255282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0e6b2-16bc-4165-bbea-ab39997dbaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.256120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0e6b2-16bc-4165-bbea-ab39997dbaeb admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.256382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0e6b2-16bc-4165-bbea-ab39997dbaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.256716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0e6b2-16bc-4165-bbea-ab39997dbaeb admin admin] PATH_INFO: `/v3/projects/0497f0d3056745aeaf7678f81638727f/users/906058793cf941448ed34f9dcc28db12/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.257100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.261818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:17.279765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e76a0-d341-41fc-95b7-8e29aa33c6c2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.287146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70107bc4-deab-43dd-afc7-a69c53c58fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.288043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70107bc4-deab-43dd-afc7-a69c53c58fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.288323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70107bc4-deab-43dd-afc7-a69c53c58fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.288467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0e6b2-16bc-4165-bbea-ab39997dbaeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0497f0d3056745aeaf7678f81638727f, user_id=906058793cf941448ed34f9dcc28db12, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.288796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70107bc4-deab-43dd-afc7-a69c53c58fb7 admin admin] PATH_INFO: `/v3/projects/e4e6fb6dcd18472c8216eb65702eb01c/users/5db7c4283f9f45a495a9bd839bccc229/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.293954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0e6b2-16bc-4165-bbea-ab39997dbaeb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.310605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:17.312759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70107bc4-deab-43dd-afc7-a69c53c58fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e6fb6dcd18472c8216eb65702eb01c, user_id=5db7c4283f9f45a495a9bd839bccc229, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.316287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70107bc4-deab-43dd-afc7-a69c53c58fb7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.332800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1045/2067] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:17 2022] PUT /identity/v3/projects/0497f0d3056745aeaf7678f81638727f/users/906058793cf941448ed34f9dcc28db12/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:17.345449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6647a57-6486-4143-b83d-c54075418277 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.345764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6647a57-6486-4143-b83d-c54075418277 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.346053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6647a57-6486-4143-b83d-c54075418277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.365921 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1023/2068] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:17 2022] PUT /identity/v3/projects/e4e6fb6dcd18472c8216eb65702eb01c/users/5db7c4283f9f45a495a9bd839bccc229/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:17.376527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.377035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.377319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.409434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c6647a57-6486-4143-b83d-c54075418277 None None] MFA Rules not processed for user `906058793cf941448ed34f9dcc28db12`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:17.410758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c6647a57-6486-4143-b83d-c54075418277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:17.429336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] MFA Rules not processed for user `5db7c4283f9f45a495a9bd839bccc229`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:17.433561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:17.537282 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1024/2069] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:17.538253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.540396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:17.545100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1046/2070] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:17.549912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6647a57-6486-4143-b83d-c54075418277 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.551488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c6647a57-6486-4143-b83d-c54075418277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:17.639519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65955-11bc-4b01-9612-df6c03d3e467 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.642696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6647a57-6486-4143-b83d-c54075418277 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.648172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.649156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.649446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.649728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] PATH_INFO: `/v3/users/2e121c7ab3e2401eb5a56962d2815b38` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.650430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.651060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e121c7ab3e2401eb5a56962d2815b38)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.651174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.651411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.651674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.653298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.667591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.671215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.691015 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1047/2071] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:57:17.691956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.704934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7085900-7079-44bc-8384-d8c0caa9429c service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:17.710670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:17.711658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.712496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.712868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.713580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:17.715822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:17.763581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e121c7ab3e2401eb5a56962d2815b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:17.764449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e121c7ab3e2401eb5a56962d2815b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:17.770384 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1048/2072] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:17 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:17.774928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:17.777423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:17.785739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e121c7ab3e2401eb5a56962d2815b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:17.805429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e121c7ab3e2401eb5a56962d2815b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:17.844956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-15672f3d-d50c-45bc-a49b-46ce6edb737a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e121c7ab3e2401eb5a56962d2815b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:17.849010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:17.862611 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1025/2073] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:17 2022] DELETE /identity/v3/users/2e121c7ab3e2401eb5a56962d2815b38 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:17.864828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:17.865326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:17.865901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:17.895025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:17.895936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:17.912039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:17.927899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2131d36-0720-4b47-84af-eafa7891b55e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.027823 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1026/2074] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:18.042966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.045368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:18.056485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.057163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.057436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.057700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.058362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.060849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.112597 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1049/2075] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:17 2022] POST /identity/v3/users => generated 292 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:18.112597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.113557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:18.138626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc8f36-b3fc-4566-af83-5b904a6ac01e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.151479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed0b50-96df-4b49-a933-078b349d34bb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.154407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.155993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.156552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.157139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] PATH_INFO: `/v3/users/5db7c4283f9f45a495a9bd839bccc229` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.157792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.158692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.159093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db7c4283f9f45a495a9bd839bccc229)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.159337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.160160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.162135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.172224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.175781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.221597 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1027/2076] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:57:18.222880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.235134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c397a0d-659e-475e-8b15-773c7b305297 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.241335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.241971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.242251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.242680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.243313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.245220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.262116 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1028/2077] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:18.263260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.271609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db7c4283f9f45a495a9bd839bccc229'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:18.272373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db7c4283f9f45a495a9bd839bccc229'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:18.285298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db7c4283f9f45a495a9bd839bccc229'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:18.302109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cd328-d63e-4098-8a1b-ea0b3b500788 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.306630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db7c4283f9f45a495a9bd839bccc229'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:18.310696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6000405-44ce-4709-a535-0fed449d7095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db7c4283f9f45a495a9bd839bccc229'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:18.311003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.311703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.311978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.312250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] PATH_INFO: `/v3/projects/edf104c9f1ab417bbd1bba5a7627f443/users/87ed26b2c06d42c6bfcb4fa2786a6b38/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.343306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf104c9f1ab417bbd1bba5a7627f443, user_id=87ed26b2c06d42c6bfcb4fa2786a6b38, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.347346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.355336 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1050/2078] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:18 2022] DELETE /identity/v3/users/5db7c4283f9f45a495a9bd839bccc229 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:18.360576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.360950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.361425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.379700 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1029/2079] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:18 2022] PUT /identity/v3/projects/edf104c9f1ab417bbd1bba5a7627f443/users/87ed26b2c06d42c6bfcb4fa2786a6b38/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:18.385575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:18.385876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.387127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:18.388848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:18.504389 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1051/2080] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:18.515457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2fdd0-bb06-4833-aed6-4c35b0bbe341 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.517946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.522718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:18.530006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.530688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:18.532025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.532311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.532560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.533376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.534997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.554506 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1030/2081] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:18.559823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.568414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:18.575068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd74eb-0dd1-4529-83a3-e472ac497bfe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.584202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32852c71-8770-45af-8e37-55ff99251888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.585543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32852c71-8770-45af-8e37-55ff99251888 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.590555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32852c71-8770-45af-8e37-55ff99251888 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.590555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32852c71-8770-45af-8e37-55ff99251888 admin admin] PATH_INFO: `/v3/projects/edf104c9f1ab417bbd1bba5a7627f443/users/87ed26b2c06d42c6bfcb4fa2786a6b38/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.591353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32852c71-8770-45af-8e37-55ff99251888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf104c9f1ab417bbd1bba5a7627f443, user_id=87ed26b2c06d42c6bfcb4fa2786a6b38, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.595542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32852c71-8770-45af-8e37-55ff99251888 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.619021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1031/2082] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:18 2022] PUT /identity/v3/projects/edf104c9f1ab417bbd1bba5a7627f443/users/87ed26b2c06d42c6bfcb4fa2786a6b38/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:18.630483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.630969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.631434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.657865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa07242-6ff2-4c6c-b267-29dbfce3c2b4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.673206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] MFA Rules not processed for user `87ed26b2c06d42c6bfcb4fa2786a6b38`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:18.675044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:18.738297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.738472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.738852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.739360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.788107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1032/2083] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:18.802666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:18.804829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:18.832834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.834540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.848979 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1052/2084] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:18.936482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14d7a1-7c00-4a2d-8c58-011f38eabd27 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:18.944476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:18.945305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:18.945769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:18.946288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:18.965226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:18.967478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:18.984015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1033/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:19.086566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:19.097005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fefa2-3aa3-4de8-a8d8-cc73f13df798 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:19.101847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:19.102504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.102836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.103037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] PATH_INFO: `/v3/users/906058793cf941448ed34f9dcc28db12` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.104231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] RBAC: Authorizing `identity:delete_user(user_id=906058793cf941448ed34f9dcc28db12)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:19.106237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:19.147355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:19.151947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:19.168475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc56d8-139a-41ba-a34f-eb61b7ec4fc1 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:19.174630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:19.175520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.175977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.177532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] PATH_INFO: `/v3/projects/b19fa985d89f459ba0e811b1779dc602` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.178021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19fa985d89f459ba0e811b1779dc602)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:19.180166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:19.193855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906058793cf941448ed34f9dcc28db12'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.196485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906058793cf941448ed34f9dcc28db12'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.227460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906058793cf941448ed34f9dcc28db12'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.241977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906058793cf941448ed34f9dcc28db12'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.243142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-67c90b7d-4680-4809-9dca-21b6b38f702b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906058793cf941448ed34f9dcc28db12'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.246498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:19.252537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19fa985d89f459ba0e811b1779dc602', 'request_id': 'req-e2e79819-357e-47e6-9c37-a2cb56efc0a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.263444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19fa985d89f459ba0e811b1779dc602', 'request_id': 'req-e2e79819-357e-47e6-9c37-a2cb56efc0a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.281715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:19.287116 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1053/2086] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:19 2022] DELETE /identity/v3/users/906058793cf941448ed34f9dcc28db12 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:19.290345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.291852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.292332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19fa985d89f459ba0e811b1779dc602', 'request_id': 'req-e2e79819-357e-47e6-9c37-a2cb56efc0a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.297753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.311800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] The token cache is being invalidate because project b19fa985d89f459ba0e811b1779dc602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:19.312183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.313085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.316121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1034/2087] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:19 2022] DELETE /identity/v3/projects/b19fa985d89f459ba0e811b1779dc602 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:19.320222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:19.323131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:19.341914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:19.353059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:19.477762 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1054/2088] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:19.479680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:19.482092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e79819-357e-47e6-9c37-a2cb56efc0a2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:19.482638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:19.490247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:19.491489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.491962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.493013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] PATH_INFO: `/v3/users/5430c5ee5cbf4702b9f5056a47b02ad2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.494318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5430c5ee5cbf4702b9f5056a47b02ad2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:19.496744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:19.504141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513be77-faee-4ff4-95f5-e3b94d165b07 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:19.509842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:19.510598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.510879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.511326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] PATH_INFO: `/v3/projects/e4e6fb6dcd18472c8216eb65702eb01c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.512476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e6fb6dcd18472c8216eb65702eb01c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:19.514459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:19.633828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5430c5ee5cbf4702b9f5056a47b02ad2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.634774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5430c5ee5cbf4702b9f5056a47b02ad2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.637840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e6fb6dcd18472c8216eb65702eb01c', 'request_id': 'req-efb9bea2-245a-40e1-988b-19a93a699ca8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.644030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5430c5ee5cbf4702b9f5056a47b02ad2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.650809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e6fb6dcd18472c8216eb65702eb01c', 'request_id': 'req-efb9bea2-245a-40e1-988b-19a93a699ca8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.659728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e6fb6dcd18472c8216eb65702eb01c', 'request_id': 'req-efb9bea2-245a-40e1-988b-19a93a699ca8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.667821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5430c5ee5cbf4702b9f5056a47b02ad2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.679020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] The token cache is being invalidate because project e4e6fb6dcd18472c8216eb65702eb01c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:19.679338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.681346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.684453 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1055/2089] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:19 2022] DELETE /identity/v3/projects/e4e6fb6dcd18472c8216eb65702eb01c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:19.684976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:19.688820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:19.698544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae6b07cb-a9ba-4c31-93b7-736d82a47659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5430c5ee5cbf4702b9f5056a47b02ad2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:19.712598 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1035/2090] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:19 2022] DELETE /identity/v3/users/5430c5ee5cbf4702b9f5056a47b02ad2 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:19.715413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6480b0-7bc2-4805-b581-3040b965b924 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.715667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6480b0-7bc2-4805-b581-3040b965b924 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.716008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6480b0-7bc2-4805-b581-3040b965b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.794374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7a6480b0-7bc2-4805-b581-3040b965b924 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:19.795834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7a6480b0-7bc2-4805-b581-3040b965b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:19.867900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9bea2-245a-40e1-988b-19a93a699ca8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:19.916965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:19.975313 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1036/2091] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:19.986253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cec43-f11b-450c-9269-0ed5ea1e9dbf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.986253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cec43-f11b-450c-9269-0ed5ea1e9dbf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.986555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cec43-f11b-450c-9269-0ed5ea1e9dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.994488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:19.995719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:19.995719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:19.995948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:19.998092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:20.015771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:20.030937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.032634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.049896 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1056/2092] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:20.054069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-380cec43-f11b-450c-9269-0ed5ea1e9dbf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:20.055662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-380cec43-f11b-450c-9269-0ed5ea1e9dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:20.085632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.086469 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1037/2093] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:20.102094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e045cc-cbe8-48b3-a509-c2e1bd49897b service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.119785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.120420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.120738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.121012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.121673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:20.121972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.123598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.156032 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1057/2094] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:57:20.161772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cec43-f11b-450c-9269-0ed5ea1e9dbf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.186623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cec43-f11b-450c-9269-0ed5ea1e9dbf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.198872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.199520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.199801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.200074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.200750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.210885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.263628 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1038/2095] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:20 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:20.268283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.285946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24724e20-90b2-4e32-8c05-b31ec053a7fb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.438397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.439983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.439983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.439983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.440234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.442502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.497196 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1058/2096] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:20 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:20.499779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.530255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b49b4-0bd1-4b16-90a0-09b2d1c9a829 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.561475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.568000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.568291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.568576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.569313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.571327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.629617 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1039/2097] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:20.633614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.645866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e03fa-c8a0-42b7-b4f1-7c962e1fd4e2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.652467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.653506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.654061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.654536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.655402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.657781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.690682 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1059/2098] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:20.695245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.707272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde9c98-60dd-45db-bfa1-a308e6706908 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.728149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c18d9b-442b-4863-a82c-c6ea8b2bd262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.729173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c18d9b-442b-4863-a82c-c6ea8b2bd262 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.729468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c18d9b-442b-4863-a82c-c6ea8b2bd262 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.732364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c18d9b-442b-4863-a82c-c6ea8b2bd262 admin admin] PATH_INFO: `/v3/projects/7b84f72741ed4c84953144b829216730/users/f6dc529352674021ab4cc1e44e53c1dd/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.766147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c18d9b-442b-4863-a82c-c6ea8b2bd262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b84f72741ed4c84953144b829216730, user_id=f6dc529352674021ab4cc1e44e53c1dd, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.769493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c18d9b-442b-4863-a82c-c6ea8b2bd262 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.774788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.778318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:20.809773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39e761-9141-472f-918b-04624c2deae7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.815121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1040/2099] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:20 2022] PUT /identity/v3/projects/7b84f72741ed4c84953144b829216730/users/f6dc529352674021ab4cc1e44e53c1dd/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:20.816698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebe83-83f0-4c20-8442-a3edccda3802 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.816997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebe83-83f0-4c20-8442-a3edccda3802 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.817931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebe83-83f0-4c20-8442-a3edccda3802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.858185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ca9ebe83-83f0-4c20-8442-a3edccda3802 None None] MFA Rules not processed for user `f6dc529352674021ab4cc1e44e53c1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:20.859438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ca9ebe83-83f0-4c20-8442-a3edccda3802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:20.938629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.939401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.939678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.939978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] PATH_INFO: `/v3/projects/991c193b9ac144c18fc25251ae253764` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:20.942082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=991c193b9ac144c18fc25251ae253764)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:20.943844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:20.959864 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1041/2100] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:20.965179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ebe83-83f0-4c20-8442-a3edccda3802 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:20.980792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ebe83-83f0-4c20-8442-a3edccda3802 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:20.991465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:20.997254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:20.998681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:20.998978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:20.999678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.000065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.002177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.013064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991c193b9ac144c18fc25251ae253764', 'request_id': 'req-e6dfc366-8e0a-44e4-8643-c4f9604dc253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.025517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991c193b9ac144c18fc25251ae253764', 'request_id': 'req-e6dfc366-8e0a-44e4-8643-c4f9604dc253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.036070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991c193b9ac144c18fc25251ae253764', 'request_id': 'req-e6dfc366-8e0a-44e4-8643-c4f9604dc253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.047612 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1042/2101] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:20 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:21.049819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] The token cache is being invalidate because project 991c193b9ac144c18fc25251ae253764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:21.050085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.050572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6dfc366-8e0a-44e4-8643-c4f9604dc253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.051875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.053650 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1060/2102] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:20 2022] DELETE /identity/v3/projects/991c193b9ac144c18fc25251ae253764 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:21.060081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.062411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.062933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.063965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.092616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:21.094864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.201538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316e867-a1a6-4df6-a85c-bf84c9c5e99b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.214404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab72fd3-e6e1-4875-95b1-8522cd9aa1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.215594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab72fd3-e6e1-4875-95b1-8522cd9aa1cc admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.216101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab72fd3-e6e1-4875-95b1-8522cd9aa1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.216606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab72fd3-e6e1-4875-95b1-8522cd9aa1cc admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.217771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab72fd3-e6e1-4875-95b1-8522cd9aa1cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.221831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab72fd3-e6e1-4875-95b1-8522cd9aa1cc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.260696 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1061/2103] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:21.261720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.263148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.263588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:21.279118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c774797-e27e-460d-bb95-da412e2f2b30 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.279797 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1043/2104] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:21 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:21.282333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d6a82-11c9-4327-bdfe-1ec5fa08249a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.283249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d6a82-11c9-4327-bdfe-1ec5fa08249a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.283927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d6a82-11c9-4327-bdfe-1ec5fa08249a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.302982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.304239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.304698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.305871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] PATH_INFO: `/v3/projects/0497f0d3056745aeaf7678f81638727f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.307683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0497f0d3056745aeaf7678f81638727f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.311682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-498d6a82-11c9-4327-bdfe-1ec5fa08249a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:21.312482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-498d6a82-11c9-4327-bdfe-1ec5fa08249a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.314572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.324200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1044/2105] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:21.328857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d6a82-11c9-4327-bdfe-1ec5fa08249a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.341734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d6a82-11c9-4327-bdfe-1ec5fa08249a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.349064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.350107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.350624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.351143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.352323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.355334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.372997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1045/2106] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:21.372997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.386021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ddb70-b149-4011-bd6b-3a82e44c1fdd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.400370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0497f0d3056745aeaf7678f81638727f', 'request_id': 'req-aad0a111-e130-4de5-b0bf-1fa02150ffc3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.411468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0497f0d3056745aeaf7678f81638727f', 'request_id': 'req-aad0a111-e130-4de5-b0bf-1fa02150ffc3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.421493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0497f0d3056745aeaf7678f81638727f', 'request_id': 'req-aad0a111-e130-4de5-b0bf-1fa02150ffc3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.458763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] The token cache is being invalidate because project 0497f0d3056745aeaf7678f81638727f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:21.459061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.461561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.465291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.465748 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1062/2107] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:21 2022] DELETE /identity/v3/projects/0497f0d3056745aeaf7678f81638727f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:21.466249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.466610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.466795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.467591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:21.467901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.468515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.470402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.471262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.489007 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1046/2108] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:21.491003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.494483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.511951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eed1c7-e255-46e8-ad56-dd184d2e6776 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.515535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0a111-e130-4de5-b0bf-1fa02150ffc3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.517984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.519271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.519581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.519851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] PATH_INFO: `/v3/users/87ed26b2c06d42c6bfcb4fa2786a6b38` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.522800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ed26b2c06d42c6bfcb4fa2786a6b38)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.524537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.526009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.527453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.527724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.528032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] PATH_INFO: `/v3/projects/13e39f4646184f20ad1a670504b3b7f7/users/dbecb205a61b421aa1314c2014a729af/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.561565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e39f4646184f20ad1a670504b3b7f7, user_id=dbecb205a61b421aa1314c2014a729af, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.567492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.612512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1063/2109] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:21 2022] PUT /identity/v3/projects/13e39f4646184f20ad1a670504b3b7f7/users/dbecb205a61b421aa1314c2014a729af/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:21.614941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.621585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.626590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ed26b2c06d42c6bfcb4fa2786a6b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.628331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ed26b2c06d42c6bfcb4fa2786a6b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.637266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ed26b2c06d42c6bfcb4fa2786a6b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.649176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ed26b2c06d42c6bfcb4fa2786a6b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.655671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:21.673027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ed26b2c06d42c6bfcb4fa2786a6b38'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:21.693022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1047/2110] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:21 2022] DELETE /identity/v3/users/87ed26b2c06d42c6bfcb4fa2786a6b38 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:21.694359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.696510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:21.735083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd00fe-feae-4137-b8d6-8b23a3da634d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.819577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:21.860309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2fa2b-1a13-459f-9822-25f9d5d65f52 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.881132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.882206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.882682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.883214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.884411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.887652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.906471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:21.907282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:21.907681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:21.908055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:21.909305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:21.912536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:21.927079 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1048/2111] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:21.933260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.965594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ec986-f3fc-404f-8e78-e0053184e173 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:21.967426 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1064/2112] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:21 2022] POST /identity/v3/projects => generated 373 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:21.974477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:21.979646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.093884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ed097-5d3b-4b03-baa3-e3d0fa715d2c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:22.101610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.103169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.103452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.103835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.104495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:22.106593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:22.111587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785b19c-7c87-4ebf-86d6-1eed7142728a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.112394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785b19c-7c87-4ebf-86d6-1eed7142728a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.112769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785b19c-7c87-4ebf-86d6-1eed7142728a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.113149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785b19c-7c87-4ebf-86d6-1eed7142728a admin admin] PATH_INFO: `/v3/projects/13e39f4646184f20ad1a670504b3b7f7/users/dbecb205a61b421aa1314c2014a729af/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.116125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785b19c-7c87-4ebf-86d6-1eed7142728a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e39f4646184f20ad1a670504b3b7f7, user_id=dbecb205a61b421aa1314c2014a729af, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:22.120104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785b19c-7c87-4ebf-86d6-1eed7142728a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:22.136066 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1065/2114] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:21 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:22.136736 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1049/2114] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:21 2022] PUT /identity/v3/projects/13e39f4646184f20ad1a670504b3b7f7/users/dbecb205a61b421aa1314c2014a729af/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:22.148428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:22.148724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.149101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.149271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.162583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03505b77-af1b-4569-8bec-bc8ed87580bf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:22.235538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] MFA Rules not processed for user `dbecb205a61b421aa1314c2014a729af`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:22.244524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.253297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:22.282568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.285606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.285896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.286172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.286857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:22.293142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:22.307809 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1066/2115] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:22.313707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:22.349274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a664d3-b5f6-4925-955f-41246f49ad34 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:22.358731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.359519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.359804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.360611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.360931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:22.363105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:22.388640 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1067/2116] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:22.393307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:22.406692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c927474-fc09-4709-8617-46015f3820af admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:22.409668 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1050/2117] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:22.419998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:22.420699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83ebd-54e9-4ac4-a336-84062b3784d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.421420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83ebd-54e9-4ac4-a336-84062b3784d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.422005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83ebd-54e9-4ac4-a336-84062b3784d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.422114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.422831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83ebd-54e9-4ac4-a336-84062b3784d9 admin admin] PATH_INFO: `/v3/projects/e79eae6310354ca58046e59dfac23009/users/9544cdb0392f416197a586d99e8a3210/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.440630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339a93-bc0a-45a9-8b29-694d55bdf9e6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:22.447442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.447860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.448188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.448465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] PATH_INFO: `/v3/projects/edf104c9f1ab417bbd1bba5a7627f443` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.450268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf104c9f1ab417bbd1bba5a7627f443)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:22.452301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:22.467964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed83ebd-54e9-4ac4-a336-84062b3784d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79eae6310354ca58046e59dfac23009, user_id=9544cdb0392f416197a586d99e8a3210, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:22.471420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed83ebd-54e9-4ac4-a336-84062b3784d9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:22.497214 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1068/2118] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:22 2022] PUT /identity/v3/projects/e79eae6310354ca58046e59dfac23009/users/9544cdb0392f416197a586d99e8a3210/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:22.510516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc914c-983d-45de-81d0-80491918d643 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.510977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc914c-983d-45de-81d0-80491918d643 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.511252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc914c-983d-45de-81d0-80491918d643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.559416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf104c9f1ab417bbd1bba5a7627f443', 'request_id': 'req-a8fd79ef-dc01-44a1-a7d2-26398386f57b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:22.565409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-26dc914c-983d-45de-81d0-80491918d643 None None] MFA Rules not processed for user `9544cdb0392f416197a586d99e8a3210`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:22.566951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-26dc914c-983d-45de-81d0-80491918d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.576539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf104c9f1ab417bbd1bba5a7627f443', 'request_id': 'req-a8fd79ef-dc01-44a1-a7d2-26398386f57b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:22.580454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:22.590082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf104c9f1ab417bbd1bba5a7627f443', 'request_id': 'req-a8fd79ef-dc01-44a1-a7d2-26398386f57b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:22.606387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] The token cache is being invalidate because project edf104c9f1ab417bbd1bba5a7627f443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:22.606722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:22.607442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8fd79ef-dc01-44a1-a7d2-26398386f57b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:22.612792 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1051/2119] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:22 2022] DELETE /identity/v3/projects/edf104c9f1ab417bbd1bba5a7627f443 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:22.614192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.614611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.615309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.640276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:22.641694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.708550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1069/2120] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:22.709984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc914c-983d-45de-81d0-80491918d643 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:22.712249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-26dc914c-983d-45de-81d0-80491918d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.753843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1052/2121] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:22.761661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:22.769332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.808113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd9d0-140d-47a8-b5d6-fd040a8f307d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:22.821782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca6cba-4a6c-440b-9451-ad0f8189cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.823936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6cba-4a6c-440b-9451-ad0f8189cd44 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.824962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6cba-4a6c-440b-9451-ad0f8189cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.825632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6cba-4a6c-440b-9451-ad0f8189cd44 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.827100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca6cba-4a6c-440b-9451-ad0f8189cd44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:22.833338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca6cba-4a6c-440b-9451-ad0f8189cd44 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:22.879136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc914c-983d-45de-81d0-80491918d643 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:22.889562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:22.890280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.890766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.891214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.894177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:22.916125 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1053/2122] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:22 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:22.922520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f25de-c6c1-471d-b4e1-2ba020a22552 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:22.922870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f25de-c6c1-471d-b4e1-2ba020a22552 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:22.923219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f25de-c6c1-471d-b4e1-2ba020a22552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:22.954811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c62f25de-c6c1-471d-b4e1-2ba020a22552 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:22.955610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:22.957174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c62f25de-c6c1-471d-b4e1-2ba020a22552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.005392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.007148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.019339 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1070/2123] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:23.019856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.021352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.043456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f9944-6951-409f-a67c-d9515e58a336 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.069828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1054/2124] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:23.070727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f25de-c6c1-471d-b4e1-2ba020a22552 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.084252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f25de-c6c1-471d-b4e1-2ba020a22552 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.090800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.091740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.092076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.092325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.093017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.095184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.128378 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1055/2125] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:23 2022] POST /identity/v3/users => generated 293 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:23.128378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.128378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.145769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12c8b5-1e45-43be-bb2f-ceb42b300eb4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.145997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.146748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.147016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.147797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.149976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.151529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.152188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.152465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.152768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] PATH_INFO: `/v3/users/f6dc529352674021ab4cc1e44e53c1dd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.154253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6dc529352674021ab4cc1e44e53c1dd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.156218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.259641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6dc529352674021ab4cc1e44e53c1dd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:23.260924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:23.263006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6dc529352674021ab4cc1e44e53c1dd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:23.264894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.266616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.277660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6dc529352674021ab4cc1e44e53c1dd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:23.281686 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1071/2126] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:23.282154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.290539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6dc529352674021ab4cc1e44e53c1dd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:23.295840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3150922-15b4-4bbc-bde6-62be03f823ea service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.304005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26371-a940-4819-9aef-5c8ae46eb99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.305197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26371-a940-4819-9aef-5c8ae46eb99b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.305342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26371-a940-4819-9aef-5c8ae46eb99b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.305572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26371-a940-4819-9aef-5c8ae46eb99b admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.307484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26371-a940-4819-9aef-5c8ae46eb99b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:23.307983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26371-a940-4819-9aef-5c8ae46eb99b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.308097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6dc529352674021ab4cc1e44e53c1dd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:23.309950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26371-a940-4819-9aef-5c8ae46eb99b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.319976 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1056/2127] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:23 2022] DELETE /identity/v3/users/f6dc529352674021ab4cc1e44e53c1dd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:23.322024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.324708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.335077 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1072/2128] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:23.337078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.337535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.338890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.369017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:23.371332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.430699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1f721-dbd2-4089-8bf1-1cb49e571bbe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.435046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:23.451031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.451674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.451951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.452222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.452920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.454968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.461508 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1073/2129] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:23.462936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.464512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.477662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637b72-c263-4f24-9c05-c2d86d932bc5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.482187 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1057/2130] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:23.484946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.485580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.485852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.486118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.486220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.486758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.488708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.498975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70442f-47d6-408f-b9ba-8e98ee72c8c7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.508519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.509906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.510353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.510904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] PATH_INFO: `/v3/projects/a3199e83259042efada3f2a565c624ff/users/f74e77337957458794bd6c942c97b344/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.532558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1074/2131] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:23 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:23.535002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.539204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:23.571415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3199e83259042efada3f2a565c624ff, user_id=f74e77337957458794bd6c942c97b344, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.574415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.597851 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1058/2132] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:23 2022] PUT /identity/v3/projects/a3199e83259042efada3f2a565c624ff/users/f74e77337957458794bd6c942c97b344/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:23.620747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.698982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3d60e-c67e-48a3-a184-78a2a2bddf7b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.733233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88e68f-ca42-49ae-a755-d347a444c4e0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.741179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.741891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.742181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.742443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.743164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.746106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.778818 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1059/2133] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:23 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:23.781209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.784529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.785347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.785638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.785915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.797144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a049e4-8db0-4cf9-b820-24843cd7815f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.804333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.805088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.805523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.805789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.806463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.808087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.819883 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1060/2134] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:23.821010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.831251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.836302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.838930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597749ce-4eb4-42b2-963d-65b3c5a2d2c9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.849884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.850528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.850898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.851159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.852116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.856463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.863513 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1075/2135] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:23.866686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.866879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:23.893815 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1061/2136] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:23.894725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.897221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a7d4-d6ff-4bfa-abb1-8392f2861931 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.904170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2ec1e-5453-414e-9c85-a9cd17b6656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.905993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ec1e-5453-414e-9c85-a9cd17b6656e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.906096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ec1e-5453-414e-9c85-a9cd17b6656e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.906455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ec1e-5453-414e-9c85-a9cd17b6656e admin admin] PATH_INFO: `/v3/projects/a3199e83259042efada3f2a565c624ff/users/f74e77337957458794bd6c942c97b344/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.909905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2ec1e-5453-414e-9c85-a9cd17b6656e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3199e83259042efada3f2a565c624ff, user_id=f74e77337957458794bd6c942c97b344, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.913850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2ec1e-5453-414e-9c85-a9cd17b6656e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.916296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd845a71-78ff-4294-adbb-cc2cdbd3929b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:23.926366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:23.927096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.927737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.928012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.928831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:23.930700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:23.951357 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1076/2137] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:23 2022] PUT /identity/v3/projects/a3199e83259042efada3f2a565c624ff/users/f74e77337957458794bd6c942c97b344/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:23.952375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbf0e9-d4d0-45a8-a4c6-9783eaf4a1c0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:23.952681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbf0e9-d4d0-45a8-a4c6-9783eaf4a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:23.952967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbf0e9-d4d0-45a8-a4c6-9783eaf4a1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:23.955700 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1062/2138] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:23.960161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:23.980952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfaea9-aa43-42cd-b7d9-a515731dfd05 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:24.009911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aebbf0e9-d4d0-45a8-a4c6-9783eaf4a1c0 None None] MFA Rules not processed for user `f74e77337957458794bd6c942c97b344`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:24.012374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aebbf0e9-d4d0-45a8-a4c6-9783eaf4a1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.083458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951ecf8-3155-4040-984b-9d76e81a537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:24.084279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951ecf8-3155-4040-984b-9d76e81a537a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.084598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951ecf8-3155-4040-984b-9d76e81a537a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.085646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951ecf8-3155-4040-984b-9d76e81a537a admin admin] PATH_INFO: `/v3/projects/b91b697e0bb543b8b4b158915f258e5a/users/c9f48488312744f88aca7715c3b40adf/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.119106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951ecf8-3155-4040-984b-9d76e81a537a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b91b697e0bb543b8b4b158915f258e5a, user_id=c9f48488312744f88aca7715c3b40adf, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:24.124706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951ecf8-3155-4040-984b-9d76e81a537a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:24.138279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:24.156703 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1077/2139] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:24.156703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.156703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.156703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.157128 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1063/2140] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:23 2022] PUT /identity/v3/projects/b91b697e0bb543b8b4b158915f258e5a/users/c9f48488312744f88aca7715c3b40adf/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:24.168125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6f2ef-147f-4611-afdc-d81eb1a35343 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.168536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6f2ef-147f-4611-afdc-d81eb1a35343 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.169063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6f2ef-147f-4611-afdc-d81eb1a35343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.208855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] MFA Rules not processed for user `00423eb1e4c24763ae0a546225ac1dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:24.210616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.216392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c5f6f2ef-147f-4611-afdc-d81eb1a35343 None None] MFA Rules not processed for user `9544cdb0392f416197a586d99e8a3210`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:24.218685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c5f6f2ef-147f-4611-afdc-d81eb1a35343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.316052 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1064/2141] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:24.317673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.318109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.318538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.350043 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1078/2142] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:24.350999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:24.353072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.387197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] MFA Rules not processed for user `c9f48488312744f88aca7715c3b40adf`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:24.391221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.491926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9bc4f-e258-4484-b634-a740a1f6dcdc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:24.498689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:24.499327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.499616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.499894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.502889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.507131 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1065/2143] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:24.507430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:24.511249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.519362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:24.520100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:24.522048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:24.536863 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1079/2144] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:24.538088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:24.631833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b59740-3e37-49f6-8035-9c01e7a52e9b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:24.641539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:24.642644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.642644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.642845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] PATH_INFO: `/v3/projects/7b84f72741ed4c84953144b829216730` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.644195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b84f72741ed4c84953144b829216730)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:24.646373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:24.683465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd21ac6-542a-49db-bbe3-8c4f3c064230 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:24.689977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:24.690760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.691993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.692437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.693288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:24.695306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:24.713322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b84f72741ed4c84953144b829216730', 'request_id': 'req-46f5549b-7e86-461c-a8fa-a92f639749dc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:24.723056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b84f72741ed4c84953144b829216730', 'request_id': 'req-46f5549b-7e86-461c-a8fa-a92f639749dc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:24.734593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1080/2145] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:24 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:24.736372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b84f72741ed4c84953144b829216730', 'request_id': 'req-46f5549b-7e86-461c-a8fa-a92f639749dc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:24.738987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:24.760481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] The token cache is being invalidate because project 7b84f72741ed4c84953144b829216730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:24.761876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:24.764320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:24.768524 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1066/2146] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:24 2022] DELETE /identity/v3/projects/7b84f72741ed4c84953144b829216730 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:24.770546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:24.772593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:24.777150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.874114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56220242-5daf-4b6c-b6a8-50d99da5c205 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:24.883606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:24.884401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.885764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.886290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.887538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:24.890456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:24.935133 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1081/2147] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:24 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:24.939174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:24.941391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:24.950875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5549b-7e86-461c-a8fa-a92f639749dc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:24.955730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:24.956530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.956806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.957259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] PATH_INFO: `/v3/users/dbecb205a61b421aa1314c2014a729af` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.958626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbecb205a61b421aa1314c2014a729af)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:24.959679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43f2b4-024e-4e1c-900c-03153f2e555a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:24.960959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:24.969091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:24.970307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:24.971066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:24.971533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:24.972783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:24.975837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:25.021088 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1082/2148] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:25.025067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:25.053968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:25.061427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76478c87-9cfb-41db-a9cc-dc3d476e5afa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:25.077544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:25.078404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:25.078776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:25.079397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] PATH_INFO: `/v3/projects/07d3842bdd5a4434b7143b97082b8a19/users/d498ecdaf6d84f1ab34c5403d24a3340/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:25.101211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d3842bdd5a4434b7143b97082b8a19, user_id=d498ecdaf6d84f1ab34c5403d24a3340, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:25.103630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbecb205a61b421aa1314c2014a729af'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:25.104773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:25.107819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbecb205a61b421aa1314c2014a729af'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:25.127631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbecb205a61b421aa1314c2014a729af'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:25.141311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbecb205a61b421aa1314c2014a729af'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:25.150528 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1083/2149] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:25 2022] PUT /identity/v3/projects/07d3842bdd5a4434b7143b97082b8a19/users/d498ecdaf6d84f1ab34c5403d24a3340/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:25.155067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:25.158336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:25.171172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbecb205a61b421aa1314c2014a729af'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:25.189134 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1067/2150] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:24 2022] DELETE /identity/v3/users/dbecb205a61b421aa1314c2014a729af => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:25.311641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677f16-7bc9-4d08-acc6-be1dbea9ba84 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:25.437015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc5307-1546-4097-b5fc-d7ab6e06f4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:25.437515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc5307-1546-4097-b5fc-d7ab6e06f4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:25.437991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc5307-1546-4097-b5fc-d7ab6e06f4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:25.438462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc5307-1546-4097-b5fc-d7ab6e06f4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:25.439949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc5307-1546-4097-b5fc-d7ab6e06f4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:25.441938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc5307-1546-4097-b5fc-d7ab6e06f4d6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:25.461090 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1084/2151] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:25.462893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:25.479208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b7944-656f-4df5-b53a-18aa97441cb5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:25.499929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:25.500799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:25.501088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:25.501526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] PATH_INFO: `/v3/projects/07d3842bdd5a4434b7143b97082b8a19/users/d498ecdaf6d84f1ab34c5403d24a3340/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:25.503797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d3842bdd5a4434b7143b97082b8a19, user_id=d498ecdaf6d84f1ab34c5403d24a3340, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:25.511214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:25.542265 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1068/2152] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:25 2022] PUT /identity/v3/projects/07d3842bdd5a4434b7143b97082b8a19/users/d498ecdaf6d84f1ab34c5403d24a3340/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:25.564045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:25.564209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:25.565465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:25.650003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:25.691786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] MFA Rules not processed for user `d498ecdaf6d84f1ab34c5403d24a3340`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:25.697366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:25.972797 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1085/2153] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:26.372249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:26.381024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:26.618682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c6b0e-d87f-4599-b1b0-3b6b96b75a03 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:26.630569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:26.631972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:26.633110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:26.633248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:26.633248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:26.633482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] PATH_INFO: `/v3/projects/13e39f4646184f20ad1a670504b3b7f7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:26.636406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e39f4646184f20ad1a670504b3b7f7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:26.640344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:26.802084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e39f4646184f20ad1a670504b3b7f7', 'request_id': 'req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:26.819364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e39f4646184f20ad1a670504b3b7f7', 'request_id': 'req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:26.832205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd21de3-3ce9-46d5-8648-c613c93dc3a5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:26.839035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e39f4646184f20ad1a670504b3b7f7', 'request_id': 'req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:26.864984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] The token cache is being invalidate because project 13e39f4646184f20ad1a670504b3b7f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:26.865336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:26.866900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d32a6ca5-8af3-47c8-8dc0-9bc867ba7173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:26.873512 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1069/2154] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:26 2022] DELETE /identity/v3/projects/13e39f4646184f20ad1a670504b3b7f7 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:26.876474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6feae-8446-4dcd-a564-7cbe727aef56 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:26.876853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6feae-8446-4dcd-a564-7cbe727aef56 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:26.877143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6feae-8446-4dcd-a564-7cbe727aef56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:26.906772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-54d6feae-8446-4dcd-a564-7cbe727aef56 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:26.908276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-54d6feae-8446-4dcd-a564-7cbe727aef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:26.957126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:26.999433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.000787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.001286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.001914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.009366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:27.071271 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1070/2155] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:27.073650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9e3a2-7e9a-4c34-80a0-9b71d850ef62 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.074071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9e3a2-7e9a-4c34-80a0-9b71d850ef62 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.074440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9e3a2-7e9a-4c34-80a0-9b71d850ef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.123178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a5c9e3a2-7e9a-4c34-80a0-9b71d850ef62 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:27.125003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a5c9e3a2-7e9a-4c34-80a0-9b71d850ef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:27.131403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:27.140404 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1071/2156] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:27.141826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9e3a2-7e9a-4c34-80a0-9b71d850ef62 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:27.157073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9e3a2-7e9a-4c34-80a0-9b71d850ef62 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:27.162855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.163528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.163863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.164152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.165304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:27.165304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.166990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.189519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1072/2157] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:27.191351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:27.217453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0997999-8b85-4f6a-8170-eec9a8d6b136 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:27.232127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.233006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.233386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.233760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.234675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.237206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.280548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.285105 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1073/2158] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:27 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:27.285276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.288584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:27.309635 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1086/2159] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:27.363803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:27.366030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:27.394801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d064e7d-48c0-4b11-8609-589438ef771c service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:27.456244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d8414-7125-4f01-8c6f-8b5abc7816fd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:27.465184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.466525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.467312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.467769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.468846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.472177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.512477 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1074/2160] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:27 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:27.517920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:27.534527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443dcda-d437-4e15-a886-226aea1cf172 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:27.542514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.543746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.544188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.544591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.545492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.547704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.584484 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1075/2161] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:27.586403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:27.588519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb6909-b91d-4f38-9baa-cbe1aebcc885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.589605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb6909-b91d-4f38-9baa-cbe1aebcc885 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.590151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb6909-b91d-4f38-9baa-cbe1aebcc885 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.590878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb6909-b91d-4f38-9baa-cbe1aebcc885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.594365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4bdb6909-b91d-4f38-9baa-cbe1aebcc885 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:27.611064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4896d4-4943-40bc-8d9d-40bc02ba5129 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:27.623273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.624311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.624638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.625060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.627453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.645313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.686348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1076/2162] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:27.693130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:27.709640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078f9d9-f2c2-4d32-80bb-ffd2180e8381 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:27.718611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a5b46-5b3b-4271-a03c-681830f1971f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:27.719485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a5b46-5b3b-4271-a03c-681830f1971f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.719835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a5b46-5b3b-4271-a03c-681830f1971f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.720249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a5b46-5b3b-4271-a03c-681830f1971f admin admin] PATH_INFO: `/v3/projects/80c516d3f45940cb87608783a155a19a/users/956ffa7f397d4271ad1104f4e6fb6ecc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.728506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb6909-b91d-4f38-9baa-cbe1aebcc885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.730583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb6909-b91d-4f38-9baa-cbe1aebcc885 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.742558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1087/2163] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:27 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:27.768451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a5b46-5b3b-4271-a03c-681830f1971f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c516d3f45940cb87608783a155a19a, user_id=956ffa7f397d4271ad1104f4e6fb6ecc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:27.770125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a5b46-5b3b-4271-a03c-681830f1971f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:27.775975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fedb84-69fc-41cb-8982-6d2ccbbb3650 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.776355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fedb84-69fc-41cb-8982-6d2ccbbb3650 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.776775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fedb84-69fc-41cb-8982-6d2ccbbb3650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:27.861599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1077/2164] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:27 2022] PUT /identity/v3/projects/80c516d3f45940cb87608783a155a19a/users/956ffa7f397d4271ad1104f4e6fb6ecc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:27.881065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-67fedb84-69fc-41cb-8982-6d2ccbbb3650 None None] MFA Rules not processed for user `d7d3aef1d4da47b2ab5454c3eb0346a6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:27.883573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-67fedb84-69fc-41cb-8982-6d2ccbbb3650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:27.889429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:27.896327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:27.896908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:28.080289 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1088/2165] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:28.091152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] MFA Rules not processed for user `956ffa7f397d4271ad1104f4e6fb6ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:28.094267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:28.126269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:28.205263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1078/2166] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:28.211316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fedb84-69fc-41cb-8982-6d2ccbbb3650 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:28.229061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:28.254199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:28.258245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:28.395029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fedb84-69fc-41cb-8982-6d2ccbbb3650 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:28.406954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:28.408067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:28.410476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:28.410849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:28.411834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:28.416488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:28.470564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0c642-bbe6-47ae-b2f2-232cbb330120 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:28.492426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:28.494072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:28.494351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:28.494740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:28.501231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:28.506356 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1089/2167] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:28 2022] POST /identity/v3/projects => generated 381 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:28.508456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:28.655543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:28.659304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:28.689250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026a6ee-9618-4478-8c8f-52692fda29e1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:28.696591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:28.697643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:28.697943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:28.698434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:28.699088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:28.702703 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1079/2168] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:28.708508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:28.780208 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1090/2169] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:28 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:28.780541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:28.799523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad94a8c-adcd-4235-9095-34067ae9f958 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:28.822285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:28.822979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:28.823254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:28.823522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:28.826438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:28.834274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:28.864516 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1080/2170] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:28.867349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:28.885517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac923c-79a8-4265-8027-6e6c754851cc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:28.903541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:28.904275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:28.904558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:28.904862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] PATH_INFO: `/v3/projects/0fce2d1bb1e54e8b9160f853cafcb946/users/9749659a22464bfe98f36aaadd66eb34/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:28.955462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fce2d1bb1e54e8b9160f853cafcb946, user_id=9749659a22464bfe98f36aaadd66eb34, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:28.958271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:28.987948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1091/2171] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:28 2022] PUT /identity/v3/projects/0fce2d1bb1e54e8b9160f853cafcb946/users/9749659a22464bfe98f36aaadd66eb34/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:28.992186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:29.007586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fc1f-9ecd-4c60-8dae-b8f4b161d9a1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:29.150288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:29.168250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116c9cd-e815-4746-9515-abc153579360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:29.168952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116c9cd-e815-4746-9515-abc153579360 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:29.169231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116c9cd-e815-4746-9515-abc153579360 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:29.169542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116c9cd-e815-4746-9515-abc153579360 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:29.170212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116c9cd-e815-4746-9515-abc153579360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:29.172079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116c9cd-e815-4746-9515-abc153579360 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:29.217391 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1081/2172] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:29.221869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:29.234979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b442-2a61-4ce2-bf25-66ec3231fcd7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:29.239659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:29.240342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:29.240621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:29.240945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] PATH_INFO: `/v3/projects/0fce2d1bb1e54e8b9160f853cafcb946/users/9749659a22464bfe98f36aaadd66eb34/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:29.243332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fce2d1bb1e54e8b9160f853cafcb946, user_id=9749659a22464bfe98f36aaadd66eb34, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:29.246197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:29.259983 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1092/2173] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:29 2022] PUT /identity/v3/projects/0fce2d1bb1e54e8b9160f853cafcb946/users/9749659a22464bfe98f36aaadd66eb34/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:29.272906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5f32-6da4-4c59-a9e5-820987fdb4c8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:29.273484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5f32-6da4-4c59-a9e5-820987fdb4c8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:29.273811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5f32-6da4-4c59-a9e5-820987fdb4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:29.312998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-944c5f32-6da4-4c59-a9e5-820987fdb4c8 None None] MFA Rules not processed for user `9749659a22464bfe98f36aaadd66eb34`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:29.314365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-944c5f32-6da4-4c59-a9e5-820987fdb4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:29.431869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1082/2174] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:29.957767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:30.051320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:30.094773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674719e-1529-4e10-8569-9c2d68b4a5e7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:30.102045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:30.104202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:30.104681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:30.105131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:30.116231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:30.119420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:30.134641 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1093/2175] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:31.707797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c5f32-6da4-4c59-a9e5-820987fdb4c8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:31.843542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c5f32-6da4-4c59-a9e5-820987fdb4c8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:31.850888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:31.857102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:31.858204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:31.858635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] PATH_INFO: `/v3/users/956ffa7f397d4271ad1104f4e6fb6ecc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:31.861214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=956ffa7f397d4271ad1104f4e6fb6ecc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:31.865706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:31.932742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:31.937029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:31.947447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956ffa7f397d4271ad1104f4e6fb6ecc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:31.948142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956ffa7f397d4271ad1104f4e6fb6ecc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:31.958967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8af99-1733-4e43-96df-a28f4ca0019b service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:31.968019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956ffa7f397d4271ad1104f4e6fb6ecc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:31.969944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:31.971142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:31.972125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:31.972680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] PATH_INFO: `/v3/users/c9f48488312744f88aca7715c3b40adf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:31.975049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f48488312744f88aca7715c3b40adf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:31.977790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:31.980648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956ffa7f397d4271ad1104f4e6fb6ecc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:31.991981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:32.003107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-691e6cb5-b39b-4a76-ba79-cef1ec51d6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956ffa7f397d4271ad1104f4e6fb6ecc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:32.032337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1083/2176] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:31 2022] DELETE /identity/v3/users/956ffa7f397d4271ad1104f4e6fb6ecc => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:32.035717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:32.036121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:32.036569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:32.087584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:32.090730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:32.106864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f48488312744f88aca7715c3b40adf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:32.107783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f48488312744f88aca7715c3b40adf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:32.130169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f48488312744f88aca7715c3b40adf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:32.142756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f48488312744f88aca7715c3b40adf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:32.143532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2361b726-41e4-43c6-b2dc-178c69029bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f48488312744f88aca7715c3b40adf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:32.183237 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1094/2177] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:31 2022] DELETE /identity/v3/users/c9f48488312744f88aca7715c3b40adf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:32.189638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:32.189993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:32.190190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:32.221132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:32.222530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:32.304224 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1084/2178] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:32.319418 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1095/2179] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:32.342519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:32.342669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:32.344210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:32.344336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:32.453166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:32.475536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c0a7a-6a80-45cc-ab6a-ae73b6c2fe7f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:32.478119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb9d4-8a70-43cb-9938-18379a89e78f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:32.488253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:32.488927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:32.489208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:32.489602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:32.491846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:32.492899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:32.493455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:32.494151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:32.500630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:32.502140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:32.512804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:32.514188 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1096/2180] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:32.514899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:32.515167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:32.516982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:32.532725 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1085/2181] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:32.680541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e74d8-56c4-41fa-8372-7f260178a2ec service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:32.704763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:32.705603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:32.705871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:32.706137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:32.713649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:32.850514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:32.853574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:32.872337 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1097/2182] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:32 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:33.881116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:33.882913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:33.894625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:33.896089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:33.904349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782add20-0a35-42d7-8417-a24f80b71960 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:33.911678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:33.912580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:33.913070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:33.913864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] PATH_INFO: `/v3/projects/b91b697e0bb543b8b4b158915f258e5a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:33.918399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b91b697e0bb543b8b4b158915f258e5a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:33.919893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e683-2007-4ef2-81d6-58dae2892f05 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:33.921179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:33.959608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:33.964732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:33.964732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:33.964732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] PATH_INFO: `/v3/projects/80c516d3f45940cb87608783a155a19a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:33.968010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c516d3f45940cb87608783a155a19a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:33.973285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:34.060609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b91b697e0bb543b8b4b158915f258e5a', 'request_id': 'req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.070359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b91b697e0bb543b8b4b158915f258e5a', 'request_id': 'req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.088110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b91b697e0bb543b8b4b158915f258e5a', 'request_id': 'req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.101133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c516d3f45940cb87608783a155a19a', 'request_id': 'req-90445cd6-679d-4f9d-94c6-6900fea64872'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.121042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c516d3f45940cb87608783a155a19a', 'request_id': 'req-90445cd6-679d-4f9d-94c6-6900fea64872'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.124187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] The token cache is being invalidate because project b91b697e0bb543b8b4b158915f258e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:34.125021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.136160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.138079 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1086/2183] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:33 2022] DELETE /identity/v3/projects/b91b697e0bb543b8b4b158915f258e5a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:34.142267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:34.144101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:34.145798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c516d3f45940cb87608783a155a19a', 'request_id': 'req-90445cd6-679d-4f9d-94c6-6900fea64872'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.158264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:34.165923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] The token cache is being invalidate because project 80c516d3f45940cb87608783a155a19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:34.166614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.167405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.175386 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1098/2184] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:33 2022] DELETE /identity/v3/projects/80c516d3f45940cb87608783a155a19a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:34.177089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:34.181095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:34.181935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:34.274602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83cb49-b4e9-43e9-90a7-f8c3589db0fe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:34.323811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90445cd6-679d-4f9d-94c6-6900fea64872 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:34.331030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:34.331700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:34.332028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:34.332305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] PATH_INFO: `/v3/users/9749659a22464bfe98f36aaadd66eb34` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:34.334173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9749659a22464bfe98f36aaadd66eb34)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:34.336910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:34.391256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:34.391954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:34.392299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:34.393587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] PATH_INFO: `/v3/users/d498ecdaf6d84f1ab34c5403d24a3340` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:34.394917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d498ecdaf6d84f1ab34c5403d24a3340)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:34.397275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:34.420211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9749659a22464bfe98f36aaadd66eb34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.422411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9749659a22464bfe98f36aaadd66eb34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.438898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9749659a22464bfe98f36aaadd66eb34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.491912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9749659a22464bfe98f36aaadd66eb34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.497790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9749659a22464bfe98f36aaadd66eb34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.530425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd498ecdaf6d84f1ab34c5403d24a3340'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.533499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd498ecdaf6d84f1ab34c5403d24a3340'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.569046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd498ecdaf6d84f1ab34c5403d24a3340'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.579365 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1099/2185] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:34 2022] DELETE /identity/v3/users/9749659a22464bfe98f36aaadd66eb34 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:34.593216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd498ecdaf6d84f1ab34c5403d24a3340'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.633542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f8e71c-e02e-4344-b18c-12e07967647a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd498ecdaf6d84f1ab34c5403d24a3340'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:34.643433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1087/2186] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:34 2022] DELETE /identity/v3/users/d498ecdaf6d84f1ab34c5403d24a3340 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:36.536322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:36.539239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:36.692896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab26c3-5911-4013-bdf4-46a599719b59 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:36.723165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:36.782132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:36.783730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:36.784794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:36.785623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] PATH_INFO: `/v3/projects/0fce2d1bb1e54e8b9160f853cafcb946` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:36.788897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fce2d1bb1e54e8b9160f853cafcb946)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:36.793192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:36.976972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fce2d1bb1e54e8b9160f853cafcb946', 'request_id': 'req-3ba65fa4-f01f-4b89-8347-639a584c917c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.014482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fce2d1bb1e54e8b9160f853cafcb946', 'request_id': 'req-3ba65fa4-f01f-4b89-8347-639a584c917c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.049556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fce2d1bb1e54e8b9160f853cafcb946', 'request_id': 'req-3ba65fa4-f01f-4b89-8347-639a584c917c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.163340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] The token cache is being invalidate because project 0fce2d1bb1e54e8b9160f853cafcb946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:37.165286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.169383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.174381 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1100/2187] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:36 2022] DELETE /identity/v3/projects/0fce2d1bb1e54e8b9160f853cafcb946 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:37.182147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad892c-36bd-4fa8-b6a5-68579785fbf1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.182706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad892c-36bd-4fa8-b6a5-68579785fbf1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.182706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad892c-36bd-4fa8-b6a5-68579785fbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.231111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-09ad892c-36bd-4fa8-b6a5-68579785fbf1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:37.232462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-09ad892c-36bd-4fa8-b6a5-68579785fbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:37.275858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:37.280207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:37.381151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:37.476856 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1088/2188] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:37.488489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f14e1-e1bf-4397-acdb-528b04167e78 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.491012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f14e1-e1bf-4397-acdb-528b04167e78 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.492058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f14e1-e1bf-4397-acdb-528b04167e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.496116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba65fa4-f01f-4b89-8347-639a584c917c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:37.501939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:37.502605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.502885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.503152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] PATH_INFO: `/v3/projects/07d3842bdd5a4434b7143b97082b8a19` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.505130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d3842bdd5a4434b7143b97082b8a19)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:37.507449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:37.566314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-616f14e1-e1bf-4397-acdb-528b04167e78 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:37.568117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-616f14e1-e1bf-4397-acdb-528b04167e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:37.596875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d3842bdd5a4434b7143b97082b8a19', 'request_id': 'req-68021fa4-b27b-4986-9970-792d08acb845'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.616958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d3842bdd5a4434b7143b97082b8a19', 'request_id': 'req-68021fa4-b27b-4986-9970-792d08acb845'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.619070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:37.635185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d3842bdd5a4434b7143b97082b8a19', 'request_id': 'req-68021fa4-b27b-4986-9970-792d08acb845'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.666394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] The token cache is being invalidate because project 07d3842bdd5a4434b7143b97082b8a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:37.667363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.668939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-68021fa4-b27b-4986-9970-792d08acb845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:37.673723 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1101/2189] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:37 2022] DELETE /identity/v3/projects/07d3842bdd5a4434b7143b97082b8a19 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:37.679381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c518448-fddc-4b79-8f82-870187c36e00 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.681378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c518448-fddc-4b79-8f82-870187c36e00 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.682557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c518448-fddc-4b79-8f82-870187c36e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.710469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4c518448-fddc-4b79-8f82-870187c36e00 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:37.723089 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1089/2190] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:37.726626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f14e1-e1bf-4397-acdb-528b04167e78 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:37.730287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c518448-fddc-4b79-8f82-870187c36e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:37.739144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f14e1-e1bf-4397-acdb-528b04167e78 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:37.746081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:37.746931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.747274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.748525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.749356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:37.750011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:37.751891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:37.761715 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1102/2191] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:37.784077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb134473-95ba-41c7-bf1a-90b4a7f0ad9a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.784517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb134473-95ba-41c7-bf1a-90b4a7f0ad9a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.784914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb134473-95ba-41c7-bf1a-90b4a7f0ad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.810709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1090/2192] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:57:37.812215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:37.843668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bb134473-95ba-41c7-bf1a-90b4a7f0ad9a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:37.846125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabc827-6709-4219-a41f-0b615f316fd7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:37.847674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb134473-95ba-41c7-bf1a-90b4a7f0ad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:37.860262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:37.862416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.864694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.865917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.867872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:37.872600 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1103/2193] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:37.876441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:37.877574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb134473-95ba-41c7-bf1a-90b4a7f0ad9a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:37.888637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb134473-95ba-41c7-bf1a-90b4a7f0ad9a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:37.893775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:37.894489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:37.894814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:37.895038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:37.895722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:37.896195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:37.899720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:37.937172 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1104/2194] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:37.938958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:37.957324 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1091/2195] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:37 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:37.958929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:37.965507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6d806-8ee0-4c25-8308-ba56ec62ce27 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.063152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.064085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.064357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.064621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.065535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.067471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.080403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ccbe44-818d-4e38-be59-26af7374bdc6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.085921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.087551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.087886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.088202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.089041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.091498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.101258 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1105/2196] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:37 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:38.102853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.104894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:38.125533 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1092/2197] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:37 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:38.125533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.136513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a244d37-fb34-4247-bf0c-95f4f165fb4a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.184311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:38.185477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3e9c3-4206-4812-be0b-86386c6fe28c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.192530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.193748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.194052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.194320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] PATH_INFO: `/v3/users/b3dd80698e4c471abe7cd81809fa206c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.196078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3dd80698e4c471abe7cd81809fa206c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.198621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.222643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.223293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.223576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.223857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.224811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.227214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.264798 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1093/2198] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:38 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:38.265996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.274754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:38.277535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3dd80698e4c471abe7cd81809fa206c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:38.280765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3dd80698e4c471abe7cd81809fa206c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:38.283239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42797a03-d8d1-414d-b874-27c50bab689c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.290572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.291365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.291678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.291959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.292845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.295697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.299963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3dd80698e4c471abe7cd81809fa206c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:38.312051 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1094/2199] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:38.313410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.316738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:38.327404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3dd80698e4c471abe7cd81809fa206c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:38.330777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3dd80698e4c471abe7cd81809fa206c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:38.337665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d8137-aa79-469c-b58b-c81911622b9b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.362302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc48f6-4cda-4f38-b511-ad3b347a7442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.363228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc48f6-4cda-4f38-b511-ad3b347a7442 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.363753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc48f6-4cda-4f38-b511-ad3b347a7442 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.364241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc48f6-4cda-4f38-b511-ad3b347a7442 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.365469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc48f6-4cda-4f38-b511-ad3b347a7442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.369603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc48f6-4cda-4f38-b511-ad3b347a7442 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.382099 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1106/2200] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:38 2022] DELETE /identity/v3/users/b3dd80698e4c471abe7cd81809fa206c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:38.386342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.390473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:38.397672 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1095/2201] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:38.399713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec1a07-154b-48c4-a965-a42cc0881b88 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.399713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec1a07-154b-48c4-a965-a42cc0881b88 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.400026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec1a07-154b-48c4-a965-a42cc0881b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.423359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-19ec1a07-154b-48c4-a965-a42cc0881b88 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:38.424807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-19ec1a07-154b-48c4-a965-a42cc0881b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:38.501913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1096/2202] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:38.503152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec1a07-154b-48c4-a965-a42cc0881b88 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.514709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349708a-8b3c-423b-947a-0ecbff234fcd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.515191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec1a07-154b-48c4-a965-a42cc0881b88 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.519618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.520419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.520727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.521001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.521652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.522668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.523273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.524935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.525750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.526506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.527378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.529060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.539531 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1107/2203] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:38.539531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.540227 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1097/2204] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:38.544360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:38.544626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.556312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0944fc0-672c-4758-8a51-1eda833df119 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.563964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.564848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.565421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.565710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] PATH_INFO: `/v3/projects/0db301529f1b4836841b0c1a1ce9bb42/users/b5240b965bea4754842630a02fe410b6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.590900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db301529f1b4836841b0c1a1ce9bb42, user_id=b5240b965bea4754842630a02fe410b6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.595236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.619171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1098/2205] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:38 2022] PUT /identity/v3/projects/0db301529f1b4836841b0c1a1ce9bb42/users/b5240b965bea4754842630a02fe410b6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:38.620425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:38.630145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc43891-c105-4e3f-a176-431e42e2e719 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.704605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394f3f1-44b2-49f5-8f06-c372c8911d17 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:38.704857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99ef7e-7682-4f09-b401-3c1497682182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.705431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ef7e-7682-4f09-b401-3c1497682182 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.705730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ef7e-7682-4f09-b401-3c1497682182 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.706386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ef7e-7682-4f09-b401-3c1497682182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.710723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975599f-b4ce-4310-bda6-4c2b45495d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:38.711476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975599f-b4ce-4310-bda6-4c2b45495d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.711756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975599f-b4ce-4310-bda6-4c2b45495d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.712034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975599f-b4ce-4310-bda6-4c2b45495d45 admin admin] PATH_INFO: `/v3/projects/6e8a121c6c654613a6a92df03c3754fa/users/4319ec4b90e1481b8be19e8f9dde1ea4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.726079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99ef7e-7682-4f09-b401-3c1497682182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.741429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99ef7e-7682-4f09-b401-3c1497682182 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.755634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975599f-b4ce-4310-bda6-4c2b45495d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8a121c6c654613a6a92df03c3754fa, user_id=4319ec4b90e1481b8be19e8f9dde1ea4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:38.761148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975599f-b4ce-4310-bda6-4c2b45495d45 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:38.767490 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1108/2206] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:38.768847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97816c16-de6f-4498-992d-c4eeec69d6d3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.769132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97816c16-de6f-4498-992d-c4eeec69d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.769998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97816c16-de6f-4498-992d-c4eeec69d6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.770107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:38.793990 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1099/2207] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:38 2022] PUT /identity/v3/projects/6e8a121c6c654613a6a92df03c3754fa/users/4319ec4b90e1481b8be19e8f9dde1ea4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:38.804919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b79181-8741-434b-b23a-a9f2e0d1987f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:38.805313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b79181-8741-434b-b23a-a9f2e0d1987f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:38.805597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b79181-8741-434b-b23a-a9f2e0d1987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:38.925511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-97816c16-de6f-4498-992d-c4eeec69d6d3 None None] MFA Rules not processed for user `b5240b965bea4754842630a02fe410b6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:38.929623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-97816c16-de6f-4498-992d-c4eeec69d6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:38.950803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c4b79181-8741-434b-b23a-a9f2e0d1987f None None] MFA Rules not processed for user `4319ec4b90e1481b8be19e8f9dde1ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:38.954939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c4b79181-8741-434b-b23a-a9f2e0d1987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:38.957816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:39.103687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1100/2208] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:39.103687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787d30b-8996-4038-8995-800264689582 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.103687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787d30b-8996-4038-8995-800264689582 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.103687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787d30b-8996-4038-8995-800264689582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.139324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8787d30b-8996-4038-8995-800264689582 None None] MFA Rules not processed for user `7ba3c51770094c4ea7fec1e597ae13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:39.140558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8787d30b-8996-4038-8995-800264689582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:39.176377 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1109/2209] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:39.176377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97816c16-de6f-4498-992d-c4eeec69d6d3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.245718 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1101/2210] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:39.246711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787d30b-8996-4038-8995-800264689582 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.289377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97816c16-de6f-4498-992d-c4eeec69d6d3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.297657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.299431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.299949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.300490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.303333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.306230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.352111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787d30b-8996-4038-8995-800264689582 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.352622 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1110/2211] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:39 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:39.356541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.391891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:39.432914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.434317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.434786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.435504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.436780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.441485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.476049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383a56a-3e5b-4eda-a31f-0807f41c9390 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.481658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.482685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.483153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.483642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.484420 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1102/2212] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:39 2022] POST /identity/v3/projects => generated 373 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:39.484679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.486996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.489261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.527573 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1111/2213] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:39 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:39.528615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.541927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1ede5-f1c1-4005-a5a3-a9bb6a37a2e0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.593732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:39.597613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cadd5a-5cc3-4422-839c-39f5e59aacf3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.606777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.609863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.610416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.611435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.613755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.616374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.651091 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1103/2214] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:39 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:39.655039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.664969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.665608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.666372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.666660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.667503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.669634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.684638 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1112/2215] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:39.703762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.703854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bfb64-af72-4a69-b933-f2cd198dcf15 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.703941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3e55a-0ab0-4d9c-a0ac-8f4787bd08a1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.704803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.705446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.705724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.706056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.706646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.707656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.708448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.708852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.708960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.709242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] PATH_INFO: `/v3/projects/d1a7023a446f417e9764a1c113453548/users/d8b69329ff4d4a79b7cd0d056ae722e1/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.727688 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1104/2216] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:39.730451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a7023a446f417e9764a1c113453548, user_id=d8b69329ff4d4a79b7cd0d056ae722e1, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.732480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.741068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.762028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d92ff0-ac53-48cd-a7d3-4bc3e4781798 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.778036 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1113/2217] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:39 2022] PUT /identity/v3/projects/d1a7023a446f417e9764a1c113453548/users/d8b69329ff4d4a79b7cd0d056ae722e1/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:39.781997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.916557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62356c-daab-4218-a62e-ebe96d93b4d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.921972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.922632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.922909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.923181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.923928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.927485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.941386 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1114/2218] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:39.942580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:39.944816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:39.947080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.949888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.950034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.950772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] PATH_INFO: `/v3/projects/70a026c62fdc492595692981b5048113/users/14c589f8ca394271a00492169322be3b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.960966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16593080-f7d2-4217-bbe0-6ae674480d00 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:39.970690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:39.972074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:39.972074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:39.972074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] PATH_INFO: `/v3/projects/ec89ada1460d4275812339f2d6a93042` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:39.976487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec89ada1460d4275812339f2d6a93042)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.979207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:39.987713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a026c62fdc492595692981b5048113, user_id=14c589f8ca394271a00492169322be3b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:39.990986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:40.011337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1105/2219] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:39 2022] PUT /identity/v3/projects/70a026c62fdc492595692981b5048113/users/14c589f8ca394271a00492169322be3b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:40.013653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:40.103016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:40.146314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec89ada1460d4275812339f2d6a93042', 'request_id': 'req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.160558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec89ada1460d4275812339f2d6a93042', 'request_id': 'req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.161900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993975a2-a56b-4646-b731-0adae5574abf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:40.176329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec89ada1460d4275812339f2d6a93042', 'request_id': 'req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.203421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] The token cache is being invalidate because project ec89ada1460d4275812339f2d6a93042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:40.204134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.205066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.211965 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1115/2220] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:39 2022] DELETE /identity/v3/projects/ec89ada1460d4275812339f2d6a93042 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:40.211965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:40.214194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:40.240753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:40.281947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:40.282606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:40.282880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:40.283153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] PATH_INFO: `/v3/projects/d1a7023a446f417e9764a1c113453548/users/d8b69329ff4d4a79b7cd0d056ae722e1/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:40.286159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a7023a446f417e9764a1c113453548, user_id=d8b69329ff4d4a79b7cd0d056ae722e1, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:40.288953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:40.313330 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1106/2221] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:40 2022] PUT /identity/v3/projects/d1a7023a446f417e9764a1c113453548/users/d8b69329ff4d4a79b7cd0d056ae722e1/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:40.313330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:40.313330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:40.335836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc3636-9b17-4d7a-b720-48facbd2cfd5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:40.398380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2e142-90ef-48e4-92c5-85f7830c7a17 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:40.416431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:40.418377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dcff8-bd43-4cb6-b0ec-d294957f9a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:40.419046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:40.433453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:40.433453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] PATH_INFO: `/v3/users/4feaaf44afea40d392a5eb503c315413` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:40.433642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dcff8-bd43-4cb6-b0ec-d294957f9a51 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:40.433642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dcff8-bd43-4cb6-b0ec-d294957f9a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:40.433642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dcff8-bd43-4cb6-b0ec-d294957f9a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:40.433642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4dcff8-bd43-4cb6-b0ec-d294957f9a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:40.433642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4dcff8-bd43-4cb6-b0ec-d294957f9a51 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:40.435027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4feaaf44afea40d392a5eb503c315413)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:40.438291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:40.447055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1116/2222] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:40.448905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580d63-47de-45ea-971f-d75f6dd1958b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:40.449890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580d63-47de-45ea-971f-d75f6dd1958b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:40.450133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580d63-47de-45ea-971f-d75f6dd1958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:40.485564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0d580d63-47de-45ea-971f-d75f6dd1958b None None] MFA Rules not processed for user `d8b69329ff4d4a79b7cd0d056ae722e1`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:40.486785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0d580d63-47de-45ea-971f-d75f6dd1958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:40.515098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4feaaf44afea40d392a5eb503c315413'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.516252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4feaaf44afea40d392a5eb503c315413'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.529758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4feaaf44afea40d392a5eb503c315413'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.544303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4feaaf44afea40d392a5eb503c315413'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.548373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:40.572245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4feaaf44afea40d392a5eb503c315413'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:40.586971 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1107/2223] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:40 2022] DELETE /identity/v3/users/4feaaf44afea40d392a5eb503c315413 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:40.588015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:40.594315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:40.596819 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1117/2224] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:40.611391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:40.611823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:40.612765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:40.652209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] MFA Rules not processed for user `4319ec4b90e1481b8be19e8f9dde1ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:40.655667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:40.742729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692ff7b-f9f7-43ae-8e48-e96be29ec2b3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:40.748551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941c69e-36e7-4eb7-b2c8-799c782e6c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:40.749565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c69e-36e7-4eb7-b2c8-799c782e6c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:40.750023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c69e-36e7-4eb7-b2c8-799c782e6c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:40.750514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c69e-36e7-4eb7-b2c8-799c782e6c8a admin admin] PATH_INFO: `/v3/projects/70a026c62fdc492595692981b5048113/users/14c589f8ca394271a00492169322be3b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:40.753651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941c69e-36e7-4eb7-b2c8-799c782e6c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a026c62fdc492595692981b5048113, user_id=14c589f8ca394271a00492169322be3b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:40.757173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941c69e-36e7-4eb7-b2c8-799c782e6c8a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:40.778500 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1108/2225] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:40 2022] PUT /identity/v3/projects/70a026c62fdc492595692981b5048113/users/14c589f8ca394271a00492169322be3b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:40.783016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:40.791117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:40.791492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:40.792064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:40.876780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] MFA Rules not processed for user `14c589f8ca394271a00492169322be3b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:40.878052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:40.899228 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1118/2226] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:40.909399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:40.911276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:41.023182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1109/2227] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:41.049453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2730345-b0c7-4e14-ad6c-9d52d7eb3288 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:41.170726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:41.171509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:41.171794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:41.172064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:41.174559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:41.283467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:41.288282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:41.321816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1119/2228] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:41.514752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:41.519154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:41.540638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27334b7b-3d96-4025-9b37-64408d7a9f89 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:41.548096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3c142-bc8c-469e-bd34-d16810cb5c01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:41.548840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3c142-bc8c-469e-bd34-d16810cb5c01 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:41.549152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3c142-bc8c-469e-bd34-d16810cb5c01 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:41.549423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3c142-bc8c-469e-bd34-d16810cb5c01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:41.560818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3c142-bc8c-469e-bd34-d16810cb5c01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:41.563754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3c142-bc8c-469e-bd34-d16810cb5c01 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:41.576459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:41.579203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:41.579343 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1110/2229] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:41.629529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:41.699060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497cbbc-9828-4dc8-b99a-e4fb7178fccb service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:41.707342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:41.708423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:41.708978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:41.709472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] PATH_INFO: `/v3/projects/02e1a07303744cf099095529bf775200` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:41.711147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e1a07303744cf099095529bf775200)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:41.713554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:41.821820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e1a07303744cf099095529bf775200', 'request_id': 'req-43012d82-4a2f-4874-865e-1b1bd52c5269'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:41.838815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e1a07303744cf099095529bf775200', 'request_id': 'req-43012d82-4a2f-4874-865e-1b1bd52c5269'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:41.856253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e1a07303744cf099095529bf775200', 'request_id': 'req-43012d82-4a2f-4874-865e-1b1bd52c5269'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:41.893608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] The token cache is being invalidate because project 02e1a07303744cf099095529bf775200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:41.895491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:41.899511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43012d82-4a2f-4874-865e-1b1bd52c5269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:41.903658 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1120/2230] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:41 2022] DELETE /identity/v3/projects/02e1a07303744cf099095529bf775200 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:41.910875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4ab3-e58d-46d7-a636-9024cd12282a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:41.911314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4ab3-e58d-46d7-a636-9024cd12282a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:41.911764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4ab3-e58d-46d7-a636-9024cd12282a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:41.965149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-90ea4ab3-e58d-46d7-a636-9024cd12282a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:41.965149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90ea4ab3-e58d-46d7-a636-9024cd12282a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:42.092414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:42.162754 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1111/2231] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:42.168239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79fdf4-03db-42ec-ae6a-e698bbc5ee1e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.168549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79fdf4-03db-42ec-ae6a-e698bbc5ee1e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.168892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79fdf4-03db-42ec-ae6a-e698bbc5ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.192457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7c79fdf4-03db-42ec-ae6a-e698bbc5ee1e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:42.194217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c79fdf4-03db-42ec-ae6a-e698bbc5ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:42.205083 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1121/2232] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:42.210420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea4ab3-e58d-46d7-a636-9024cd12282a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:42.223083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea4ab3-e58d-46d7-a636-9024cd12282a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:42.241275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:42.242041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.242381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.242662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.243401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:42.243709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:42.247902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:42.298647 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1112/2233] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:42.300580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79fdf4-03db-42ec-ae6a-e698bbc5ee1e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:42.315151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79fdf4-03db-42ec-ae6a-e698bbc5ee1e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:42.328047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:42.329795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.333114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.333602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.334510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:42.341508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:42.410942 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1122/2234] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:42 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:42.412111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:42.430273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7bd6-c485-44ce-b068-d7d6bfab35d1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:42.515644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:42.516314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.516549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.516860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.517636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:42.519683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:42.524384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:42.526427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:42.591425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1113/2235] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:42 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:42.597467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:42.638646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204517a-9835-4cca-9153-3fbf2db67d30 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:42.669244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:42.670283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.670606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.670967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.683633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:42.686346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:42.715897 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1114/2236] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:42.720923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:42.749720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762381f-af9f-4d62-b330-15a801ea25af admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:42.756404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:42.757233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.758545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.758830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.759734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:42.763013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:42.792829 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1115/2237] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:42.797914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:42.811626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74de8b-de47-42b5-adb5-8fa9c160c7b1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:42.817722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f90f5c-d154-406a-927b-21256282f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:42.818455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f90f5c-d154-406a-927b-21256282f5c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.818836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f90f5c-d154-406a-927b-21256282f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.819261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f90f5c-d154-406a-927b-21256282f5c0 admin admin] PATH_INFO: `/v3/projects/7e6987197b0a4c7d9b89d48402845f3f/users/aa2474e7686c4a758ed93064edf47867/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.849898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f90f5c-d154-406a-927b-21256282f5c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6987197b0a4c7d9b89d48402845f3f, user_id=aa2474e7686c4a758ed93064edf47867, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:42.853307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f90f5c-d154-406a-927b-21256282f5c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:42.855058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f88f80-b5fd-4093-ae25-b794236b6839 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:42.873877 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1116/2238] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:42 2022] PUT /identity/v3/projects/7e6987197b0a4c7d9b89d48402845f3f/users/aa2474e7686c4a758ed93064edf47867/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:42.881956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:42.884122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.885714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3035-0f71-4194-a02b-9920de98b154 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:42.885965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.886077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3035-0f71-4194-a02b-9920de98b154 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:42.886256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3035-0f71-4194-a02b-9920de98b154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.886682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:42.895433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:42.927384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:42.960023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fb1f3035-0f71-4194-a02b-9920de98b154 None None] MFA Rules not processed for user `aa2474e7686c4a758ed93064edf47867`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:42.961867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fb1f3035-0f71-4194-a02b-9920de98b154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:43.022625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:43.026951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:43.052283 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1123/2239] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:57:43.075747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1117/2240] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:43.075971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:43.097152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c809f5f-aacb-4c66-b9b7-395a2ec0cf0c service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:43.183631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:43.184427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:43.184831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:43.185178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:43.185921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:43.188339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:43.252481 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1124/2241] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:43 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:43.258461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3035-0f71-4194-a02b-9920de98b154 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:43.278001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3035-0f71-4194-a02b-9920de98b154 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:43.285426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:43.431861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:43.432595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:43.432976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:43.433526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:43.434485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:43.439568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:43.490006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1118/2242] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:43 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:43.496120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:43.515234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f299-db0b-4bad-a738-68aa3956702f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:43.527917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:43.529098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:43.529441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:43.529847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:43.530717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:43.532901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:43.550970 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1125/2243] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:43.556131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:43.576688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0c817-6415-444d-8b58-59b9a2674326 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:43.586205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:43.587342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:43.587620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:43.587891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] PATH_INFO: `/v3/projects/4d2d9d09939f4d6fb8d0a9f4c9068813/users/b3e38e5269224ed2aed7bf6b00141a14/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:43.630411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2d9d09939f4d6fb8d0a9f4c9068813, user_id=b3e38e5269224ed2aed7bf6b00141a14, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:43.636367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:43.675610 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1119/2244] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:43 2022] PUT /identity/v3/projects/4d2d9d09939f4d6fb8d0a9f4c9068813/users/b3e38e5269224ed2aed7bf6b00141a14/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:43.680806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:43.692567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017a56-7c07-4eed-a659-c06c52a95fb0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:43.770403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:43.771422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:43.771940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:43.772689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:43.773954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:43.775971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:43.795446 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1126/2245] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:43.796910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:43.824748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90d7d-dd60-4984-8bdc-8803a2f8260a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:43.830527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dcad4-7212-44c3-a503-e3058e6edce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:43.831327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dcad4-7212-44c3-a503-e3058e6edce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:43.831747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dcad4-7212-44c3-a503-e3058e6edce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:43.832102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dcad4-7212-44c3-a503-e3058e6edce2 admin admin] PATH_INFO: `/v3/projects/4d2d9d09939f4d6fb8d0a9f4c9068813/users/b3e38e5269224ed2aed7bf6b00141a14/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:43.834945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dcad4-7212-44c3-a503-e3058e6edce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2d9d09939f4d6fb8d0a9f4c9068813, user_id=b3e38e5269224ed2aed7bf6b00141a14, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:43.839308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dcad4-7212-44c3-a503-e3058e6edce2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:43.855022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1120/2246] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:43 2022] PUT /identity/v3/projects/4d2d9d09939f4d6fb8d0a9f4c9068813/users/b3e38e5269224ed2aed7bf6b00141a14/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:43.868018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:43.868604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:43.869107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:43.869870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:43.873831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:43.949715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] MFA Rules not processed for user `b3e38e5269224ed2aed7bf6b00141a14`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:43.951214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:43.974409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:44.046352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac5fe-8fcd-4c57-aba2-081f7bd69fa8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:44.104311 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1121/2247] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:44.105480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:44.107158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:44.208850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563a033-fca6-4904-bcd8-9b15efef5e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:44.209698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563a033-fca6-4904-bcd8-9b15efef5e05 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:44.209984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563a033-fca6-4904-bcd8-9b15efef5e05 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:44.210950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563a033-fca6-4904-bcd8-9b15efef5e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:44.214534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1563a033-fca6-4904-bcd8-9b15efef5e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:44.234807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:44.280538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88477a3-231e-4bea-8c4f-c1a12cde7011 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:44.301121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:44.302221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:44.305040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:44.305555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] PATH_INFO: `/v3/users/b5240b965bea4754842630a02fe410b6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:44.311925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5240b965bea4754842630a02fe410b6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:44.314520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:44.409878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563a033-fca6-4904-bcd8-9b15efef5e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:44.412481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563a033-fca6-4904-bcd8-9b15efef5e05 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:44.419169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5240b965bea4754842630a02fe410b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:44.421482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5240b965bea4754842630a02fe410b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:44.433162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5240b965bea4754842630a02fe410b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:44.452493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5240b965bea4754842630a02fe410b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:44.453531 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1127/2248] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:44.454695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:44.455171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:44.455292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:44.480798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e887a40-25e4-4997-aa87-f88b0e9f8b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5240b965bea4754842630a02fe410b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:44.497172 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1122/2249] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:44 2022] DELETE /identity/v3/users/b5240b965bea4754842630a02fe410b6 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:44.501130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:44.502030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:44.502494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:44.519067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] MFA Rules not processed for user `aa2474e7686c4a758ed93064edf47867`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:44.520473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:44.642818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:44.647606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:44.702003 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1128/2250] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:44.868176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1123/2251] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:44.884039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:44.885749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:44.905763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:45.074931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3bd5a-7fea-4431-80c9-fda81de51e45 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:45.081384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:45.082979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:45.083323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:45.083669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:45.098469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:45.100854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:45.116965 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1129/2252] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:45.142547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:45.144157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:45.274322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:45.282058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc152e05-b943-489e-b217-a586c9d91466 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:45.325121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7548d8-9741-4398-bd1d-6ad20d0d35cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:45.325924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7548d8-9741-4398-bd1d-6ad20d0d35cb service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:45.326185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7548d8-9741-4398-bd1d-6ad20d0d35cb service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:45.326461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7548d8-9741-4398-bd1d-6ad20d0d35cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:45.330967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bf7548d8-9741-4398-bd1d-6ad20d0d35cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:45.355388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7548d8-9741-4398-bd1d-6ad20d0d35cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:45.357074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7548d8-9741-4398-bd1d-6ad20d0d35cb service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:45.375644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1124/2253] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:45.724690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:45.729163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:45.752229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6d7e-2d70-4b18-85e4-fd6ddbbdc794 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:45.771965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:45.772627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:45.772931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:45.773210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] PATH_INFO: `/v3/projects/0db301529f1b4836841b0c1a1ce9bb42` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:45.776520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db301529f1b4836841b0c1a1ce9bb42)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:45.778294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:45.787810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e90a58-ecc9-4894-82a8-c5f601e2c147 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:45.787810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e90a58-ecc9-4894-82a8-c5f601e2c147 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:45.788574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e90a58-ecc9-4894-82a8-c5f601e2c147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:45.823537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d2e90a58-ecc9-4894-82a8-c5f601e2c147 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:45.825365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d2e90a58-ecc9-4894-82a8-c5f601e2c147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:45.882523 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1125/2254] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 09:57:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:45.897057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db301529f1b4836841b0c1a1ce9bb42', 'request_id': 'req-a3e7bc7b-2498-4c0f-87ae-284bc852833e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:45.914432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db301529f1b4836841b0c1a1ce9bb42', 'request_id': 'req-a3e7bc7b-2498-4c0f-87ae-284bc852833e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:45.924227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db301529f1b4836841b0c1a1ce9bb42', 'request_id': 'req-a3e7bc7b-2498-4c0f-87ae-284bc852833e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:45.952428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] The token cache is being invalidate because project 0db301529f1b4836841b0c1a1ce9bb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:45.953702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:45.955301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:45.959046 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1130/2255] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:45 2022] DELETE /identity/v3/projects/0db301529f1b4836841b0c1a1ce9bb42 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:45.965540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e90a58-ecc9-4894-82a8-c5f601e2c147 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:45.996589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e90a58-ecc9-4894-82a8-c5f601e2c147 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:45.997858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:46.000018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:46.142908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7bc7b-2498-4c0f-87ae-284bc852833e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:46.149050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:46.149735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:46.150018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:46.150271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] PATH_INFO: `/v3/users/14c589f8ca394271a00492169322be3b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:46.152557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c589f8ca394271a00492169322be3b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:46.160846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532f729-a9d7-4f57-85de-ec3e9064ea60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:46.161701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532f729-a9d7-4f57-85de-ec3e9064ea60 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:46.162077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532f729-a9d7-4f57-85de-ec3e9064ea60 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:46.163588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532f729-a9d7-4f57-85de-ec3e9064ea60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:46.164537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:46.169245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d532f729-a9d7-4f57-85de-ec3e9064ea60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:46.187283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532f729-a9d7-4f57-85de-ec3e9064ea60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:46.189075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532f729-a9d7-4f57-85de-ec3e9064ea60 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:46.199975 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1131/2256] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:46.291860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c589f8ca394271a00492169322be3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:46.293075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c589f8ca394271a00492169322be3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:46.308915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c589f8ca394271a00492169322be3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:46.320891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c589f8ca394271a00492169322be3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:46.342662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b94a5e0-d3b6-40c4-ae73-7c4be5cb80a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c589f8ca394271a00492169322be3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:46.355273 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1126/2257] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:45 2022] DELETE /identity/v3/users/14c589f8ca394271a00492169322be3b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:46.647990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811b9b-0892-4093-b775-f4b459650ded None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:46.648437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811b9b-0892-4093-b775-f4b459650ded None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:46.648917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811b9b-0892-4093-b775-f4b459650ded None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:46.650509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1132/2258] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:57:46.666860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:46.667503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:46.667972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:46.705961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:46.707501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:46.829953 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1127/2259] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:57:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:46.865504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64811b9b-0892-4093-b775-f4b459650ded None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:46.868078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-64811b9b-0892-4093-b775-f4b459650ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:46.896797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64811b9b-0892-4093-b775-f4b459650ded None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:46.921253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:46.921978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:46.922303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:46.922584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:46.950124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:46.954714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:46.992956 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1133/2260] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:47.324121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:47.325928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:47.344424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:47.404558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:47.407612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:47.493847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:47.518137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e891e-dd9a-4c51-9b11-fd958331a556 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:47.525779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8cec1-84c9-445b-a48c-4587d97f8392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:47.527457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8cec1-84c9-445b-a48c-4587d97f8392 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:47.527457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8cec1-84c9-445b-a48c-4587d97f8392 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:47.527457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8cec1-84c9-445b-a48c-4587d97f8392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:47.535507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-29d8cec1-84c9-445b-a48c-4587d97f8392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:47.568272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fbbad-220f-4efd-8912-cb7fe05fb19c service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:47.571376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8cec1-84c9-445b-a48c-4587d97f8392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:47.573765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8cec1-84c9-445b-a48c-4587d97f8392 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:47.578159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:47.579313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:47.579813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:47.580406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] PATH_INFO: `/v3/projects/70a026c62fdc492595692981b5048113` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:47.581988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a026c62fdc492595692981b5048113)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:47.584390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:47.598194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1128/2261] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:47.721148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a026c62fdc492595692981b5048113', 'request_id': 'req-7b7db0b1-dd13-4771-95aa-24fca42bfeda'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:47.736789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a026c62fdc492595692981b5048113', 'request_id': 'req-7b7db0b1-dd13-4771-95aa-24fca42bfeda'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:47.749529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a026c62fdc492595692981b5048113', 'request_id': 'req-7b7db0b1-dd13-4771-95aa-24fca42bfeda'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:47.775509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] The token cache is being invalidate because project 70a026c62fdc492595692981b5048113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:47.775832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:47.776518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b7db0b1-dd13-4771-95aa-24fca42bfeda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:47.778916 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1134/2262] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:47 2022] DELETE /identity/v3/projects/70a026c62fdc492595692981b5048113 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:47.786477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a894f0-1bc2-472d-abe3-1b8af3edbc52 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:47.788622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a894f0-1bc2-472d-abe3-1b8af3edbc52 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:47.788622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a894f0-1bc2-472d-abe3-1b8af3edbc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:47.846343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-51a894f0-1bc2-472d-abe3-1b8af3edbc52 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:47.847851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-51a894f0-1bc2-472d-abe3-1b8af3edbc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:47.915049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171bd56-94e8-4726-8dfd-ae3bd3d06e3a None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:47.915501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171bd56-94e8-4726-8dfd-ae3bd3d06e3a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:47.915683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171bd56-94e8-4726-8dfd-ae3bd3d06e3a None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:47.917394 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1135/2263] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:57:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:57:47.947468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bccc-56a2-455e-bbd9-b030234aeae0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:47.948143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bccc-56a2-455e-bbd9-b030234aeae0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:47.948374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bccc-56a2-455e-bbd9-b030234aeae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:47.983979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bf64bccc-56a2-455e-bbd9-b030234aeae0 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:47.985458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bf64bccc-56a2-455e-bbd9-b030234aeae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:48.002944 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1129/2264] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:48.012241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adf472-8ef0-4ca5-85aa-deb4b2d7cb88 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.012617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adf472-8ef0-4ca5-85aa-deb4b2d7cb88 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.013072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adf472-8ef0-4ca5-85aa-deb4b2d7cb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.061166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a4adf472-8ef0-4ca5-85aa-deb4b2d7cb88 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:48.062970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a4adf472-8ef0-4ca5-85aa-deb4b2d7cb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:48.075883 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1130/2265] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:48.080254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adf472-8ef0-4ca5-85aa-deb4b2d7cb88 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:48.109194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adf472-8ef0-4ca5-85aa-deb4b2d7cb88 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:48.129731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:48.131020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.131432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.132419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.139187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:48.139187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:48.141309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:48.170875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1136/2266] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:57:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:48.189463 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1131/2267] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:48.193162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64bccc-56a2-455e-bbd9-b030234aeae0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:48.218790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64bccc-56a2-455e-bbd9-b030234aeae0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:48.232328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:48.234650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.237145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.238450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.239492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:48.248292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:48.276751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:48.278802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:48.326891 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1137/2268] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:48 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:48.330656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:48.376355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e04c-0ef6-4e44-9f42-56fccdc77827 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:48.382901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:48.440108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5a6f1-06db-4323-bf57-9b2c74cf1d52 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:48.443101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:48.449452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:48.450093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.450420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.450693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.451500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:48.454124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:48.458179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb54c8-149f-4d92-a585-a7741dfd4205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:48.459055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb54c8-149f-4d92-a585-a7741dfd4205 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.459572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb54c8-149f-4d92-a585-a7741dfd4205 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.459915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb54c8-149f-4d92-a585-a7741dfd4205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.485749 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1138/2269] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:48 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:48.489917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:48.501570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0300b6b-79ab-42db-9f36-975c2658c402 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:48.509257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:48.510245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.510727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.511193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.512087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:48.514009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:48.531614 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1139/2270] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:48.532920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:48.550844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572693-22cb-4995-a4b0-3ce2aa03220d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:48.563795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:48.564955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.565467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.566515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.567807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:48.571166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:48.584491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb54c8-149f-4d92-a585-a7741dfd4205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:48.588347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb54c8-149f-4d92-a585-a7741dfd4205 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:48.592280 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1140/2271] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:48.600155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:48.628377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb2fa-f87d-407a-b920-4d11bb044a3a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:48.630737 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1132/2272] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:48.638389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:48.639128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.639404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.639671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] PATH_INFO: `/v3/projects/fc9efd4e873645e5bc70b71d8f7b2070/users/1ea9ba5d8d654878bc6a0aa344a9afd4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.661095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9efd4e873645e5bc70b71d8f7b2070, user_id=1ea9ba5d8d654878bc6a0aa344a9afd4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:48.664566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:48.686681 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1141/2273] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:48 2022] PUT /identity/v3/projects/fc9efd4e873645e5bc70b71d8f7b2070/users/1ea9ba5d8d654878bc6a0aa344a9afd4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:48.698023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922464b-fa88-4cef-aa67-e24ec5d05214 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:48.698307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922464b-fa88-4cef-aa67-e24ec5d05214 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:48.698583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922464b-fa88-4cef-aa67-e24ec5d05214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:48.742902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b922464b-fa88-4cef-aa67-e24ec5d05214 None None] MFA Rules not processed for user `1ea9ba5d8d654878bc6a0aa344a9afd4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:48.744972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b922464b-fa88-4cef-aa67-e24ec5d05214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:48.893415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1133/2274] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:48.897449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:49.008085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534b71f-b991-456f-b385-eb3c2b566d46 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:49.015903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:49.018080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:49.018446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:49.018723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:49.019381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:49.021498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:49.063168 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1142/2275] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:48 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:49.065070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922464b-fa88-4cef-aa67-e24ec5d05214 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:49.083559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922464b-fa88-4cef-aa67-e24ec5d05214 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:49.174902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:49.175523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:49.175796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:49.176060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:49.176739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:49.179010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:49.199531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:49.206250 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1134/2276] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:49 2022] POST /identity/v3/users => generated 296 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:49.210390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:49.225168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ed860-55a2-4da8-a382-efcf2203c98e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:49.231835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:49.232469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:49.232909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:49.233094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:49.233702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:49.235331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:49.247509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1143/2277] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:49.252225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:49.262839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f65dc-90ff-47c3-be29-9044bed13205 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:49.269380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:49.270336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:49.270829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:49.271243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] PATH_INFO: `/v3/projects/0d8077ff65ff4cafb22fa40ab18428a5/users/deaf8735f89a41cf903906a8ca2dd989/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:49.290013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8077ff65ff4cafb22fa40ab18428a5, user_id=deaf8735f89a41cf903906a8ca2dd989, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:49.292567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:49.315183 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1135/2278] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:49 2022] PUT /identity/v3/projects/0d8077ff65ff4cafb22fa40ab18428a5/users/deaf8735f89a41cf903906a8ca2dd989/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:49.319258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:49.330509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81b853-a4bb-4755-8cdb-5b92c86700fb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:49.390352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:49.419434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd0ee5-2a99-4e69-91c6-ae1cd82da2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:49.420389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0ee5-2a99-4e69-91c6-ae1cd82da2de admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:49.420938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0ee5-2a99-4e69-91c6-ae1cd82da2de admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:49.421426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0ee5-2a99-4e69-91c6-ae1cd82da2de admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:49.422623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd0ee5-2a99-4e69-91c6-ae1cd82da2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:49.425778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd0ee5-2a99-4e69-91c6-ae1cd82da2de admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:49.452396 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1144/2279] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:49.457567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:49.478309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566b5d2-b99b-4fa2-bd50-3be5f1daf61d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:49.495573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:49.496260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:49.496546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:49.496851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] PATH_INFO: `/v3/projects/0d8077ff65ff4cafb22fa40ab18428a5/users/deaf8735f89a41cf903906a8ca2dd989/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:49.502105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8077ff65ff4cafb22fa40ab18428a5, user_id=deaf8735f89a41cf903906a8ca2dd989, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:49.505978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:49.536482 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1136/2280] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:49 2022] PUT /identity/v3/projects/0d8077ff65ff4cafb22fa40ab18428a5/users/deaf8735f89a41cf903906a8ca2dd989/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:49.541685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f612-4fbf-463e-89e7-635672c5d40d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:49.541685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f612-4fbf-463e-89e7-635672c5d40d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:49.541999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f612-4fbf-463e-89e7-635672c5d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:49.603226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bbc8f612-4fbf-463e-89e7-635672c5d40d None None] MFA Rules not processed for user `deaf8735f89a41cf903906a8ca2dd989`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:49.604626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bbc8f612-4fbf-463e-89e7-635672c5d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:49.728363 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1145/2281] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:49.908873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:49.913308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:50.038985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8a8dc-1e4e-43bc-819f-f6076deff4a9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:50.047676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f562b2-10a1-4f5d-8b55-76c06f04528d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:50.048422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f562b2-10a1-4f5d-8b55-76c06f04528d service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:50.048786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f562b2-10a1-4f5d-8b55-76c06f04528d service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:50.049057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f562b2-10a1-4f5d-8b55-76c06f04528d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:50.061837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f562b2-10a1-4f5d-8b55-76c06f04528d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:50.063625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f562b2-10a1-4f5d-8b55-76c06f04528d service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:50.077985 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1137/2282] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:50.867957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8f612-4fbf-463e-89e7-635672c5d40d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:51.014629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8f612-4fbf-463e-89e7-635672c5d40d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:51.022370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:51.023516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:51.023999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:51.024468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] PATH_INFO: `/v3/users/1ea9ba5d8d654878bc6a0aa344a9afd4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:51.026411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea9ba5d8d654878bc6a0aa344a9afd4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:51.030094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:51.133866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea9ba5d8d654878bc6a0aa344a9afd4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:51.134994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea9ba5d8d654878bc6a0aa344a9afd4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:51.147150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea9ba5d8d654878bc6a0aa344a9afd4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:51.159753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea9ba5d8d654878bc6a0aa344a9afd4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:51.161919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea9ba5d8d654878bc6a0aa344a9afd4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:51.164814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:51.215870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:51.217070 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1146/2283] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:50 2022] DELETE /identity/v3/users/1ea9ba5d8d654878bc6a0aa344a9afd4 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:51.217932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:51.217932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:51.266115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:51.267667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:51.354774 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1138/2284] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:51.371242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:51.373357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:51.555389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981725e-9fed-4044-966c-4c2d45248c31 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:51.566593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:51.567521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:51.568366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:51.569279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:51.585921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:51.587863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:51.612722 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1147/2285] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:57:52.215333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:52.217833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:52.266142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32005-aa2a-4b52-ac62-150b8bc68b91 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:52.299799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:52.303691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:52.304037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:52.307874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] PATH_INFO: `/v3/projects/fc9efd4e873645e5bc70b71d8f7b2070` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:52.313524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc9efd4e873645e5bc70b71d8f7b2070)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:52.318881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:52.427338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:52.435749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc9efd4e873645e5bc70b71d8f7b2070', 'request_id': 'req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.446694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc9efd4e873645e5bc70b71d8f7b2070', 'request_id': 'req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.464727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc9efd4e873645e5bc70b71d8f7b2070', 'request_id': 'req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.499157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] The token cache is being invalidate because project fc9efd4e873645e5bc70b71d8f7b2070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:52.499632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.501148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.506989 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1139/2286] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:52 2022] DELETE /identity/v3/projects/fc9efd4e873645e5bc70b71d8f7b2070 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:52.508374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:52.531993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335bce-6583-48d9-8420-f53f608246ed service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:52.604028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:52.604823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:52.605103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:52.605380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] PATH_INFO: `/v3/users/deaf8735f89a41cf903906a8ca2dd989` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:52.606646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deaf8735f89a41cf903906a8ca2dd989)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:52.608747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:52.708391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deaf8735f89a41cf903906a8ca2dd989'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.709225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deaf8735f89a41cf903906a8ca2dd989'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.723491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deaf8735f89a41cf903906a8ca2dd989'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.739319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deaf8735f89a41cf903906a8ca2dd989'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.740336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea9eaa1-b3d0-4ec0-a7ec-c38a1f53b8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deaf8735f89a41cf903906a8ca2dd989'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:52.780716 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1148/2287] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:52 2022] DELETE /identity/v3/users/deaf8735f89a41cf903906a8ca2dd989 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:53.433055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:53.435172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:53.450675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97862a8e-7dbd-4e97-94e1-f9b61dc9aee5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:53.565069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:53.565749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:53.566028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:53.566304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] PATH_INFO: `/v3/projects/0d8077ff65ff4cafb22fa40ab18428a5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:53.567546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8077ff65ff4cafb22fa40ab18428a5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:53.569901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:53.643039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8077ff65ff4cafb22fa40ab18428a5', 'request_id': 'req-e26580de-e770-4c11-9e0f-fd69b31c4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:53.652420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8077ff65ff4cafb22fa40ab18428a5', 'request_id': 'req-e26580de-e770-4c11-9e0f-fd69b31c4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:53.663877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8077ff65ff4cafb22fa40ab18428a5', 'request_id': 'req-e26580de-e770-4c11-9e0f-fd69b31c4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:53.698797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] The token cache is being invalidate because project 0d8077ff65ff4cafb22fa40ab18428a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:53.698797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:53.698797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e26580de-e770-4c11-9e0f-fd69b31c4244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:53.698797 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1140/2288] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:53 2022] DELETE /identity/v3/projects/0d8077ff65ff4cafb22fa40ab18428a5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:53.704893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2ff58-abba-4a07-bd39-7783afd09019 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:53.704893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2ff58-abba-4a07-bd39-7783afd09019 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:53.704893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2ff58-abba-4a07-bd39-7783afd09019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:53.724533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:53.745362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8df2ff58-abba-4a07-bd39-7783afd09019 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:53.746791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8df2ff58-abba-4a07-bd39-7783afd09019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:53.840080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1149/2289] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:53.848219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ab16-ac5a-4c1f-b46d-0fd12c91cb0a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:53.848459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ab16-ac5a-4c1f-b46d-0fd12c91cb0a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:53.848765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ab16-ac5a-4c1f-b46d-0fd12c91cb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:53.877759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2cf1ab16-ac5a-4c1f-b46d-0fd12c91cb0a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:53.879007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2cf1ab16-ac5a-4c1f-b46d-0fd12c91cb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:53.894354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1141/2290] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:53.895015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2ff58-abba-4a07-bd39-7783afd09019 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:53.907707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2ff58-abba-4a07-bd39-7783afd09019 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:53.925724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dd4fa-1172-46a7-a66f-bcfd7a3fdaff None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:53.926004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dd4fa-1172-46a7-a66f-bcfd7a3fdaff None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:53.926286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dd4fa-1172-46a7-a66f-bcfd7a3fdaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:53.930879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:53.931607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:53.931792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:53.932062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:53.932775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:53.933794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:53.943767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:53.959095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a60dd4fa-1172-46a7-a66f-bcfd7a3fdaff None None] MFA Rules not processed for user `00423eb1e4c24763ae0a546225ac1dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:53.960623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a60dd4fa-1172-46a7-a66f-bcfd7a3fdaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:53.975637 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1150/2291] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:57:53.976945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:54.005694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3da06-edbf-453f-9386-5e802f1e6fe7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:54.025877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:54.026526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.026796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.027067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.027716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:54.030421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:54.083950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:54.090789 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1151/2292] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:53 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:54.095429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:54.104866 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1142/2293] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:54.200221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa16c80-769d-4204-8996-24e013403048 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:54.205951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:54.206589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.206867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.207137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.207790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:54.209916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:54.248246 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1152/2294] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:54 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:57:54.249451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dd4fa-1172-46a7-a66f-bcfd7a3fdaff None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:54.263286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dd4fa-1172-46a7-a66f-bcfd7a3fdaff None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:54.269031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:54.269671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.269949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.270221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.270899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:54.272943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:54.289365 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1143/2295] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:54.294035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:54.309415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f4e2e-3b63-49b6-82e9-e18fef109521 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:54.315671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b5787-b21c-464e-8cf4-94075b400e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:54.316370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b5787-b21c-464e-8cf4-94075b400e90 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.316819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b5787-b21c-464e-8cf4-94075b400e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.317007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b5787-b21c-464e-8cf4-94075b400e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.317757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b5787-b21c-464e-8cf4-94075b400e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:54.320160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b5787-b21c-464e-8cf4-94075b400e90 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:54.336771 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1153/2296] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:57:54.343035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:54.358598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbe935-cb72-4bf6-b52c-43dd7c83a891 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:54.365346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:54.366206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.366608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.366974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] PATH_INFO: `/v3/projects/97ce5e66d63a4b329e23d823358de67c/users/5de63f9d38a644c593ec59b4758dd102/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.388843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ce5e66d63a4b329e23d823358de67c, user_id=5de63f9d38a644c593ec59b4758dd102, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:54.392925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:54.415377 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1144/2297] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:54 2022] PUT /identity/v3/projects/97ce5e66d63a4b329e23d823358de67c/users/5de63f9d38a644c593ec59b4758dd102/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:54.431314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2211820-64ac-44e9-a940-25775c732ca2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.432279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2211820-64ac-44e9-a940-25775c732ca2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.433815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2211820-64ac-44e9-a940-25775c732ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.505597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e2211820-64ac-44e9-a940-25775c732ca2 None None] MFA Rules not processed for user `5de63f9d38a644c593ec59b4758dd102`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:54.508863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e2211820-64ac-44e9-a940-25775c732ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:54.585200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:54.659764 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1154/2298] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:54.666092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:54.765926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb731e6-11a4-4320-a55f-a77019911653 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:54.775105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:54.781020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.784143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.784585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.785697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:54.791769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:54.849780 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1145/2299] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:54 2022] POST /identity/v3/projects => generated 389 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:54.853924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2211820-64ac-44e9-a940-25775c732ca2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:54.868643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2211820-64ac-44e9-a940-25775c732ca2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:54.969544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:54.970099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:54.970603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:54.970896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:54.972063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:54.973921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:55.022242 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1155/2300] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:54 2022] POST /identity/v3/users => generated 301 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:55.023632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:55.039971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbae03-49e3-400a-8bbe-783c187afe89 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:55.058672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:55.059475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:55.060033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:55.060246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:55.061093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:55.063646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:55.081143 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1146/2301] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:55.087280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:55.100591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb44a92-c9a6-480c-a386-763301408e17 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:55.106312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:55.106986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:55.107257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:55.107529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] PATH_INFO: `/v3/projects/13ff23326cbd4f07bb68f0ddbe653320/users/c533b81c8eda406e8bded7610880fb6b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:55.125702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ff23326cbd4f07bb68f0ddbe653320, user_id=c533b81c8eda406e8bded7610880fb6b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:55.128517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:55.149307 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1156/2302] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:55 2022] PUT /identity/v3/projects/13ff23326cbd4f07bb68f0ddbe653320/users/c533b81c8eda406e8bded7610880fb6b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:55.153844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:55.166022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4f9a-666d-4658-bd26-6b046f0d205f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:55.254214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117deefa-b412-4832-88af-ac584e161523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:55.254966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117deefa-b412-4832-88af-ac584e161523 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:55.255307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117deefa-b412-4832-88af-ac584e161523 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:55.255651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117deefa-b412-4832-88af-ac584e161523 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:55.256397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117deefa-b412-4832-88af-ac584e161523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:55.259249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117deefa-b412-4832-88af-ac584e161523 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:55.272694 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1147/2303] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:55.276810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:55.294485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e29782-7258-4634-a774-c51a82fe3f61 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:55.299876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:55.300723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:55.301064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:55.301397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] PATH_INFO: `/v3/projects/13ff23326cbd4f07bb68f0ddbe653320/users/c533b81c8eda406e8bded7610880fb6b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:55.304315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ff23326cbd4f07bb68f0ddbe653320, user_id=c533b81c8eda406e8bded7610880fb6b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:55.308331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:55.336694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1157/2304] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:55 2022] PUT /identity/v3/projects/13ff23326cbd4f07bb68f0ddbe653320/users/c533b81c8eda406e8bded7610880fb6b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:55.346161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71559-acff-4fd1-861b-25a951d43384 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:55.346463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71559-acff-4fd1-861b-25a951d43384 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:55.346698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71559-acff-4fd1-861b-25a951d43384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:55.388858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fea71559-acff-4fd1-861b-25a951d43384 None None] MFA Rules not processed for user `c533b81c8eda406e8bded7610880fb6b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:55.390164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fea71559-acff-4fd1-861b-25a951d43384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:55.423431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:55.487110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1148/2305] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:55 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:55.500519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:55.502407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:55.650491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bc30b-b0d2-4936-91ce-47a71f6a465d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:55.657044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129686-c29e-49ab-934b-19112ab5f767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:55.657775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129686-c29e-49ab-934b-19112ab5f767 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:55.658061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129686-c29e-49ab-934b-19112ab5f767 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:55.658301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129686-c29e-49ab-934b-19112ab5f767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:55.669171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129686-c29e-49ab-934b-19112ab5f767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:55.670807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129686-c29e-49ab-934b-19112ab5f767 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:55.682326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1158/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:55 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:56.024840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea71559-acff-4fd1-861b-25a951d43384 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:56.093195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea71559-acff-4fd1-861b-25a951d43384 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:56.101664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:56.102920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:56.103348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:56.104176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] PATH_INFO: `/v3/users/5de63f9d38a644c593ec59b4758dd102` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:56.105444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de63f9d38a644c593ec59b4758dd102)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:56.107557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:56.165407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de63f9d38a644c593ec59b4758dd102'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:56.166246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de63f9d38a644c593ec59b4758dd102'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:56.173908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de63f9d38a644c593ec59b4758dd102'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:56.185157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de63f9d38a644c593ec59b4758dd102'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:56.202177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de63f9d38a644c593ec59b4758dd102'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:56.209926 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1149/2307] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:56 2022] DELETE /identity/v3/users/5de63f9d38a644c593ec59b4758dd102 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:56.215095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:56.215613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:56.215613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:56.245066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:56.246478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:56.260024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:56.329468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1159/2308] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:56.339793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:56.341348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:56.393692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:56.413395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fa1e-6393-4fa3-a0ed-7b59155874ca admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:56.420100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:56.421254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:56.421529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:56.422070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:56.438562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:56.439872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:56.452549 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1150/2309] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:57:57.137909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:57.142300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:57.170621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392a48c-43ed-4624-8f00-f6432bbe7fde None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:57.187999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:57.188952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:57.190753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:57.192260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] PATH_INFO: `/v3/projects/97ce5e66d63a4b329e23d823358de67c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:57.194916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ce5e66d63a4b329e23d823358de67c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:57.199573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:57.287092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ce5e66d63a4b329e23d823358de67c', 'request_id': 'req-308ec8e4-36e5-4736-bf68-935e9186f32a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.301855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ce5e66d63a4b329e23d823358de67c', 'request_id': 'req-308ec8e4-36e5-4736-bf68-935e9186f32a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.314218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ce5e66d63a4b329e23d823358de67c', 'request_id': 'req-308ec8e4-36e5-4736-bf68-935e9186f32a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.339035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] The token cache is being invalidate because project 97ce5e66d63a4b329e23d823358de67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:57.339333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.340775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.343239 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1160/2310] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:57 2022] DELETE /identity/v3/projects/97ce5e66d63a4b329e23d823358de67c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:57.349420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:57.385951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea28f6-de4c-48b1-a20f-47aaa6e7d7fe service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:57.462658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:57.463462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:57.463937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:57.464261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] PATH_INFO: `/v3/users/c533b81c8eda406e8bded7610880fb6b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:57.465615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c533b81c8eda406e8bded7610880fb6b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:57.467700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:57.551421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c533b81c8eda406e8bded7610880fb6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.552443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c533b81c8eda406e8bded7610880fb6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.563590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c533b81c8eda406e8bded7610880fb6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.576687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c533b81c8eda406e8bded7610880fb6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.599909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b89a4ce-bb41-4840-8715-373d7b98ee70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c533b81c8eda406e8bded7610880fb6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:57.608884 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1151/2311] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:57:57 2022] DELETE /identity/v3/users/c533b81c8eda406e8bded7610880fb6b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:58.055175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:58.056976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:58.074614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ec8e4-36e5-4736-bf68-935e9186f32a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:58.189711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:58.192003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.192592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.193713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] PATH_INFO: `/v3/projects/13ff23326cbd4f07bb68f0ddbe653320` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.195215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ff23326cbd4f07bb68f0ddbe653320)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:58.199840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:58.311924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:58.324353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ff23326cbd4f07bb68f0ddbe653320', 'request_id': 'req-dcb8da88-c944-4798-8d10-5777cda396ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:58.346509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ff23326cbd4f07bb68f0ddbe653320', 'request_id': 'req-dcb8da88-c944-4798-8d10-5777cda396ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:58.354104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ff23326cbd4f07bb68f0ddbe653320', 'request_id': 'req-dcb8da88-c944-4798-8d10-5777cda396ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:58.376821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] The token cache is being invalidate because project 13ff23326cbd4f07bb68f0ddbe653320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:57:58.377920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:58.378875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dcb8da88-c944-4798-8d10-5777cda396ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:57:58.381506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1161/2312] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:57:58 2022] DELETE /identity/v3/projects/13ff23326cbd4f07bb68f0ddbe653320 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:57:58.387150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96add6-57f7-41d0-92c7-2d859ce688a7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.388230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96add6-57f7-41d0-92c7-2d859ce688a7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.388735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96add6-57f7-41d0-92c7-2d859ce688a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.416563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-af96add6-57f7-41d0-92c7-2d859ce688a7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:58.419245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af96add6-57f7-41d0-92c7-2d859ce688a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:58.521153 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1152/2313] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:57:58.529237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee86a-c394-4090-848a-6643cf730d68 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.529766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee86a-c394-4090-848a-6643cf730d68 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.530384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee86a-c394-4090-848a-6643cf730d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.557596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cbcee86a-c394-4090-848a-6643cf730d68 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:58.559028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cbcee86a-c394-4090-848a-6643cf730d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:58.569290 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1162/2314] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:58.574491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96add6-57f7-41d0-92c7-2d859ce688a7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:58.580060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:58.587844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96add6-57f7-41d0-92c7-2d859ce688a7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:58.597662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:58.598306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.598617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.606706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.607734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:57:58.608054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:58.615202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:58.635429 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1153/2315] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:57:58.645783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcee86a-c394-4090-848a-6643cf730d68 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:58.669600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcee86a-c394-4090-848a-6643cf730d68 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:58.695551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:58.696205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.696474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.696833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.697478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:58.699416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:58.740608 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1163/2316] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:58 2022] POST /identity/v3/projects => generated 411 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:58.745456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:58.756955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ec8d5-91ce-4ac4-8699-91bae8746cdd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:58.832124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:58.833076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.833490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.833944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.834590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:58.837839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:58.884978 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1154/2317] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:58 2022] POST /identity/v3/users => generated 306 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:58.888099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:58.904748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb85b87-8283-4f07-9b65-008683a05a26 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:58.911418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:58.912054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.912323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.912591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.913342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:58.915264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:58.927109 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1164/2318] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:58.931319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:58.944563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890af4-6321-4867-82ee-344b90e40413 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:58.960695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777641e-37d7-414f-acdf-32e557f9f948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:58.960695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777641e-37d7-414f-acdf-32e557f9f948 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:58.960695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777641e-37d7-414f-acdf-32e557f9f948 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:58.960695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777641e-37d7-414f-acdf-32e557f9f948 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:58.961724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777641e-37d7-414f-acdf-32e557f9f948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:58.961724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777641e-37d7-414f-acdf-32e557f9f948 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:58.966561 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1155/2319] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:58.982235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:58.987823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54870f-1138-4a6c-9caa-6ec526bea69a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:59.009966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:59.011052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:59.011599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:59.012616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] PATH_INFO: `/v3/projects/5e69aef703eb40a694c1f6f936dd5e05/users/8bc909e7c4214abd83b1e0bd9659e430/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:59.030274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e69aef703eb40a694c1f6f936dd5e05, user_id=8bc909e7c4214abd83b1e0bd9659e430, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:59.032961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:59.059127 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1165/2320] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:58 2022] PUT /identity/v3/projects/5e69aef703eb40a694c1f6f936dd5e05/users/8bc909e7c4214abd83b1e0bd9659e430/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:59.075137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03af7d-c8d9-421d-8c3b-e6def1cf26e6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:59.075794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03af7d-c8d9-421d-8c3b-e6def1cf26e6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:59.076501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03af7d-c8d9-421d-8c3b-e6def1cf26e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:59.126021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4a03af7d-c8d9-421d-8c3b-e6def1cf26e6 None None] MFA Rules not processed for user `8bc909e7c4214abd83b1e0bd9659e430`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:57:59.129247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4a03af7d-c8d9-421d-8c3b-e6def1cf26e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:57:59.272810 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1156/2321] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:57:59 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:57:59.278104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:59.386044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fd5b0-63ec-424f-8acf-f1d194925809 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:59.391705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:59.392471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:59.393021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:59.393414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:59.396359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:59.398539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:59.455715 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1166/2322] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:57:59 2022] POST /identity/v3/projects => generated 411 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:59.460785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03af7d-c8d9-421d-8c3b-e6def1cf26e6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:59.481706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03af7d-c8d9-421d-8c3b-e6def1cf26e6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:59.597089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:59.641453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:59.642166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:59.642443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:59.642718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:59.643430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:59.653047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:59.720593 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1157/2323] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:57:59 2022] POST /identity/v3/users => generated 312 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:57:59.721634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:59.747486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdeffd-c9ab-4e28-8078-5f841144e205 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:59.752842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:59.753478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:59.753771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:59.754025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:59.754679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:59.756308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:59.778127 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1167/2324] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:57:59.779446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:59.803380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f0ce-601c-4974-9147-187ec7146eb7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:59.809294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:57:59.809989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:57:59.811432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:57:59.812785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] PATH_INFO: `/v3/projects/982ca491585d4e288c4e330ad9d157b5/users/4fd3b1cd2c534d3e973ba2534e43bf92/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:57:59.851737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=982ca491585d4e288c4e330ad9d157b5, user_id=4fd3b1cd2c534d3e973ba2534e43bf92, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:57:59.854468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:57:59.883221 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1158/2325] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:57:59 2022] PUT /identity/v3/projects/982ca491585d4e288c4e330ad9d157b5/users/4fd3b1cd2c534d3e973ba2534e43bf92/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:57:59.887671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:57:59.900842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d0cf-edfc-4d99-8563-5023bac07999 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:57:59.963082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:00.029157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d683df3-7521-4ecf-a8d9-92367c8e0395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:00.030040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d683df3-7521-4ecf-a8d9-92367c8e0395 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:00.030322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d683df3-7521-4ecf-a8d9-92367c8e0395 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:00.030602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d683df3-7521-4ecf-a8d9-92367c8e0395 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:00.031617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d683df3-7521-4ecf-a8d9-92367c8e0395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:00.033790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d683df3-7521-4ecf-a8d9-92367c8e0395 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:00.048058 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1168/2326] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:00.052882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:00.067745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8894ddc-b527-4b9b-a07f-2be976c16727 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:00.074561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:00.075560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:00.076055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:00.076440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] PATH_INFO: `/v3/projects/982ca491585d4e288c4e330ad9d157b5/users/4fd3b1cd2c534d3e973ba2534e43bf92/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:00.079758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=982ca491585d4e288c4e330ad9d157b5, user_id=4fd3b1cd2c534d3e973ba2534e43bf92, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:00.082646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:00.103699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1159/2327] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:00 2022] PUT /identity/v3/projects/982ca491585d4e288c4e330ad9d157b5/users/4fd3b1cd2c534d3e973ba2534e43bf92/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:00.116125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:00.116956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:00.117773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:00.161257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] MFA Rules not processed for user `4fd3b1cd2c534d3e973ba2534e43bf92`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:00.163911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:00.266140 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1169/2328] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:00 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:00.544249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:00.546257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:00.649906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e611b-63ac-4aa4-a3bd-92864fbce428 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:00.656517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:00.657299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:00.657605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:00.657975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:00.669172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:00.670888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:00.683339 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1160/2329] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:00 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:00.762198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:00.771505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:00.811343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94994301-8bde-492e-a4a4-929451b90b6b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:00.819557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbab4b7-63ea-46fd-8887-0a391cce0b41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:00.820199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbab4b7-63ea-46fd-8887-0a391cce0b41 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:00.821736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbab4b7-63ea-46fd-8887-0a391cce0b41 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:00.821736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbab4b7-63ea-46fd-8887-0a391cce0b41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:00.898114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbab4b7-63ea-46fd-8887-0a391cce0b41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:00.900359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbab4b7-63ea-46fd-8887-0a391cce0b41 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:00.913400 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1170/2330] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:00 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:00.960349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:01.055420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c472494-9096-48e7-91c6-3d557131fc51 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:01.060504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:01.061200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:01.061614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:01.062014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] PATH_INFO: `/v3/users/8bc909e7c4214abd83b1e0bd9659e430` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:01.063206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc909e7c4214abd83b1e0bd9659e430)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:01.065160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:01.112271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:01.125292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc909e7c4214abd83b1e0bd9659e430'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:01.125823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc909e7c4214abd83b1e0bd9659e430'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:01.135228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc909e7c4214abd83b1e0bd9659e430'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:01.144442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc909e7c4214abd83b1e0bd9659e430'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:01.159807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc909e7c4214abd83b1e0bd9659e430'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:01.167359 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1161/2331] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:00 2022] DELETE /identity/v3/users/8bc909e7c4214abd83b1e0bd9659e430 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:01.172022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:01.172270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:01.172574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:01.192203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:01.193480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:01.251418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:01.262392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1171/2332] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:01.272272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:01.273785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:01.335625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52607a-c90b-4290-bece-2a15ffbd83ab admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:01.340183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:01.340849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:01.341131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:01.341408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:01.351905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:01.353401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:01.364015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1162/2333] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:01.833734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:01.835787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:01.899781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:01.901330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:01.924058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323fe2-aaaa-4285-96b2-87757caa03dd service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:01.937480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:01.938562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:01.939077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:01.939561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] PATH_INFO: `/v3/projects/5e69aef703eb40a694c1f6f936dd5e05` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:01.941985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e69aef703eb40a694c1f6f936dd5e05)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:01.945101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:02.012647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43694ca-75da-4f41-8495-c5684c33fff7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:02.025745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:02.026038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:02.026038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:02.026316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:02.029061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:02.053911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e69aef703eb40a694c1f6f936dd5e05', 'request_id': 'req-f438c83d-54a9-4985-93b5-6e038017df64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.073333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e69aef703eb40a694c1f6f936dd5e05', 'request_id': 'req-f438c83d-54a9-4985-93b5-6e038017df64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.110938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e69aef703eb40a694c1f6f936dd5e05', 'request_id': 'req-f438c83d-54a9-4985-93b5-6e038017df64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.115041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:02.118962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:02.133095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] The token cache is being invalidate because project 5e69aef703eb40a694c1f6f936dd5e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:02.134076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.134169 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1172/2334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:02.134674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f438c83d-54a9-4985-93b5-6e038017df64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.136634 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1163/2335] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:01 2022] DELETE /identity/v3/projects/5e69aef703eb40a694c1f6f936dd5e05 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:02.140540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:02.152439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6c63a-04e1-40b7-b186-7c641861b0d1 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:02.242911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:02.243723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:02.244008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:02.244282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] PATH_INFO: `/v3/users/4fd3b1cd2c534d3e973ba2534e43bf92` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:02.245545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd3b1cd2c534d3e973ba2534e43bf92)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:02.248644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:02.315864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:02.342297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd3b1cd2c534d3e973ba2534e43bf92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.344325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd3b1cd2c534d3e973ba2534e43bf92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.356745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd3b1cd2c534d3e973ba2534e43bf92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.370699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd3b1cd2c534d3e973ba2534e43bf92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.371515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd3b1cd2c534d3e973ba2534e43bf92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:02.403784 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1173/2336] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:02 2022] DELETE /identity/v3/users/4fd3b1cd2c534d3e973ba2534e43bf92 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:02.595645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb1e96-ab11-4c82-9913-8edf77594a79 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:02.596050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb1e96-ab11-4c82-9913-8edf77594a79 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:02.596402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb1e96-ab11-4c82-9913-8edf77594a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:02.623189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5aeb1e96-ab11-4c82-9913-8edf77594a79 None None] MFA Rules not processed for user `4319ec4b90e1481b8be19e8f9dde1ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:02.624644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5aeb1e96-ab11-4c82-9913-8edf77594a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:02.668201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:02.700651 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1164/2337] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:03.512689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:03.517514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:03.662586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a3166-2566-4698-9fe8-714432d1551a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:03.669920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:03.670797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:03.671218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:03.671658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] PATH_INFO: `/v3/projects/982ca491585d4e288c4e330ad9d157b5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:03.673988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] RBAC: Authorizing `identity:delete_project(project_id=982ca491585d4e288c4e330ad9d157b5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:03.676745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:03.748287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '982ca491585d4e288c4e330ad9d157b5', 'request_id': 'req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:03.761552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '982ca491585d4e288c4e330ad9d157b5', 'request_id': 'req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:03.771514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '982ca491585d4e288c4e330ad9d157b5', 'request_id': 'req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:03.793952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] The token cache is being invalidate because project 982ca491585d4e288c4e330ad9d157b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:03.794396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:03.795100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8112e3ea-84ea-4dc8-9d60-fea4d89dd90c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:03.798644 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1174/2338] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:03 2022] DELETE /identity/v3/projects/982ca491585d4e288c4e330ad9d157b5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:03.808082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887dae-06fb-4e3d-9c29-9564203da78f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:03.808539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887dae-06fb-4e3d-9c29-9564203da78f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:03.808733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887dae-06fb-4e3d-9c29-9564203da78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:03.836903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-75887dae-06fb-4e3d-9c29-9564203da78f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:03.839558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-75887dae-06fb-4e3d-9c29-9564203da78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:03.928013 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1165/2339] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:03.937297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08357b05-8d33-474c-9b39-0fe388e351f5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:03.937630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08357b05-8d33-474c-9b39-0fe388e351f5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:03.937967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08357b05-8d33-474c-9b39-0fe388e351f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:03.964259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-08357b05-8d33-474c-9b39-0fe388e351f5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:03.966394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08357b05-8d33-474c-9b39-0fe388e351f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:03.979987 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1175/2340] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:03.980805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887dae-06fb-4e3d-9c29-9564203da78f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:03.994831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887dae-06fb-4e3d-9c29-9564203da78f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:03.999842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:04.000467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.000782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.001056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.001718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:04.002020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:04.003787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:04.019007 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1166/2341] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:58:04.023132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08357b05-8d33-474c-9b39-0fe388e351f5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:04.036322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08357b05-8d33-474c-9b39-0fe388e351f5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:04.041430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:04.043330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.043330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.043330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.043662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:04.045886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:04.085725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1176/2342] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:04 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:04.087697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:04.105246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dfc96-2324-423a-8f39-d6d8cf1998b5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:04.179993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:04.180856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.181263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.181614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.182343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:04.184735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:04.214360 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1167/2343] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:04 2022] POST /identity/v3/users => generated 302 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:04.219396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:04.232298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bdf80-ec1a-41e0-ab80-42f515262505 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:04.237489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:04.238117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.238398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.238676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.239339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:04.241043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:04.247489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:04.254619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1177/2344] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:04.260271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:04.275079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180694c5-8354-43af-bc4f-54479b07931f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:04.280416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d370f2-a55a-4bfd-adc2-6255cceae7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:04.281082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d370f2-a55a-4bfd-adc2-6255cceae7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.281431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d370f2-a55a-4bfd-adc2-6255cceae7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.281776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d370f2-a55a-4bfd-adc2-6255cceae7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.282872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d370f2-a55a-4bfd-adc2-6255cceae7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:04.284193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d370f2-a55a-4bfd-adc2-6255cceae7f3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:04.300729 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1168/2345] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:04.305288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:04.325836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fcae9-93c1-40c8-96f1-b54fd04f54a0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:04.331668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:04.332550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.332890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.333206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] PATH_INFO: `/v3/projects/b07a770a637643a0bdf3fa219a46b756/users/d67531a4fd4040a5b824e805c685eab0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.363193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07a770a637643a0bdf3fa219a46b756, user_id=d67531a4fd4040a5b824e805c685eab0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:04.368025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:04.388759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1178/2346] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:04 2022] PUT /identity/v3/projects/b07a770a637643a0bdf3fa219a46b756/users/d67531a4fd4040a5b824e805c685eab0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:04.391510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.391893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.392233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.451486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:04.451486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:04.519021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:04.543590 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1169/2347] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:04.558287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:04.562247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:04.713058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af049a7c-f4b0-4b70-877f-1b1f01aed78e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:04.720743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f3f8a-6acf-4702-a735-e2c1539d431f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:04.721387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f3f8a-6acf-4702-a735-e2c1539d431f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:04.721664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f3f8a-6acf-4702-a735-e2c1539d431f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:04.721937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f3f8a-6acf-4702-a735-e2c1539d431f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:04.761867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f3f8a-6acf-4702-a735-e2c1539d431f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:04.768773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f3f8a-6acf-4702-a735-e2c1539d431f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:04.807381 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1179/2348] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:08.739169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:08.742142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:08.760359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a0be-50b9-45e7-b5f3-ba7ad3950ff0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:08.771172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:08.772249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:08.772797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:08.773299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] PATH_INFO: `/v3/users/7ba3c51770094c4ea7fec1e597ae13a0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:08.775180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba3c51770094c4ea7fec1e597ae13a0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:08.780311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:08.864852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba3c51770094c4ea7fec1e597ae13a0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:08.865678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba3c51770094c4ea7fec1e597ae13a0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:08.887556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba3c51770094c4ea7fec1e597ae13a0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:08.906247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba3c51770094c4ea7fec1e597ae13a0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:08.929202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba3c51770094c4ea7fec1e597ae13a0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:08.950005 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1170/2349] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:08 2022] DELETE /identity/v3/users/7ba3c51770094c4ea7fec1e597ae13a0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:08.952145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:08.952712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:08.952712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:08.995087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:09.001925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:09.209934 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1180/2350] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:09.222221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:09.223840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:09.379659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11413c19-681a-4c52-b225-bada1a16c2be admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:09.394206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:09.395266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:09.395648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:09.396019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:09.412773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:09.414512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:09.428903 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1171/2351] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:09.771097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:09.772849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:09.796603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ecf96-1e67-4f1c-a922-10df795fb0a7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:09.812851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:09.813699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:09.813977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:09.814250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] PATH_INFO: `/v3/projects/3132070d235941498e3f64d9d7075f6b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:09.815630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3132070d235941498e3f64d9d7075f6b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:09.817535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:09.948557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3132070d235941498e3f64d9d7075f6b', 'request_id': 'req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:09.967010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3132070d235941498e3f64d9d7075f6b', 'request_id': 'req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:09.974684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3132070d235941498e3f64d9d7075f6b', 'request_id': 'req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:09.999661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] The token cache is being invalidate because project 3132070d235941498e3f64d9d7075f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:10.000093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:10.002089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:10.004771 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1181/2352] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:09 2022] DELETE /identity/v3/projects/3132070d235941498e3f64d9d7075f6b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:10.009334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:10.027525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f7e5c-4437-4edc-8a9c-99c5b80c2478 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:10.181384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:10.182122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:10.182315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:10.182581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] PATH_INFO: `/v3/users/324bddae7f37458e9217889fdf43c6eb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:10.183749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=324bddae7f37458e9217889fdf43c6eb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:10.190207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:10.206990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:10.312005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324bddae7f37458e9217889fdf43c6eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:10.316074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324bddae7f37458e9217889fdf43c6eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:10.342048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324bddae7f37458e9217889fdf43c6eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:10.389300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324bddae7f37458e9217889fdf43c6eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:10.468879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17d43257-18d7-45c3-86c0-42eae40e12ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324bddae7f37458e9217889fdf43c6eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:10.510351 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1172/2353] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:10 2022] DELETE /identity/v3/users/324bddae7f37458e9217889fdf43c6eb => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:12.138500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:12.142917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:12.157776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:12.170292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb33c9-7218-4eee-ba66-4919d2e7d2a2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:12.276011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:12.277440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:12.277440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:12.277440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] PATH_INFO: `/v3/users/d7d3aef1d4da47b2ab5454c3eb0346a6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:12.279763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d3aef1d4da47b2ab5454c3eb0346a6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:12.283354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:12.380900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d3aef1d4da47b2ab5454c3eb0346a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:12.381982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d3aef1d4da47b2ab5454c3eb0346a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:12.406927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d3aef1d4da47b2ab5454c3eb0346a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:12.424108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d3aef1d4da47b2ab5454c3eb0346a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:12.425342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d3aef1d4da47b2ab5454c3eb0346a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:12.459056 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1182/2354] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:12 2022] DELETE /identity/v3/users/d7d3aef1d4da47b2ab5454c3eb0346a6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:12.460251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:12.460797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:12.460947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:12.502654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:12.504253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:12.709035 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1173/2355] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:12.737230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:12.739278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:12.890216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee30e-24ce-4ced-a17f-1996643cdce6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:12.898394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32993a36-39c5-4a59-80fe-f07c320291aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:12.899114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32993a36-39c5-4a59-80fe-f07c320291aa service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:12.899487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32993a36-39c5-4a59-80fe-f07c320291aa service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:12.899760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32993a36-39c5-4a59-80fe-f07c320291aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:12.925638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32993a36-39c5-4a59-80fe-f07c320291aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:12.927140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32993a36-39c5-4a59-80fe-f07c320291aa service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:12.948078 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1183/2356] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:13.026970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:13.028536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:13.057384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5e210-d63c-476a-9fe1-8c588aa7959e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:13.068124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:13.069225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:13.069657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:13.070070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] PATH_INFO: `/v3/projects/d0ffaa13af1744fd92951b2207c6bea2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:13.074162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ffaa13af1744fd92951b2207c6bea2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:13.080859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:13.266262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ffaa13af1744fd92951b2207c6bea2', 'request_id': 'req-b599be64-7b29-4850-94e8-e7c918a1bafe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:13.276482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ffaa13af1744fd92951b2207c6bea2', 'request_id': 'req-b599be64-7b29-4850-94e8-e7c918a1bafe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:13.299860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ffaa13af1744fd92951b2207c6bea2', 'request_id': 'req-b599be64-7b29-4850-94e8-e7c918a1bafe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:13.342338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] The token cache is being invalidate because project d0ffaa13af1744fd92951b2207c6bea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:13.342645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:13.343865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b599be64-7b29-4850-94e8-e7c918a1bafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:13.348477 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1174/2357] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:13 2022] DELETE /identity/v3/projects/d0ffaa13af1744fd92951b2207c6bea2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:13.350509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c0a19-ea8b-408d-9114-e6da8c0de8cc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:13.350963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c0a19-ea8b-408d-9114-e6da8c0de8cc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:13.350963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c0a19-ea8b-408d-9114-e6da8c0de8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:13.406937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1f1c0a19-ea8b-408d-9114-e6da8c0de8cc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:13.408256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1f1c0a19-ea8b-408d-9114-e6da8c0de8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:13.474771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:13.593362 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1184/2358] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:13.607831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62622-f78c-4b3a-9737-e5e87f61077e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:13.608441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62622-f78c-4b3a-9737-e5e87f61077e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:13.627890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee62622-f78c-4b3a-9737-e5e87f61077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:13.658304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4ee62622-f78c-4b3a-9737-e5e87f61077e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:13.660091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ee62622-f78c-4b3a-9737-e5e87f61077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:13.670158 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1175/2359] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:13.678870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c0a19-ea8b-408d-9114-e6da8c0de8cc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:13.717152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c0a19-ea8b-408d-9114-e6da8c0de8cc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:13.723294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:13.724612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:13.728155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:13.728765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:13.729694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:13.730359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:13.733012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:13.759418 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1185/2360] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:13.770635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62622-f78c-4b3a-9737-e5e87f61077e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:13.789434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:13.803111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee62622-f78c-4b3a-9737-e5e87f61077e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:13.863712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:13.863712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:13.863712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:13.863712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:13.864874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:13.870736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:14.027177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1176/2361] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:13 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:14.033637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:14.050809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3b0b3-36c1-40bd-bdf4-0247f6dce1a1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:14.070335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:14.074980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:14.162887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:14.163177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:14.163424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:14.163702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:14.164425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:14.166719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:14.267477 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1186/2362] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:14 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:14.272476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:14.291188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6cdf-b1d3-4fcf-b9c3-be45627613f9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:14.305760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bff20-7758-4e1a-bb4e-2b3f451efae9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:14.317435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:14.318115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:14.318397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:14.318675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] PATH_INFO: `/v3/projects/b33d41b411144dbbbdb012e694857d27` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:14.324097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33d41b411144dbbbdb012e694857d27)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:14.326052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:14.330022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:14.331831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:14.332771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:14.333904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:14.338109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:14.365634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:14.412806 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1187/2363] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:14.417478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:14.436395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33d41b411144dbbbdb012e694857d27', 'request_id': 'req-6071549d-59fc-48a1-b867-e595088ac174'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.448694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33d41b411144dbbbdb012e694857d27', 'request_id': 'req-6071549d-59fc-48a1-b867-e595088ac174'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.455298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723bd3a-302a-47ff-852a-4ac1805af689 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:14.464270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33d41b411144dbbbdb012e694857d27', 'request_id': 'req-6071549d-59fc-48a1-b867-e595088ac174'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.482443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] The token cache is being invalidate because project b33d41b411144dbbbdb012e694857d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:14.482748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.484960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.487788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1177/2364] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:14 2022] DELETE /identity/v3/projects/b33d41b411144dbbbdb012e694857d27 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:14.491626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:14.497294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:14.620483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071549d-59fc-48a1-b867-e595088ac174 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:14.629349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:14.630277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:14.630722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:14.631154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:14.631620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:14.632289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:14.633665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:14.634412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:14.635921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:14.636344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] PATH_INFO: `/v3/users/f1c3ed62097a4e89bf360a966c764762` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:14.639878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c3ed62097a4e89bf360a966c764762)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:14.644304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:14.660788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1188/2365] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:58:14.661949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:14.664345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:14.674303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:14.685331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b61332-67e7-4abd-b554-c4427859a9c0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:14.705097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6996d5f-d6af-44dc-8b4c-56ba4a06cd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:14.708133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6996d5f-d6af-44dc-8b4c-56ba4a06cd38 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:14.708133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6996d5f-d6af-44dc-8b4c-56ba4a06cd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:14.708626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6996d5f-d6af-44dc-8b4c-56ba4a06cd38 admin admin] PATH_INFO: `/v3/projects/25d46074b23b4a0c802e05437436fe3a/users/b2288601ff7742ab8cfddd8e3c254982/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:14.747446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c3ed62097a4e89bf360a966c764762'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.748627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c3ed62097a4e89bf360a966c764762'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.756311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6996d5f-d6af-44dc-8b4c-56ba4a06cd38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d46074b23b4a0c802e05437436fe3a, user_id=b2288601ff7742ab8cfddd8e3c254982, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:14.760108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6996d5f-d6af-44dc-8b4c-56ba4a06cd38 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:14.762425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c3ed62097a4e89bf360a966c764762'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.780476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c3ed62097a4e89bf360a966c764762'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.801164 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1189/2366] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:14 2022] PUT /identity/v3/projects/25d46074b23b4a0c802e05437436fe3a/users/b2288601ff7742ab8cfddd8e3c254982/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:14.814159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489855e-6f5b-426a-926d-79b4256e262c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:14.814534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489855e-6f5b-426a-926d-79b4256e262c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:14.815043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489855e-6f5b-426a-926d-79b4256e262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:14.823862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c3ed62097a4e89bf360a966c764762'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:14.835575 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1178/2367] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:14 2022] DELETE /identity/v3/users/f1c3ed62097a4e89bf360a966c764762 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:14.841163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:14.874225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2489855e-6f5b-426a-926d-79b4256e262c None None] MFA Rules not processed for user `b2288601ff7742ab8cfddd8e3c254982`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:14.875593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2489855e-6f5b-426a-926d-79b4256e262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:14.997260 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1190/2368] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:14.998708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:15.000686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:15.178928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c384d-7724-4a1e-a5bb-f10f49e48163 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:15.208240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:15.208240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.208550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.211615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.212465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:15.214967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:15.267905 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1179/2369] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:14 2022] POST /identity/v3/projects => generated 377 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:15.271771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489855e-6f5b-426a-926d-79b4256e262c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:15.289781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489855e-6f5b-426a-926d-79b4256e262c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:15.378880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:15.379643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.379996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.380387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.382006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:15.385545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:15.428160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:15.430782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:15.451417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215626-7c4b-4b61-85d2-b7b7220d0b22 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:15.455533 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1191/2370] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:15 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:15.462292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:15.471614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:15.474093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.474692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.475192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] PATH_INFO: `/v3/projects/06e7347f7bbc45ce8de8e29de5255fa9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.479505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e7347f7bbc45ce8de8e29de5255fa9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:15.481995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:15.483266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ff1f9-cc29-411a-be11-c654648948db admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:15.488823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:15.493513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.493953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.494242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.495125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:15.498380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:15.514824 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1192/2371] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:58:15.517000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:15.536556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53f36e-4679-447b-af0c-f4a9aadbe00d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:15.545327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:15.548590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.548590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.549144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] PATH_INFO: `/v3/projects/9a2445b438a34ba0bf66223a09dcdde5/users/e46ca6451816417c8f19989cca9a3ed4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.593556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2445b438a34ba0bf66223a09dcdde5, user_id=e46ca6451816417c8f19989cca9a3ed4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:15.600682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:15.641296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e7347f7bbc45ce8de8e29de5255fa9', 'request_id': 'req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:15.644894 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1193/2372] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:15 2022] PUT /identity/v3/projects/9a2445b438a34ba0bf66223a09dcdde5/users/e46ca6451816417c8f19989cca9a3ed4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:15.646592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:15.653490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e7347f7bbc45ce8de8e29de5255fa9', 'request_id': 'req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:15.683044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e7347f7bbc45ce8de8e29de5255fa9', 'request_id': 'req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:15.695523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:15.706710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] The token cache is being invalidate because project 06e7347f7bbc45ce8de8e29de5255fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:15.707014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:15.709292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1d45ed02-4e8d-4f13-90d9-241b643aa2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:15.713608 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1180/2373] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:15 2022] DELETE /identity/v3/projects/06e7347f7bbc45ce8de8e29de5255fa9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:15.725105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.728649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.728649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.763268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:15.765717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:15.784309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:15.814979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da23cac-910e-4bd4-91fd-e81057ef4726 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:15.821404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8abc4-3dd7-405c-b02f-dd7076852907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:15.824312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8abc4-3dd7-405c-b02f-dd7076852907 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.824710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8abc4-3dd7-405c-b02f-dd7076852907 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.825081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8abc4-3dd7-405c-b02f-dd7076852907 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.826210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8abc4-3dd7-405c-b02f-dd7076852907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:15.828697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8abc4-3dd7-405c-b02f-dd7076852907 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:15.865817 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1194/2374] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:15.868234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6577-ff8f-4271-b4c7-34b8f2398dc1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.868709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6577-ff8f-4271-b4c7-34b8f2398dc1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.869163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6577-ff8f-4271-b4c7-34b8f2398dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.927023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-539a6577-ff8f-4271-b4c7-34b8f2398dc1 None None] MFA Rules not processed for user `d67531a4fd4040a5b824e805c685eab0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:15.930560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-539a6577-ff8f-4271-b4c7-34b8f2398dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:15.966323 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1181/2375] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:15.966323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:15.966323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:15.976374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9097-adf4-4b15-ab99-371e2d55a527 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:15.990946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144b61-7cba-4a79-914d-9521687ba33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:15.991838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b61-7cba-4a79-914d-9521687ba33d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:15.992451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b61-7cba-4a79-914d-9521687ba33d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:15.992857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b61-7cba-4a79-914d-9521687ba33d admin admin] PATH_INFO: `/v3/projects/9a2445b438a34ba0bf66223a09dcdde5/users/e46ca6451816417c8f19989cca9a3ed4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:15.997402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144b61-7cba-4a79-914d-9521687ba33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2445b438a34ba0bf66223a09dcdde5, user_id=e46ca6451816417c8f19989cca9a3ed4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.001616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144b61-7cba-4a79-914d-9521687ba33d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.041203 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1182/2376] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:15 2022] PUT /identity/v3/projects/9a2445b438a34ba0bf66223a09dcdde5/users/e46ca6451816417c8f19989cca9a3ed4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:16.045908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.046382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.046782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.104626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:16.119859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:16.186144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1195/2377] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:16.191779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3e2a9-304a-4612-9168-31636c05fab2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.192101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3e2a9-304a-4612-9168-31636c05fab2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.192428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3e2a9-304a-4612-9168-31636c05fab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.245938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7bf3e2a9-304a-4612-9168-31636c05fab2 None None] MFA Rules not processed for user `e46ca6451816417c8f19989cca9a3ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:16.252468 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1183/2378] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:16.252645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7bf3e2a9-304a-4612-9168-31636c05fab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:16.252811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.252811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:16.274006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e16e-7adf-4f84-a55b-2bd85dfb89a7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.279767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.280634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.280872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.281155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.281924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.284010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.336505 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1184/2379] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:16 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:16.337825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.401174 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1196/2380] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:16.402235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3e2a9-304a-4612-9168-31636c05fab2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.416721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3e2a9-304a-4612-9168-31636c05fab2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.449165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa861e9-91cb-4c54-b194-86f52bb8bac0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.468702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.468702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.468702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.469505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.477546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:16.478152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.480939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.500709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:16.517256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1185/2381] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:16.517256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.521206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:16.526617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:16.540473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.541606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.544534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.545072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.545947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.548581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.592706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1197/2382] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:16 2022] POST /identity/v3/users => generated 309 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:16.594504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.612871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac30227-12de-45fe-ab39-5c3ad0b92b8b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.620556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.621434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.622248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.623246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.624050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.628785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.691805 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1198/2383] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:16 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:16.692734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.693477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbff05-a95a-4e9b-a6a0-d8d4db876cbe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.822336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.823214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.823502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.823779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.823918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9658a8e-7692-4306-80db-82aba3f2923d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.830555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.831182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.831616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.831887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.832553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.834265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.849337 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1199/2384] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:16.849337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.861592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d04a3-1024-4b3c-94ec-dde06e85009f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.868041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.869295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.869583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.869856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.870503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.872480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.922848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1200/2385] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:16 2022] POST /identity/v3/users => generated 287 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:16.926229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:16.944496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d4141-a125-424e-927b-5fea02c9c7d7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:16.955851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:16.956596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:16.956905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:16.957539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] PATH_INFO: `/v3/projects/00b477951d834dca9ce99392b85d9c16/users/f353d5a0cabf4c24ac7ee5c9e7f44007/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:16.968986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.970956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:16.979404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b477951d834dca9ce99392b85d9c16, user_id=f353d5a0cabf4c24ac7ee5c9e7f44007, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:16.982802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:17.001821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1186/2386] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:17.002712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:17.007252 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1201/2387] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:16 2022] PUT /identity/v3/projects/00b477951d834dca9ce99392b85d9c16/users/f353d5a0cabf4c24ac7ee5c9e7f44007/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:17.011299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:17.026969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0865d-9bdb-42a6-9120-b04ebb62ca88 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:17.096357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:17.124336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6df4c9-26e8-4b1e-9564-750fcced780f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:17.130021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:17.130613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:17.130939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:17.131217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:17.131866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:17.134601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:17.151482 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1202/2388] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:17.166706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:17.175167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:17.182039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:17.187307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55899a26-78ea-4eba-8097-0cd5a68f1346 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:17.189257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:17.189891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:17.192764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:17.193277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:17.193389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:17.193592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] PATH_INFO: `/v3/projects/00b477951d834dca9ce99392b85d9c16/users/f353d5a0cabf4c24ac7ee5c9e7f44007/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:17.193941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:17.195748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b477951d834dca9ce99392b85d9c16, user_id=f353d5a0cabf4c24ac7ee5c9e7f44007, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:17.197954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:17.198891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:17.216647 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1203/2389] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:17 2022] PUT /identity/v3/projects/00b477951d834dca9ce99392b85d9c16/users/f353d5a0cabf4c24ac7ee5c9e7f44007/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:17.224203 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1187/2390] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:17.241009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:17.253307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:17.369479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec00e0-a385-4d4b-80c6-3c3a6622a92e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:17.386133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496e9ad-51f8-40e5-8f89-8d4a53092191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:17.386762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e9ad-51f8-40e5-8f89-8d4a53092191 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:17.387111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e9ad-51f8-40e5-8f89-8d4a53092191 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:17.387720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e9ad-51f8-40e5-8f89-8d4a53092191 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:17.389712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496e9ad-51f8-40e5-8f89-8d4a53092191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:17.393667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496e9ad-51f8-40e5-8f89-8d4a53092191 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:17.433357 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1204/2391] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:17.437568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:17.453116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21310049-06c8-445e-9ec3-24556e97d0ba admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:17.483160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:17.486405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:17.486405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:17.486405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] PATH_INFO: `/v3/projects/7476bff780f0442e9be3b2c4ff4b7e41/users/0320d122f53c435588a69baafb9bcf98/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:17.524898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7476bff780f0442e9be3b2c4ff4b7e41, user_id=0320d122f53c435588a69baafb9bcf98, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:17.527963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:17.561872 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1188/2392] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:17 2022] PUT /identity/v3/projects/7476bff780f0442e9be3b2c4ff4b7e41/users/0320d122f53c435588a69baafb9bcf98/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:17.563370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:17.563661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:17.564767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:17.608222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:17.612836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:17.791640 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1205/2393] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:17.804435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:17.806830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:17.903538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1351b9-2a80-48b5-b795-dd258d0af5f0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:17.908733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e974ec-a1fc-4f90-ab8a-612930de952f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:17.909362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e974ec-a1fc-4f90-ab8a-612930de952f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:17.909638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e974ec-a1fc-4f90-ab8a-612930de952f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:17.909905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e974ec-a1fc-4f90-ab8a-612930de952f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:17.935103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e974ec-a1fc-4f90-ab8a-612930de952f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:17.940170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e974ec-a1fc-4f90-ab8a-612930de952f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:17.973241 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1189/2394] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:18.844426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:18.846117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:18.997904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2e4d7-1563-432f-b397-f5e1ff461946 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:19.015305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:19.016688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:19.017271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:19.017746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:19.020341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:19.100099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:19.156359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:19.159638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:19.204599 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1206/2395] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:20.846020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d37bdb-739c-4d04-ae88-fa6299bd50a7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:20.847643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d37bdb-739c-4d04-ae88-fa6299bd50a7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:20.848108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d37bdb-739c-4d04-ae88-fa6299bd50a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:20.924424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-34d37bdb-739c-4d04-ae88-fa6299bd50a7 None None] MFA Rules not processed for user `00423eb1e4c24763ae0a546225ac1dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:20.936206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-34d37bdb-739c-4d04-ae88-fa6299bd50a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:21.176775 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1190/2396] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:25.521969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:25.548618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c326d-c3e6-49fc-8aa6-04187c881be0 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:25.558519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:25.559539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:25.560157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:25.560582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] PATH_INFO: `/v3/users/b2288601ff7742ab8cfddd8e3c254982` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:25.563042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2288601ff7742ab8cfddd8e3c254982)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:25.572875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:25.794376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2288601ff7742ab8cfddd8e3c254982'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:25.795222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2288601ff7742ab8cfddd8e3c254982'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:25.816858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2288601ff7742ab8cfddd8e3c254982'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:25.832187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2288601ff7742ab8cfddd8e3c254982'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:25.833236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2288601ff7742ab8cfddd8e3c254982'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:25.885869 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1207/2397] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:25 2022] DELETE /identity/v3/users/b2288601ff7742ab8cfddd8e3c254982 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:25.890715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:25.891361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:25.891361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:25.944471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:25.950284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:26.032845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:26.056298 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1191/2398] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:26.071470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:26.074466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:26.252154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80dab2-d795-4183-a586-f4122a804ee6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:26.268826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:26.269588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:26.269983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:26.270415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:26.303000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:26.308757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:26.327568 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1208/2399] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:28.379029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:28.382573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:28.450896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86248de8-5bc4-41ea-9e07-dd459aac43b4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:28.487082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:28.487792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:28.488063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:28.488337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] PATH_INFO: `/v3/projects/25d46074b23b4a0c802e05437436fe3a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:28.490756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d46074b23b4a0c802e05437436fe3a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:28.493721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:28.624391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d46074b23b4a0c802e05437436fe3a', 'request_id': 'req-7e525571-0d9e-4e30-9bd2-2e1bba7772df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.642055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d46074b23b4a0c802e05437436fe3a', 'request_id': 'req-7e525571-0d9e-4e30-9bd2-2e1bba7772df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.655535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d46074b23b4a0c802e05437436fe3a', 'request_id': 'req-7e525571-0d9e-4e30-9bd2-2e1bba7772df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.673765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] The token cache is being invalidate because project 25d46074b23b4a0c802e05437436fe3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:28.674084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.675020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7e525571-0d9e-4e30-9bd2-2e1bba7772df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.677384 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1192/2400] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:28 2022] DELETE /identity/v3/projects/25d46074b23b4a0c802e05437436fe3a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:28.682301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:28.697323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a6439-cd89-4cad-9ca1-2a413e25d7c6 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:28.813523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:28.838494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:28.839183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:28.839470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:28.839712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] PATH_INFO: `/v3/users/e46ca6451816417c8f19989cca9a3ed4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:28.845181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46ca6451816417c8f19989cca9a3ed4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:28.852894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:28.949796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46ca6451816417c8f19989cca9a3ed4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.950938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46ca6451816417c8f19989cca9a3ed4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.965472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46ca6451816417c8f19989cca9a3ed4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.987557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46ca6451816417c8f19989cca9a3ed4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:28.988333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46ca6451816417c8f19989cca9a3ed4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:29.017615 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1209/2401] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:28 2022] DELETE /identity/v3/users/e46ca6451816417c8f19989cca9a3ed4 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:29.643338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:29.644587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:29.645815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:29.710089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:29.713613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:29.764390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] MFA Rules not processed for user `0320d122f53c435588a69baafb9bcf98`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:29.770370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:29.974214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e36573-a696-47c1-8cb2-47e11cc0d955 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:29.988202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:29.989228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:29.990394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:29.990702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] PATH_INFO: `/v3/projects/9a2445b438a34ba0bf66223a09dcdde5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:29.995497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a2445b438a34ba0bf66223a09dcdde5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:29.997753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:30.105751 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1193/2402] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:30.118902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:30.126027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:30.130424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a2445b438a34ba0bf66223a09dcdde5', 'request_id': 'req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:30.152951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a2445b438a34ba0bf66223a09dcdde5', 'request_id': 'req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:30.168033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a2445b438a34ba0bf66223a09dcdde5', 'request_id': 'req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:30.185779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:30.189900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] The token cache is being invalidate because project 9a2445b438a34ba0bf66223a09dcdde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:30.190798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:30.194587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fa7802b-93e0-4dea-abd0-75ecd4d3ea33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:30.197502 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1210/2403] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:29 2022] DELETE /identity/v3/projects/9a2445b438a34ba0bf66223a09dcdde5 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:30.202738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503c8bd-0859-4ad7-ac73-97ec380aa4fc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:30.203076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503c8bd-0859-4ad7-ac73-97ec380aa4fc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:30.203494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503c8bd-0859-4ad7-ac73-97ec380aa4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:30.233380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a503c8bd-0859-4ad7-ac73-97ec380aa4fc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:30.234645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a503c8bd-0859-4ad7-ac73-97ec380aa4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:30.299941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:30.339641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492301-b198-45a4-a08e-d1cbfb5d5e4d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:30.452154 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1211/2404] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:30.454296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d84b2-8483-4d0c-b4f8-df0c9614a5d1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:30.454585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d84b2-8483-4d0c-b4f8-df0c9614a5d1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:30.454857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d84b2-8483-4d0c-b4f8-df0c9614a5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:30.483721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-971d84b2-8483-4d0c-b4f8-df0c9614a5d1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:30.487263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-971d84b2-8483-4d0c-b4f8-df0c9614a5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:30.505717 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1212/2405] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:30.506914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d84b2-8483-4d0c-b4f8-df0c9614a5d1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:30.524803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d84b2-8483-4d0c-b4f8-df0c9614a5d1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:30.544226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:30.546713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:30.546808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:30.547131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:30.547724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:30.548028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:30.555391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:30.565855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:30.566500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:30.566783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:30.567050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:30.574085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:30.604171 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1213/2406] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:30.605945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:30.635133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4c0a-1a6a-456b-a573-9378a0106092 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:30.661661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:30.678921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:30.681009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:30.681334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:30.682247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:30.685833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:30.749080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:30.751782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:30.773182 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1214/2407] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:30 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:30.775965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:30.776445 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1194/2408] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:31.032906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8e30-4abf-490a-9b0c-67ca019bf1cd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:31.048238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:31.048903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:31.049190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:31.049473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:31.052423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:31.057959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:31.139040 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1215/2409] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:30 2022] POST /identity/v3/users => generated 290 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:31.140357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:31.167234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5643f1-7f0e-45f8-a63d-cf43ff429d82 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:31.201986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:31.202670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:31.203018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:31.203291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:31.204005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:31.218917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:31.254461 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1195/2410] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:31.260818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:31.291291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd9e1b-7c51-4c4d-b861-8d9cf795a355 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:31.307439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b707c-5ae2-4a82-aece-767d60350eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:31.308467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b707c-5ae2-4a82-aece-767d60350eff admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:31.310179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b707c-5ae2-4a82-aece-767d60350eff admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:31.321717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b707c-5ae2-4a82-aece-767d60350eff admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:31.322751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b707c-5ae2-4a82-aece-767d60350eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:31.345349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b707c-5ae2-4a82-aece-767d60350eff admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:31.386358 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1216/2411] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:31.387829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:31.413108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccad08-5908-4ec8-807c-b6746ead2cee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:31.433447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:31.434260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:31.434719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:31.435005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] PATH_INFO: `/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba/users/6a7fa395e505456e9c5ea6236db9a96c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:31.473530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a6a4c63e014170bae36e6b6a3cc0ba, user_id=6a7fa395e505456e9c5ea6236db9a96c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:31.492413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:31.532781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1196/2412] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:31 2022] PUT /identity/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba/users/6a7fa395e505456e9c5ea6236db9a96c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:31.536816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efc784-0445-4b86-993f-d22a9aedc676 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:31.536816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efc784-0445-4b86-993f-d22a9aedc676 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:31.536816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efc784-0445-4b86-993f-d22a9aedc676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:31.584448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-77efc784-0445-4b86-993f-d22a9aedc676 None None] MFA Rules not processed for user `6a7fa395e505456e9c5ea6236db9a96c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:31.585909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-77efc784-0445-4b86-993f-d22a9aedc676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:31.620173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:31.695019 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1217/2413] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:31.699726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:31.881436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caf809-6617-42e2-a0da-5a63717ad5b6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:31.895519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:31.896388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:31.896682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:31.897128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:31.897976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:31.900694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:31.927541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:31.974445 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1197/2414] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:31 2022] POST /identity/v3/projects => generated 377 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:31.980171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efc784-0445-4b86-993f-d22a9aedc676 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:31.997986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efc784-0445-4b86-993f-d22a9aedc676 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:32.115427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:32.115695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:32.115844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:32.116120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:32.116948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:32.119395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:32.152622 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1218/2415] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:31 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:32.154039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:32.170319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee7bb-21f2-4682-b27f-bed2e6217915 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:32.181564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:32.182387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:32.182760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:32.183196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:32.184249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:32.189457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:32.214667 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1198/2416] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:32.218844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:32.235230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a6626-b227-405b-bbfb-97aa0b7fcc0e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:32.247703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:32.251257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:32.251544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:32.252325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] PATH_INFO: `/v3/projects/e9f9eeebae264d48ba9d73857c1ca301/users/32e3c8ff216c421c910464a6229104b2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:32.279013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f9eeebae264d48ba9d73857c1ca301, user_id=32e3c8ff216c421c910464a6229104b2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:32.302473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:32.340864 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1219/2417] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:32 2022] PUT /identity/v3/projects/e9f9eeebae264d48ba9d73857c1ca301/users/32e3c8ff216c421c910464a6229104b2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:32.342552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:32.373751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efb840-7cb2-4d68-bd6b-56b6fd8425f5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:32.621826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9044f9-d64e-435a-acb8-0c0e37e2996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:32.622828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9044f9-d64e-435a-acb8-0c0e37e2996c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:32.624079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9044f9-d64e-435a-acb8-0c0e37e2996c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:32.624512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9044f9-d64e-435a-acb8-0c0e37e2996c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:32.625480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9044f9-d64e-435a-acb8-0c0e37e2996c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:32.627203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9044f9-d64e-435a-acb8-0c0e37e2996c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:32.645887 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1199/2418] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:32.649325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:32.688574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ca59-74dc-46b0-8aa5-38edfa53396a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:32.701921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:32.702824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:32.705465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:32.708163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] PATH_INFO: `/v3/projects/e9f9eeebae264d48ba9d73857c1ca301/users/32e3c8ff216c421c910464a6229104b2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:32.711974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f9eeebae264d48ba9d73857c1ca301, user_id=32e3c8ff216c421c910464a6229104b2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:32.724507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:32.760144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1220/2419] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:32 2022] PUT /identity/v3/projects/e9f9eeebae264d48ba9d73857c1ca301/users/32e3c8ff216c421c910464a6229104b2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:32.762618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:32.763221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:32.763657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:32.828064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] MFA Rules not processed for user `32e3c8ff216c421c910464a6229104b2`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:32.834225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:33.021240 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1200/2420] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:33.747283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:33.750086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:33.863323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:33.865092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:33.951141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f51fc-05c6-4d83-a67d-31f5659fa357 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:33.960715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2a2b8-7011-49aa-a331-55dd39d0b3f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:33.961781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a2b8-7011-49aa-a331-55dd39d0b3f2 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:33.962197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a2b8-7011-49aa-a331-55dd39d0b3f2 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:33.962540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a2b8-7011-49aa-a331-55dd39d0b3f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:34.011059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2a2b8-7011-49aa-a331-55dd39d0b3f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:34.012866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2a2b8-7011-49aa-a331-55dd39d0b3f2 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:34.055616 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1221/2421] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:34.059846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:34.115548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efbd98-e383-4df7-bcbd-63cd03cbcb26 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:34.124482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24131b04-3f58-43ee-b825-faaa714794cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:34.125348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24131b04-3f58-43ee-b825-faaa714794cf service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:34.127603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24131b04-3f58-43ee-b825-faaa714794cf service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:34.128467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24131b04-3f58-43ee-b825-faaa714794cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:34.133545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-24131b04-3f58-43ee-b825-faaa714794cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:34.246121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24131b04-3f58-43ee-b825-faaa714794cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:34.248709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24131b04-3f58-43ee-b825-faaa714794cf service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:34.261639 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1201/2422] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:34.463600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:34.463990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:34.464292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:34.473926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:34.503775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] MFA Rules not processed for user `f353d5a0cabf4c24ac7ee5c9e7f44007`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:34.506989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:34.658492 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1222/2423] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:34.813966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d198cb-9653-4a5b-9921-a3ed4de4e0cf None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:34.814371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d198cb-9653-4a5b-9921-a3ed4de4e0cf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:34.814791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d198cb-9653-4a5b-9921-a3ed4de4e0cf None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:34.816296 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1202/2424] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:34.824712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:34.839263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:34.853119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567b066-bbaf-4ac3-af21-6fcfdaadebc6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:34.862548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2a798-646f-4cc7-afb7-5a9d8a6cc8dd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32e3c8ff216c421c910464a6229104b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9f9eeebae264d48ba9d73857c1ca301', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:34.863284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a798-646f-4cc7-afb7-5a9d8a6cc8dd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:34.863575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a798-646f-4cc7-afb7-5a9d8a6cc8dd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:34.863849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a798-646f-4cc7-afb7-5a9d8a6cc8dd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] PATH_INFO: `/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:34.868269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2a798-646f-4cc7-afb7-5a9d8a6cc8dd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] RBAC: Authorizing `identity:get_project(project_id=e1a6a4c63e014170bae36e6b6a3cc0ba)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:34.871508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2a798-646f-4cc7-afb7-5a9d8a6cc8dd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:34.880440 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1223/2425] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:34 2022] GET /identity/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba => generated 379 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:35.678363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d198cb-9653-4a5b-9921-a3ed4de4e0cf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:35.711733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d198cb-9653-4a5b-9921-a3ed4de4e0cf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:35.717933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fa723-264f-44ab-b2b2-290d991fd317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:35.719901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa723-264f-44ab-b2b2-290d991fd317 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:35.720389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa723-264f-44ab-b2b2-290d991fd317 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:35.720863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa723-264f-44ab-b2b2-290d991fd317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:35.786812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fa723-264f-44ab-b2b2-290d991fd317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:35.788796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fa723-264f-44ab-b2b2-290d991fd317 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:35.799088 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1203/2426] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:35.924986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f139a-3cad-4ad7-a9fb-ba95b5e6f7c4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:35.925605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f139a-3cad-4ad7-a9fb-ba95b5e6f7c4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:35.925901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f139a-3cad-4ad7-a9fb-ba95b5e6f7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:35.964138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9a1f139a-3cad-4ad7-a9fb-ba95b5e6f7c4 None None] MFA Rules not processed for user `9544cdb0392f416197a586d99e8a3210`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:35.965180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d041bb-d14d-408a-a58f-9e05847be0b3 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:35.965662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d041bb-d14d-408a-a58f-9e05847be0b3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:35.965950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9a1f139a-3cad-4ad7-a9fb-ba95b5e6f7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:35.966190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d041bb-d14d-408a-a58f-9e05847be0b3 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:35.968121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1204/2427] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:35.973948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d041bb-d14d-408a-a58f-9e05847be0b3 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:36.002072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d041bb-d14d-408a-a58f-9e05847be0b3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:36.021323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d041bb-d14d-408a-a58f-9e05847be0b3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:36.033315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48224eaf-7e24-47ca-9610-4207a023b6d8 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32e3c8ff216c421c910464a6229104b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9f9eeebae264d48ba9d73857c1ca301', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:36.036036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224eaf-7e24-47ca-9610-4207a023b6d8 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:36.036545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224eaf-7e24-47ca-9610-4207a023b6d8 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:36.037044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48224eaf-7e24-47ca-9610-4207a023b6d8 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] PATH_INFO: `/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:36.040867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48224eaf-7e24-47ca-9610-4207a023b6d8 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] RBAC: Authorizing `identity:get_project(project_id=e1a6a4c63e014170bae36e6b6a3cc0ba)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:36.047162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48224eaf-7e24-47ca-9610-4207a023b6d8 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:36.061855 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1224/2428] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:36.070922 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1205/2429] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:35 2022] GET /identity/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba => generated 379 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:58:38.092915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f139a-3cad-4ad7-a9fb-ba95b5e6f7c4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:38.177824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f139a-3cad-4ad7-a9fb-ba95b5e6f7c4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:38.183454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:38.184409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:38.184881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:38.185299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] PATH_INFO: `/v3/users/6a7fa395e505456e9c5ea6236db9a96c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:38.187032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a7fa395e505456e9c5ea6236db9a96c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:38.189419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:38.240768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:38.266275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a7fa395e505456e9c5ea6236db9a96c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:38.267028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a7fa395e505456e9c5ea6236db9a96c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:38.277464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a7fa395e505456e9c5ea6236db9a96c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:38.290243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a7fa395e505456e9c5ea6236db9a96c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:38.291864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a7fa395e505456e9c5ea6236db9a96c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:38.319268 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1225/2430] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:38 2022] DELETE /identity/v3/users/6a7fa395e505456e9c5ea6236db9a96c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:38.321825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:38.322722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:38.323234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:38.351122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:38.352766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:38.439924 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1206/2431] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:38.453959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:38.455681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:38.540131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298ab2f-fba2-48d8-b126-c58e204e1632 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:38.546773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:38.547443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:38.547719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:38.548396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:38.561531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:38.566592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:38.587178 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1226/2432] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:58:38.973570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:38.975312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:39.000436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d473cb-6a56-4f77-959b-2ee1a6e98035 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:39.010957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:39.011974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:39.012377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:39.014107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] PATH_INFO: `/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:39.015510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a6a4c63e014170bae36e6b6a3cc0ba)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:39.017650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:39.046738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:39.096505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a6a4c63e014170bae36e6b6a3cc0ba', 'request_id': 'req-2957a465-4d9e-415f-b599-ec4b066e478e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.105432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a6a4c63e014170bae36e6b6a3cc0ba', 'request_id': 'req-2957a465-4d9e-415f-b599-ec4b066e478e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.118850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a6a4c63e014170bae36e6b6a3cc0ba', 'request_id': 'req-2957a465-4d9e-415f-b599-ec4b066e478e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.134994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] The token cache is being invalidate because project e1a6a4c63e014170bae36e6b6a3cc0ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:39.135571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.136587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.139038 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1207/2433] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:38 2022] DELETE /identity/v3/projects/e1a6a4c63e014170bae36e6b6a3cc0ba => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:39.143106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:39.156552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce768-f9fa-4da1-9870-0eb2e4eca844 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:39.290862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:39.291931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:39.292400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:39.292773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] PATH_INFO: `/v3/users/32e3c8ff216c421c910464a6229104b2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:39.295667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e3c8ff216c421c910464a6229104b2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:39.298231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:39.389325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e3c8ff216c421c910464a6229104b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.389928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e3c8ff216c421c910464a6229104b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.396928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:39.425174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e3c8ff216c421c910464a6229104b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.437031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e3c8ff216c421c910464a6229104b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.437586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-39051ba2-3892-48a2-8534-c00faf890d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e3c8ff216c421c910464a6229104b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:39.471305 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1227/2434] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:39 2022] DELETE /identity/v3/users/32e3c8ff216c421c910464a6229104b2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:40.140592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:40.144725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:40.194222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a465-4d9e-415f-b599-ec4b066e478e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:40.349314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:40.349982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:40.350256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:40.350525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] PATH_INFO: `/v3/projects/e9f9eeebae264d48ba9d73857c1ca301` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:40.352825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f9eeebae264d48ba9d73857c1ca301)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:40.360071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:40.463020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f9eeebae264d48ba9d73857c1ca301', 'request_id': 'req-84dc1d1b-d06e-4529-94be-565a0636ad7f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:40.471227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f9eeebae264d48ba9d73857c1ca301', 'request_id': 'req-84dc1d1b-d06e-4529-94be-565a0636ad7f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:40.482500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f9eeebae264d48ba9d73857c1ca301', 'request_id': 'req-84dc1d1b-d06e-4529-94be-565a0636ad7f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:40.499900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] The token cache is being invalidate because project e9f9eeebae264d48ba9d73857c1ca301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:40.500552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:40.501944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-84dc1d1b-d06e-4529-94be-565a0636ad7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:40.504175 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1208/2435] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:40 2022] DELETE /identity/v3/projects/e9f9eeebae264d48ba9d73857c1ca301 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:40.511355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8e91-3817-43f9-a061-44d3aa14842f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:40.511904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8e91-3817-43f9-a061-44d3aa14842f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:40.512010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8e91-3817-43f9-a061-44d3aa14842f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:40.541814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6a8d8e91-3817-43f9-a061-44d3aa14842f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:40.543156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6a8d8e91-3817-43f9-a061-44d3aa14842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:40.642351 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1228/2436] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:40.647129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf1e5-b668-4a82-a582-8f5af5508c96 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:40.647821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf1e5-b668-4a82-a582-8f5af5508c96 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:40.648328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bf1e5-b668-4a82-a582-8f5af5508c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:40.696642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-418bf1e5-b668-4a82-a582-8f5af5508c96 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:40.699416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-418bf1e5-b668-4a82-a582-8f5af5508c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:40.744428 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1209/2437] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:40.745053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d8e91-3817-43f9-a061-44d3aa14842f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:40.760827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d8e91-3817-43f9-a061-44d3aa14842f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:40.777488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:40.784118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:40.784994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:40.785592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:40.786907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:40.787434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:40.790925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:40.813290 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1229/2438] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:40.818755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf1e5-b668-4a82-a582-8f5af5508c96 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:40.834588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bf1e5-b668-4a82-a582-8f5af5508c96 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:40.851082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:40.851973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:40.852323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:40.852751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:40.853497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:40.857567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:40.904132 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1210/2439] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:40 2022] POST /identity/v3/projects => generated 393 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:40.908914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:40.931065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f212c-4aeb-455c-aa4e-4e7adf234562 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:41.055302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:41.055940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.056214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.058737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.060471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:41.069268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:41.105384 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1230/2440] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:40 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:41.108987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:41.120522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:41.132342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73464cc-ea95-488c-ab53-11c61a737c63 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:41.165884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:41.167306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.167904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.168375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.169246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:41.171453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:41.213750 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1211/2441] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:41.219518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:41.237043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc705759-b940-49b8-b570-3b5b786f4ab0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:41.243657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249dd8a-3a0d-4c35-9b80-287a5611de22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:41.244528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249dd8a-3a0d-4c35-9b80-287a5611de22 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.244954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249dd8a-3a0d-4c35-9b80-287a5611de22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.246025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249dd8a-3a0d-4c35-9b80-287a5611de22 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.246880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249dd8a-3a0d-4c35-9b80-287a5611de22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:41.248943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249dd8a-3a0d-4c35-9b80-287a5611de22 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:41.269763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:41.270819 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1231/2442] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:41.290727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42798eca-96cc-4fc7-a611-6b4cd4116909 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:41.304768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:41.305837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.306463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.308180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] PATH_INFO: `/v3/projects/ed5453928ec84da48914a387ca695902/users/ac12036ae0a0427b819ccb5b75a25184/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.371300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed5453928ec84da48914a387ca695902, user_id=ac12036ae0a0427b819ccb5b75a25184, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:41.374213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:41.403465 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1212/2443] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:41 2022] PUT /identity/v3/projects/ed5453928ec84da48914a387ca695902/users/ac12036ae0a0427b819ccb5b75a25184/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:41.416378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555a718-e5ff-48c3-a262-a312555d76b7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.416692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555a718-e5ff-48c3-a262-a312555d76b7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.417005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555a718-e5ff-48c3-a262-a312555d76b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.492560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0555a718-e5ff-48c3-a262-a312555d76b7 None None] MFA Rules not processed for user `ac12036ae0a0427b819ccb5b75a25184`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:41.495891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0555a718-e5ff-48c3-a262-a312555d76b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:41.525757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:41.558851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:41.560894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:41.673378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1232/2444] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:41.675564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555a718-e5ff-48c3-a262-a312555d76b7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:41.707048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b4ad8-351d-431c-b4da-148da202b05a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:41.713911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:41.715166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.716255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.716645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] PATH_INFO: `/v3/users/4319ec4b90e1481b8be19e8f9dde1ea4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.718747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4319ec4b90e1481b8be19e8f9dde1ea4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:41.721230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:41.788867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555a718-e5ff-48c3-a262-a312555d76b7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:41.797198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:41.797928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.798973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.799543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.800789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:41.802758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4319ec4b90e1481b8be19e8f9dde1ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:41.803675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:41.805602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4319ec4b90e1481b8be19e8f9dde1ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:41.819603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4319ec4b90e1481b8be19e8f9dde1ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:41.833118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4319ec4b90e1481b8be19e8f9dde1ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:41.842059 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1233/2445] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:41 2022] POST /identity/v3/projects => generated 395 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:41.847357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:41.849659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:41.861579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c49b541d-b1f2-4686-ab03-297172f5fa58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4319ec4b90e1481b8be19e8f9dde1ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:41.875642 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1213/2446] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:41 2022] DELETE /identity/v3/users/4319ec4b90e1481b8be19e8f9dde1ea4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:41.877703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:41.879956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:41.880281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:41.919470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:41.922362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:41.965985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee418a3-cef7-4761-8f2f-48c4ab599c59 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:42.044540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:42.059914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:42.073410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:42.074986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.075115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.076213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.077525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:42.080689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:42.096713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1214/2447] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:42.108830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:42.110676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:42.121181 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1234/2448] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:41 2022] POST /identity/v3/users => generated 304 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:42.125554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:42.159542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fddbd-8dc8-4ba5-9311-11d7657f6c5e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:42.173120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:42.174480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.174988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.175625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.176763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:42.179314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:42.193887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c560cb2-b38e-491f-a991-cbd9e11a01e5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:42.196066 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1235/2449] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:42.200925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:42.230555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:42.235309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.235309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.235309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.241695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033fc8b-3618-4787-b7ea-595b2d73f5bb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:42.261383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:42.262219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.262497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.263005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] PATH_INFO: `/v3/projects/38e89dbee95b4e308e77acde9c3a4dfc/users/107c3429510a42ae969e20671542338d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.266467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:42.271604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:42.288221 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1215/2450] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:42.327350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e89dbee95b4e308e77acde9c3a4dfc, user_id=107c3429510a42ae969e20671542338d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:42.338566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:42.361253 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1236/2451] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:42 2022] PUT /identity/v3/projects/38e89dbee95b4e308e77acde9c3a4dfc/users/107c3429510a42ae969e20671542338d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:42.365133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:42.382278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def45ef-b725-4f34-a576-694001ee48c3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:42.527946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb552c-2f47-49ad-835f-5eafe0b14a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:42.527946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb552c-2f47-49ad-835f-5eafe0b14a28 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.527946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb552c-2f47-49ad-835f-5eafe0b14a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.527946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb552c-2f47-49ad-835f-5eafe0b14a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.528472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb552c-2f47-49ad-835f-5eafe0b14a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:42.531678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb552c-2f47-49ad-835f-5eafe0b14a28 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:42.546515 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1216/2452] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:58:42.550896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:42.568564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c35a-74a5-4b24-9ebc-d3fca1c4c48b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:42.574975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:42.575731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.575990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.576345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] PATH_INFO: `/v3/projects/38e89dbee95b4e308e77acde9c3a4dfc/users/107c3429510a42ae969e20671542338d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.579447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e89dbee95b4e308e77acde9c3a4dfc, user_id=107c3429510a42ae969e20671542338d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:42.582975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:42.607449 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1237/2453] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:42 2022] PUT /identity/v3/projects/38e89dbee95b4e308e77acde9c3a4dfc/users/107c3429510a42ae969e20671542338d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:42.609027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aab752-37be-49a9-b365-a48980abf004 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.609317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aab752-37be-49a9-b365-a48980abf004 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.609804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aab752-37be-49a9-b365-a48980abf004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.649246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-06aab752-37be-49a9-b365-a48980abf004 None None] MFA Rules not processed for user `107c3429510a42ae969e20671542338d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:42.650913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06aab752-37be-49a9-b365-a48980abf004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:42.774015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1217/2454] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:42 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:42.778941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:42.848607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aab752-37be-49a9-b365-a48980abf004 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:42.850511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06aab752-37be-49a9-b365-a48980abf004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:42.881262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:42.938504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b6711-f1cb-4bcf-9a17-0713c7682fcb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:42.950499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:42.951583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:42.952642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:42.953258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:42.955011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:42.965716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.012452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aab752-37be-49a9-b365-a48980abf004 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.017442 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1238/2455] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:42 2022] POST /identity/v3/projects => generated 393 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:43.018443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:43.090615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.091469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.091704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.092126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] PATH_INFO: `/v3/projects/6e8a121c6c654613a6a92df03c3754fa` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.093684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8a121c6c654613a6a92df03c3754fa)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:43.095887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.099295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c9ac3-bcf5-4778-ae99-97897b2e03b4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.104711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.105976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.106311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.106757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.107397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:43.110137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.136327 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1239/2456] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:43 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:43.140135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:43.151245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dc780-0ca2-4602-a30f-cd040cc24f11 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.152905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8a121c6c654613a6a92df03c3754fa', 'request_id': 'req-547b2621-4424-427b-91e6-fa9daaa42bc2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.159322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:43.160575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8a121c6c654613a6a92df03c3754fa', 'request_id': 'req-547b2621-4424-427b-91e6-fa9daaa42bc2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.170556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8a121c6c654613a6a92df03c3754fa', 'request_id': 'req-547b2621-4424-427b-91e6-fa9daaa42bc2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.183203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] The token cache is being invalidate because project 6e8a121c6c654613a6a92df03c3754fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:43.183513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.184121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.185981 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1218/2457] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:42 2022] DELETE /identity/v3/projects/6e8a121c6c654613a6a92df03c3754fa => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:43.189568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:43.191119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:43.224189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.225234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.225491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.226131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.226825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:43.229645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.248694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1240/2458] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:43.249261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:43.251348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:43.256353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:43.277504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5445df2-90ff-44fc-816e-0fa7351247b6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.282288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b2621-4424-427b-91e6-fa9daaa42bc2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.283721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.284357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.284626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.284977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.285599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:43.287321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.293788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.294411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.294705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.300366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] PATH_INFO: `/v3/users/d8b69329ff4d4a79b7cd0d056ae722e1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.300366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b69329ff4d4a79b7cd0d056ae722e1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:43.302325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.306496 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1241/2459] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:43.311383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:43.328276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc0d85-863d-4b46-aa8d-28468ebeba26 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.333510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487681dc-dd69-4af8-8315-f059a312d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.334193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487681dc-dd69-4af8-8315-f059a312d7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.334464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487681dc-dd69-4af8-8315-f059a312d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.334736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487681dc-dd69-4af8-8315-f059a312d7c7 admin admin] PATH_INFO: `/v3/projects/a68f04c73d0c460386e37324fabab1e5/users/950d2e8d194040be9bc5cfd08b2e0156/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.367718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487681dc-dd69-4af8-8315-f059a312d7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68f04c73d0c460386e37324fabab1e5, user_id=950d2e8d194040be9bc5cfd08b2e0156, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:43.370996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487681dc-dd69-4af8-8315-f059a312d7c7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.389714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b69329ff4d4a79b7cd0d056ae722e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.390450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b69329ff4d4a79b7cd0d056ae722e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.403646 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1242/2460] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:43 2022] PUT /identity/v3/projects/a68f04c73d0c460386e37324fabab1e5/users/950d2e8d194040be9bc5cfd08b2e0156/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:43.416929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7e67f-1c27-48ad-a897-ae3a47c99ade None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.417300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7e67f-1c27-48ad-a897-ae3a47c99ade None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.417848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7e67f-1c27-48ad-a897-ae3a47c99ade None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.419584 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1243/2461] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:58:43 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:43.421068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.421429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.421775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.422069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b69329ff4d4a79b7cd0d056ae722e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.436318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b69329ff4d4a79b7cd0d056ae722e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.456942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] MFA Rules not processed for user `950d2e8d194040be9bc5cfd08b2e0156`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:43.458569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:43.462231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-df8fbcbc-257b-4e3b-a99a-a3e60b628807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b69329ff4d4a79b7cd0d056ae722e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:43.470136 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1219/2462] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:43 2022] DELETE /identity/v3/users/d8b69329ff4d4a79b7cd0d056ae722e1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:43.489620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.489899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.490175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.540454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:43.546821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:43.553426 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1244/2463] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:43.711976 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1220/2464] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:58:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:43.746653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:43.748112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:43.760985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:43.762439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:43.885682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:43.889062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:43.899810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091092a-9924-4065-8039-66d89d99bd7e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.907664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83244-3b2a-46df-b2b6-93a4d872c203 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:43.915062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.915885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.916161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.916452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.930507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2c3a5-dedd-40a9-af4a-35f347021faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:43.931328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2c3a5-dedd-40a9-af4a-35f347021faa service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:43.931607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2c3a5-dedd-40a9-af4a-35f347021faa service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:43.931925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2c3a5-dedd-40a9-af4a-35f347021faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:43.934947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:43.936056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c0b2c3a5-dedd-40a9-af4a-35f347021faa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:43.939725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:43.984922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1245/2465] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:44.009093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:44.011085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:44.069544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2c3a5-dedd-40a9-af4a-35f347021faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:44.072409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2c3a5-dedd-40a9-af4a-35f347021faa service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:44.113422 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1221/2466] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:43 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:44.159234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad100e8-ccf8-4654-9d60-0bc2677c23f1 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:44.171991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:44.172844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.173243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.173572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] PATH_INFO: `/v3/projects/d1a7023a446f417e9764a1c113453548` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.176197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a7023a446f417e9764a1c113453548)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:44.178357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:44.292427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a7023a446f417e9764a1c113453548', 'request_id': 'req-ff430ca9-74f6-476f-b4d9-7a13a576e180'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:44.306626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a7023a446f417e9764a1c113453548', 'request_id': 'req-ff430ca9-74f6-476f-b4d9-7a13a576e180'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:44.315168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a7023a446f417e9764a1c113453548', 'request_id': 'req-ff430ca9-74f6-476f-b4d9-7a13a576e180'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:44.331280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] The token cache is being invalidate because project d1a7023a446f417e9764a1c113453548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:44.331688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:44.337136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ff430ca9-74f6-476f-b4d9-7a13a576e180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:44.340186 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1246/2467] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:44 2022] DELETE /identity/v3/projects/d1a7023a446f417e9764a1c113453548 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:44.360006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5e459-94ca-4ace-bb81-1a547cf96a82 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.360560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.360647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5e459-94ca-4ace-bb81-1a547cf96a82 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.360762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.360838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5e459-94ca-4ace-bb81-1a547cf96a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.360951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.362191 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1247/2468] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:58:44.369455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:44.371261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:44.387568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8bc5e459-94ca-4ace-bb81-1a547cf96a82 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:44.389352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8bc5e459-94ca-4ace-bb81-1a547cf96a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:44.491752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:44.496518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:44.520279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:44.547602 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1222/2469] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:44.551550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a736-fee0-4d12-baf1-a7af6b3c76ea None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.551869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a736-fee0-4d12-baf1-a7af6b3c76ea None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.552147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a736-fee0-4d12-baf1-a7af6b3c76ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.594167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c4f6a736-fee0-4d12-baf1-a7af6b3c76ea None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:44.595556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c4f6a736-fee0-4d12-baf1-a7af6b3c76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:44.599042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557fc16-3310-4ac6-b1ec-5af532504eb0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:44.606857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107c3429510a42ae969e20671542338d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e89dbee95b4e308e77acde9c3a4dfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:44.608766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.609266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.609762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] PATH_INFO: `/v3/projects/ed5453928ec84da48914a387ca695902` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.611481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorizing `identity:get_project(project_id=ed5453928ec84da48914a387ca695902)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:44.614127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:44.616734 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1248/2470] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:44 2022] GET /identity/v3/projects/ed5453928ec84da48914a387ca695902 => generated 393 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:58:44.625724 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1223/2471] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:44.634058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:44.664418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa2fc9-de34-46f9-a388-83a7a6641292 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:44.680539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:44.681314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.681631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.681993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.682678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:44.682975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:44.684735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:44.693516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d5ee2-4eae-414e-a759-3b3909ac8dde None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.694178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d5ee2-4eae-414e-a759-3b3909ac8dde None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.694606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d5ee2-4eae-414e-a759-3b3909ac8dde None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.696304 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1224/2472] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:58:44.699710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1249/2473] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:44.703932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d5ee2-4eae-414e-a759-3b3909ac8dde None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:44.706088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:44.719927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:44.725612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d5ee2-4eae-414e-a759-3b3909ac8dde None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:44.731553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3a316-aee8-49fd-b7de-1a5357acf12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:44.732512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a316-aee8-49fd-b7de-1a5357acf12f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.733073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a316-aee8-49fd-b7de-1a5357acf12f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.733500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd94488-8259-4add-a271-aa2ee2b4da2d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:44.733952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a316-aee8-49fd-b7de-1a5357acf12f admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.734950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3a316-aee8-49fd-b7de-1a5357acf12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:44.737217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3a316-aee8-49fd-b7de-1a5357acf12f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:44.749283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107c3429510a42ae969e20671542338d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e89dbee95b4e308e77acde9c3a4dfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:44.752153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.753029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.753029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] PATH_INFO: `/v3/projects/ed5453928ec84da48914a387ca695902` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.754584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorizing `identity:get_project(project_id=ed5453928ec84da48914a387ca695902)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:44.756803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:44.759559 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1250/2474] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:44 2022] GET /identity/v3/projects/ed5453928ec84da48914a387ca695902 => generated 393 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:58:44.793861 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1225/2475] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:44 2022] POST /identity/v3/projects => generated 359 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:44.796082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:44.822498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71709ac-1bea-41da-8b32-16ff75da5b89 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:44.889467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04bead-652a-4d24-be03-9494857d2de4 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.889937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04bead-652a-4d24-be03-9494857d2de4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.890687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04bead-652a-4d24-be03-9494857d2de4 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.892968 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1226/2476] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:44.899735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04bead-652a-4d24-be03-9494857d2de4 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:44.902337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ef04bead-652a-4d24-be03-9494857d2de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:44.926088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:44.927587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:44.927934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:44.928283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:44.929357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:44.931832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:44.981088 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1251/2477] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:44 2022] POST /identity/v3/users => generated 280 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:44.989243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.006328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea9721-b613-48f5-8e2a-77f8029f4b96 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.028100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.036010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04bead-652a-4d24-be03-9494857d2de4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.036414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.036759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.037382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.038102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.041087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.043716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:45.059346 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1252/2478] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:45.066289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.079596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761fd9b-eeef-4f87-b5cf-ba7e7b06bcc7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.086354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.086978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.087433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.087710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.088504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.090153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.104875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1253/2479] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:45.109938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.121792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c92f3-57d0-4285-b8b4-6df7cf0d74d6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.129858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.130547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.130845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.131402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04bead-652a-4d24-be03-9494857d2de4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.131615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] PATH_INFO: `/v3/projects/dd453b58a0d84fa7a04c715e7e35876b/users/ade1aa40dec64038a0145816d91e052f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.139965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3baf18e-cff6-4004-b6c9-c3064d0dddfd tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107c3429510a42ae969e20671542338d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e89dbee95b4e308e77acde9c3a4dfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.140959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3baf18e-cff6-4004-b6c9-c3064d0dddfd tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.141237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3baf18e-cff6-4004-b6c9-c3064d0dddfd tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.141610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3baf18e-cff6-4004-b6c9-c3064d0dddfd tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] PATH_INFO: `/v3/projects/ed5453928ec84da48914a387ca695902` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.143659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3baf18e-cff6-4004-b6c9-c3064d0dddfd tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorizing `identity:get_project(project_id=ed5453928ec84da48914a387ca695902)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.146276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3baf18e-cff6-4004-b6c9-c3064d0dddfd tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.150572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd453b58a0d84fa7a04c715e7e35876b, user_id=ade1aa40dec64038a0145816d91e052f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.153334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.154974 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1227/2480] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:44 2022] GET /identity/v3/projects/ed5453928ec84da48914a387ca695902 => generated 393 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:58:45.176305 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1254/2481] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:45 2022] PUT /identity/v3/projects/dd453b58a0d84fa7a04c715e7e35876b/users/ade1aa40dec64038a0145816d91e052f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:45.187159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce42c2-98a0-4217-b9aa-6337516e1239 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.187439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce42c2-98a0-4217-b9aa-6337516e1239 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.187717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce42c2-98a0-4217-b9aa-6337516e1239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.224541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3cce42c2-98a0-4217-b9aa-6337516e1239 None None] MFA Rules not processed for user `ade1aa40dec64038a0145816d91e052f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:45.226284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3cce42c2-98a0-4217-b9aa-6337516e1239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:45.336854 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1228/2482] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:45.342084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.397430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:45.425565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2e56f-6dee-4adc-a9c5-8b05c2f9ef3d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.434978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.435255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.435538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.435803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.436451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.438642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.477171 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1255/2483] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:45 2022] POST /identity/v3/projects => generated 359 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:45.481409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce42c2-98a0-4217-b9aa-6337516e1239 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.496533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce42c2-98a0-4217-b9aa-6337516e1239 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.525098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.598340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.598986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.601096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.601096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.601096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.602514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.633846 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1229/2484] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:45 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:58:45.634814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.644159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb855-d50d-41c5-98a6-77f1dfd721a4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.645398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afbf7b-7621-4775-b45e-bacba5cfabea admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.650078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.650422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.650841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.651172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.651278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.651567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.651687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.651955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.652691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.654362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:45.654484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.670088 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1230/2485] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:45.674417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.691196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46c918-1b2f-44b8-90ef-6ad7fc9d16d5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.697274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.698351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.698688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.699023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] PATH_INFO: `/v3/projects/8188b85342dd4696a5ba267dfa4ab682/users/bbd0e26448d14160b09c9523e16e9e31/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.713208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:45.716521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8188b85342dd4696a5ba267dfa4ab682, user_id=bbd0e26448d14160b09c9523e16e9e31, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.720127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.727450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.729099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.740096 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1231/2486] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:45 2022] PUT /identity/v3/projects/8188b85342dd4696a5ba267dfa4ab682/users/bbd0e26448d14160b09c9523e16e9e31/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:45.742215 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1256/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:58:45.744518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.830052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c680b10-ef90-4fd9-acdd-8fcdd59dcdf0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.835225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0abfe-5d03-413f-a8d6-4762a6a936e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.835955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0abfe-5d03-413f-a8d6-4762a6a936e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.836361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0abfe-5d03-413f-a8d6-4762a6a936e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.837030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0abfe-5d03-413f-a8d6-4762a6a936e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.837480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0abfe-5d03-413f-a8d6-4762a6a936e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.839884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0abfe-5d03-413f-a8d6-4762a6a936e4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.856709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1232/2488] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:58:45.858462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:45.869911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94baae-81eb-43db-a952-98e305917d90 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:45.875066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:45.875726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.876004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.876275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] PATH_INFO: `/v3/projects/8188b85342dd4696a5ba267dfa4ab682/users/bbd0e26448d14160b09c9523e16e9e31/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.878383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8188b85342dd4696a5ba267dfa4ab682, user_id=bbd0e26448d14160b09c9523e16e9e31, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:45.881933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:45.898739 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1257/2489] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:45 2022] PUT /identity/v3/projects/8188b85342dd4696a5ba267dfa4ab682/users/bbd0e26448d14160b09c9523e16e9e31/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:45.909162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7337a-8379-44b1-a391-9efeaff61a2a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:45.909469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7337a-8379-44b1-a391-9efeaff61a2a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:45.909752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7337a-8379-44b1-a391-9efeaff61a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:45.949175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d1f7337a-8379-44b1-a391-9efeaff61a2a None None] MFA Rules not processed for user `bbd0e26448d14160b09c9523e16e9e31`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:45.950663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d1f7337a-8379-44b1-a391-9efeaff61a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:46.048290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1233/2490] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:46.163082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:46.234903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:46.259357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a84a00-4992-4924-932d-b0352ff7411e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:46.264724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:46.265334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:46.265607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:46.265872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:46.274987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:46.276587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:46.286753 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1258/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:46.451881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6efe-ce01-4931-883a-f3f5027eb820 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:46.456732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6efe-ce01-4931-883a-f3f5027eb820 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:46.456984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6efe-ce01-4931-883a-f3f5027eb820 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:46.458756 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1234/2492] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:46.466429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:46.478717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:46.562980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:46.563567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:46.563567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:46.575466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6ee6f-d8fb-46df-9ff5-152f0a0a7ce9 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:46.581708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5a07e-f319-43c4-9051-ca5d85c67959 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107c3429510a42ae969e20671542338d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e89dbee95b4e308e77acde9c3a4dfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:46.582370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a07e-f319-43c4-9051-ca5d85c67959 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:46.582647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a07e-f319-43c4-9051-ca5d85c67959 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:46.583671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a07e-f319-43c4-9051-ca5d85c67959 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] PATH_INFO: `/v3/projects/ed5453928ec84da48914a387ca695902` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:46.584343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5a07e-f319-43c4-9051-ca5d85c67959 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorizing `identity:get_project(project_id=ed5453928ec84da48914a387ca695902)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:46.586970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5a07e-f319-43c4-9051-ca5d85c67959 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:46.589024 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1259/2493] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:46 2022] GET /identity/v3/projects/ed5453928ec84da48914a387ca695902 => generated 393 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:46.591217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] MFA Rules not processed for user `00423eb1e4c24763ae0a546225ac1dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:46.592451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:46.667751 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1235/2494] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:46.673239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf93936-4215-4625-9257-642b4c208a9e None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:46.673752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf93936-4215-4625-9257-642b4c208a9e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:46.673752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf93936-4215-4625-9257-642b4c208a9e None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:46.675755 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1260/2495] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:46.681752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:46.698550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:46.711042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa92cc6-491b-4f0a-adcb-188de528d385 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:46.715867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace4b46-66a1-42fa-ab6f-8aa021d1f061 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107c3429510a42ae969e20671542338d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e89dbee95b4e308e77acde9c3a4dfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:46.716496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace4b46-66a1-42fa-ab6f-8aa021d1f061 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:46.717069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace4b46-66a1-42fa-ab6f-8aa021d1f061 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:46.717069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace4b46-66a1-42fa-ab6f-8aa021d1f061 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] PATH_INFO: `/v3/projects/ed5453928ec84da48914a387ca695902` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:46.718265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace4b46-66a1-42fa-ab6f-8aa021d1f061 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorizing `identity:get_project(project_id=ed5453928ec84da48914a387ca695902)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:46.720321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace4b46-66a1-42fa-ab6f-8aa021d1f061 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:46.722230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1236/2496] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:46 2022] GET /identity/v3/projects/ed5453928ec84da48914a387ca695902 => generated 393 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:47.127405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf93936-4215-4625-9257-642b4c208a9e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:47.247020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf93936-4215-4625-9257-642b4c208a9e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:47.254006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:47.254853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:47.255460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:47.256057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] PATH_INFO: `/v3/users/ade1aa40dec64038a0145816d91e052f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:47.257479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade1aa40dec64038a0145816d91e052f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:47.260214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:47.370489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade1aa40dec64038a0145816d91e052f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:47.371856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade1aa40dec64038a0145816d91e052f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:47.385245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade1aa40dec64038a0145816d91e052f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:47.394334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade1aa40dec64038a0145816d91e052f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:47.395440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-63484863-bdd6-4bc7-9443-812cc03a8b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade1aa40dec64038a0145816d91e052f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:47.417255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:47.446287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d19439-3b09-4364-b5fb-fc32b2f2c738 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:47.446558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d19439-3b09-4364-b5fb-fc32b2f2c738 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:47.446835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d19439-3b09-4364-b5fb-fc32b2f2c738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:47.451586 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1261/2497] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:47 2022] DELETE /identity/v3/users/ade1aa40dec64038a0145816d91e052f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:47.456266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:47.456801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:47.457227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:47.494678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-78d19439-3b09-4364-b5fb-fc32b2f2c738 None None] MFA Rules not processed for user `b4cebc85e62046d28fab159c6bb13e15`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:47.498331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-78d19439-3b09-4364-b5fb-fc32b2f2c738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:47.517831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:47.520635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:47.522773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:47.689301 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1262/2498] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:58:47.705687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:47.707249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:47.711656 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1237/2499] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:47.766283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:47.766703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:47.767078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:47.769486 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1238/2500] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 09:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:58:47.778456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 09:58:47.780996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:47.870531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a6510-aa7d-4da0-80cf-df366a846280 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:47.884545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:47.885226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:47.885502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:47.885765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:47.924756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:47.926430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:47.955721 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1263/2501] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:58:48.013332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:48.015110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:48.127110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987741f-32f3-40b7-b429-b80891b69d46 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:48.135792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7b686-be9d-4d60-91ab-56ad7efe9158 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107c3429510a42ae969e20671542338d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38e89dbee95b4e308e77acde9c3a4dfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:48.138288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7b686-be9d-4d60-91ab-56ad7efe9158 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:48.139702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7b686-be9d-4d60-91ab-56ad7efe9158 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:48.140160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7b686-be9d-4d60-91ab-56ad7efe9158 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] PATH_INFO: `/v3/projects/a68f04c73d0c460386e37324fabab1e5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:48.142463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7b686-be9d-4d60-91ab-56ad7efe9158 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorizing `identity:get_project(project_id=a68f04c73d0c460386e37324fabab1e5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:48.145290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7b686-be9d-4d60-91ab-56ad7efe9158 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:48.155355 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1239/2502] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 09:58:47 2022] GET /identity/v3/projects/a68f04c73d0c460386e37324fabab1e5 => generated 393 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:58:48.277444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:48.280246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:48.302792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c08d6-a31d-4d59-b7aa-a25ab1504498 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:48.309700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:48.310832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:48.311548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:48.312005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] PATH_INFO: `/v3/users/ac12036ae0a0427b819ccb5b75a25184` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:48.313963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac12036ae0a0427b819ccb5b75a25184)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:48.317680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:48.427687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac12036ae0a0427b819ccb5b75a25184'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:48.428402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac12036ae0a0427b819ccb5b75a25184'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:48.445101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac12036ae0a0427b819ccb5b75a25184'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:48.461941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac12036ae0a0427b819ccb5b75a25184'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:48.462722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac12036ae0a0427b819ccb5b75a25184'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:48.494238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:48.494881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:48.494999 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1264/2503] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:48 2022] DELETE /identity/v3/users/ac12036ae0a0427b819ccb5b75a25184 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:48.495440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:48.535523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:48.536792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:48.624328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:48.642664 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1240/2504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:48.673753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:48.676703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:48.691760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:48.810071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a564263-4424-4d47-93bd-3159149912c4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:48.823275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:48.823934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:48.824218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:48.824491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:48.853744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:48.858372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:48.889054 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1265/2505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:58:49.041325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:49.043328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:49.061970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804de250-d305-4f39-a51e-bad9e2fb9d3c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:49.068472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:49.069615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:49.070173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:49.071362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] PATH_INFO: `/v3/projects/dd453b58a0d84fa7a04c715e7e35876b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:49.073794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd453b58a0d84fa7a04c715e7e35876b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:49.076864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:49.229623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd453b58a0d84fa7a04c715e7e35876b', 'request_id': 'req-727b466d-391c-4d6d-92f4-3564321ed0d3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.244495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd453b58a0d84fa7a04c715e7e35876b', 'request_id': 'req-727b466d-391c-4d6d-92f4-3564321ed0d3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.265832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd453b58a0d84fa7a04c715e7e35876b', 'request_id': 'req-727b466d-391c-4d6d-92f4-3564321ed0d3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.282616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] The token cache is being invalidate because project dd453b58a0d84fa7a04c715e7e35876b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:49.283218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.284354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.286993 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1241/2506] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:49 2022] DELETE /identity/v3/projects/dd453b58a0d84fa7a04c715e7e35876b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:49.292384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:49.303591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb56083-8b8a-41f4-bb9e-d28242f91cd1 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:49.439575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:49.440416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:49.440731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:49.441011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] PATH_INFO: `/v3/users/bbd0e26448d14160b09c9523e16e9e31` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:49.442526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd0e26448d14160b09c9523e16e9e31)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:49.444460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:49.535652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd0e26448d14160b09c9523e16e9e31'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.536822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd0e26448d14160b09c9523e16e9e31'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.566602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd0e26448d14160b09c9523e16e9e31'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.587063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd0e26448d14160b09c9523e16e9e31'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.588375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd0e26448d14160b09c9523e16e9e31'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:49.636594 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1266/2507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:49 2022] DELETE /identity/v3/users/bbd0e26448d14160b09c9523e16e9e31 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:51.074673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:51.076297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:51.103595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b466d-391c-4d6d-92f4-3564321ed0d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:51.223289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:51.224509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:51.224965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:51.225577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] PATH_INFO: `/v3/projects/ed5453928ec84da48914a387ca695902` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:51.227557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed5453928ec84da48914a387ca695902)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:51.230397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:51.292932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:51.295900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:51.351779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed5453928ec84da48914a387ca695902', 'request_id': 'req-d008139b-6138-4bd4-a6ac-c99af5782d44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.354316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba10f5-cd29-4ba6-a641-7b095b702ded admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:51.362663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed5453928ec84da48914a387ca695902', 'request_id': 'req-d008139b-6138-4bd4-a6ac-c99af5782d44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.376516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed5453928ec84da48914a387ca695902', 'request_id': 'req-d008139b-6138-4bd4-a6ac-c99af5782d44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.396450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] The token cache is being invalidate because project ed5453928ec84da48914a387ca695902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:51.396926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.397839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.402420 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1242/2508] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:51 2022] DELETE /identity/v3/projects/ed5453928ec84da48914a387ca695902 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:51.406611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:51.408395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:51.418144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:51.421138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:51.486910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:51.487835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:51.488100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:51.488827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] PATH_INFO: `/v3/projects/8188b85342dd4696a5ba267dfa4ab682` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:51.490005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8188b85342dd4696a5ba267dfa4ab682)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:51.493036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:51.597957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008139b-6138-4bd4-a6ac-c99af5782d44 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:51.656231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8188b85342dd4696a5ba267dfa4ab682', 'request_id': 'req-feef5330-b0c0-4fcb-a962-a12f20babda3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.673301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8188b85342dd4696a5ba267dfa4ab682', 'request_id': 'req-feef5330-b0c0-4fcb-a962-a12f20babda3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.705079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8188b85342dd4696a5ba267dfa4ab682', 'request_id': 'req-feef5330-b0c0-4fcb-a962-a12f20babda3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.739585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] The token cache is being invalidate because project 8188b85342dd4696a5ba267dfa4ab682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:51.739952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.740743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-feef5330-b0c0-4fcb-a962-a12f20babda3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.750454 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1267/2509] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:51 2022] DELETE /identity/v3/projects/8188b85342dd4696a5ba267dfa4ab682 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:51.757828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c728d6-1a01-42eb-b360-20889b9d12d8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:51.758033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c728d6-1a01-42eb-b360-20889b9d12d8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:51.758368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c728d6-1a01-42eb-b360-20889b9d12d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:51.818847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-97c728d6-1a01-42eb-b360-20889b9d12d8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:51.820121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-97c728d6-1a01-42eb-b360-20889b9d12d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:51.834485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:51.835471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:51.835471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:51.835895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] PATH_INFO: `/v3/users/107c3429510a42ae969e20671542338d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:51.837554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=107c3429510a42ae969e20671542338d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:51.846992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:51.855237 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1268/2510] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:51.857373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2898ad-da46-4732-b1d0-674e05b9a488 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:51.858717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2898ad-da46-4732-b1d0-674e05b9a488 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:51.859097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2898ad-da46-4732-b1d0-674e05b9a488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:51.919362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2a2898ad-da46-4732-b1d0-674e05b9a488 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:51.923029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2a2898ad-da46-4732-b1d0-674e05b9a488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:51.948046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '107c3429510a42ae969e20671542338d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.950307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '107c3429510a42ae969e20671542338d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.957450 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1269/2511] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:51.959305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2898ad-da46-4732-b1d0-674e05b9a488 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:51.971511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '107c3429510a42ae969e20671542338d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.982154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2898ad-da46-4732-b1d0-674e05b9a488 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:51.990610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:51.991762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '107c3429510a42ae969e20671542338d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:51.992294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:51.992983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:51.993472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:51.994421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:51.995032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:51.996884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:52.021825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '107c3429510a42ae969e20671542338d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:52.024820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1270/2512] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:52.029778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:52.033745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:52.038171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1243/2513] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:51 2022] DELETE /identity/v3/users/107c3429510a42ae969e20671542338d => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:52.069008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6df27-c218-4a6f-9e52-cc2f2fcd68cb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:52.138869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:52.139983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:52.140500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:52.141168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:52.142210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:52.144669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:52.189815 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1271/2514] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:52 2022] POST /identity/v3/projects => generated 391 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:52.195074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:52.225432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c73a3-4795-4ead-9c34-c76b17427c56 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:52.308972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:52.385544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:52.386389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:52.386955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:52.387238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:52.387931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:52.393194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:52.457569 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1244/2515] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:52 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:52.461745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:52.478511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1420-a0b0-447c-b2fe-ea7ad9ea7c3e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:52.493053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:52.493839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:52.494208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:52.494611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:52.495585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:52.498372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:52.525239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:52.530602 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1272/2516] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:52.543830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc62d7-f759-4956-9c46-f38295c2f1fd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:52.555397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ec10a-1479-4766-84ab-e9896d658817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:52.556211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec10a-1479-4766-84ab-e9896d658817 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:52.557156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec10a-1479-4766-84ab-e9896d658817 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:52.557156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec10a-1479-4766-84ab-e9896d658817 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:52.557782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ec10a-1479-4766-84ab-e9896d658817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:52.559525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ec10a-1479-4766-84ab-e9896d658817 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:52.580552 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1245/2517] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:52.585564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:52.612185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8be0-175b-47b8-a1d9-8bc1edba0d1f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:52.618136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:52.620257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:52.620687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:52.621081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] PATH_INFO: `/v3/projects/9ead2536c2b24ffda4fb8466602496f4/users/17c11c82b72949498a9639115c907157/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:52.629786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:52.668276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ead2536c2b24ffda4fb8466602496f4, user_id=17c11c82b72949498a9639115c907157, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:52.671201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:52.714495 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1273/2518] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:52 2022] PUT /identity/v3/projects/9ead2536c2b24ffda4fb8466602496f4/users/17c11c82b72949498a9639115c907157/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:52.717197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289d875-1fac-405d-ab5b-a649fa6575f3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:52.717602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289d875-1fac-405d-ab5b-a649fa6575f3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:52.717983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289d875-1fac-405d-ab5b-a649fa6575f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:52.778902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0289d875-1fac-405d-ab5b-a649fa6575f3 None None] MFA Rules not processed for user `17c11c82b72949498a9639115c907157`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:52.781105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0289d875-1fac-405d-ab5b-a649fa6575f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:52.931028 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1246/2519] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:52.935578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:53.056956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed561-ebe0-40c2-b55f-ef629f0dc2b1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:53.070854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:53.071417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:53.071699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:53.072112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:53.072618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:53.075199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:53.132484 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1274/2520] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:52 2022] POST /identity/v3/projects => generated 391 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:53.136384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289d875-1fac-405d-ab5b-a649fa6575f3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:53.168792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289d875-1fac-405d-ab5b-a649fa6575f3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:53.296087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:53.296746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:53.301341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:53.301489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:53.302180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:53.309632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:53.342999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:53.344967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:53.388246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3598c26-b3d6-4f26-8e21-7f909b28b089 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:53.399979 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1247/2521] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:53 2022] POST /identity/v3/users => generated 302 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:53.401041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:53.410163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:53.410820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:53.411095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:53.411368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] PATH_INFO: `/v3/projects/38e89dbee95b4e308e77acde9c3a4dfc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:53.414145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e89dbee95b4e308e77acde9c3a4dfc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:53.416502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:53.447462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382b912-9a31-4afe-80c7-461a59d82e8b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:53.477531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:53.478262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:53.478539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:53.478808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:53.479543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:53.481200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:53.515223 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1248/2522] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:53.516769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:53.535575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855d86-d3df-4bca-bfc0-bff71a455b09 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:53.547603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e89dbee95b4e308e77acde9c3a4dfc', 'request_id': 'req-ee91f320-ea13-4d6a-ae57-103c7f3e4445'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:53.568296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:53.579509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e89dbee95b4e308e77acde9c3a4dfc', 'request_id': 'req-ee91f320-ea13-4d6a-ae57-103c7f3e4445'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:53.594242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e89dbee95b4e308e77acde9c3a4dfc', 'request_id': 'req-ee91f320-ea13-4d6a-ae57-103c7f3e4445'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:53.641093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] The token cache is being invalidate because project 38e89dbee95b4e308e77acde9c3a4dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:53.641435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:53.643397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:53.647598 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1275/2523] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:53 2022] DELETE /identity/v3/projects/38e89dbee95b4e308e77acde9c3a4dfc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:53.651724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:53.653901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:53.681492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:53.682186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:53.682462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:53.682732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] PATH_INFO: `/v3/projects/9867d8b59ef7448998e0e05ffc5180bf/users/14b35bdf82e847c690f7f8754bc4fb15/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:53.724265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9867d8b59ef7448998e0e05ffc5180bf, user_id=14b35bdf82e847c690f7f8754bc4fb15, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:53.727080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:53.766089 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1249/2524] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:53 2022] PUT /identity/v3/projects/9867d8b59ef7448998e0e05ffc5180bf/users/14b35bdf82e847c690f7f8754bc4fb15/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:53.770367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:53.771541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:53.781938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:53.809650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91f320-ea13-4d6a-ae57-103c7f3e4445 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:53.923806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4d661-2ddd-4c45-8c44-6956f80731b1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:53.937536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:53.938174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:53.938449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:53.938720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:53.939580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:53.943697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:53.944769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:53.945274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:53.945590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] PATH_INFO: `/v3/users/950d2e8d194040be9bc5cfd08b2e0156` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:53.947040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:53.949162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=950d2e8d194040be9bc5cfd08b2e0156)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:53.953116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:53.983153 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1250/2525] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:58:53.987272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:54.011070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cf2a6-7a65-4c1a-bd05-314c04d0ed1b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:54.059703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290d56e-0d0f-4f56-a120-97f05e7b800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:54.060402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290d56e-0d0f-4f56-a120-97f05e7b800f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:54.060714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290d56e-0d0f-4f56-a120-97f05e7b800f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:54.061032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290d56e-0d0f-4f56-a120-97f05e7b800f admin admin] PATH_INFO: `/v3/projects/9867d8b59ef7448998e0e05ffc5180bf/users/14b35bdf82e847c690f7f8754bc4fb15/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:54.064225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290d56e-0d0f-4f56-a120-97f05e7b800f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9867d8b59ef7448998e0e05ffc5180bf, user_id=14b35bdf82e847c690f7f8754bc4fb15, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:54.066910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290d56e-0d0f-4f56-a120-97f05e7b800f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:54.095503 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1251/2526] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:53 2022] PUT /identity/v3/projects/9867d8b59ef7448998e0e05ffc5180bf/users/14b35bdf82e847c690f7f8754bc4fb15/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:54.106270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bb4e5-473f-443d-a28f-096124c8504a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:54.107278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bb4e5-473f-443d-a28f-096124c8504a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:54.107278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bb4e5-473f-443d-a28f-096124c8504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:54.117049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950d2e8d194040be9bc5cfd08b2e0156'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:54.119996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950d2e8d194040be9bc5cfd08b2e0156'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:54.148358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950d2e8d194040be9bc5cfd08b2e0156'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:54.173469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950d2e8d194040be9bc5cfd08b2e0156'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:54.175736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950d2e8d194040be9bc5cfd08b2e0156'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:54.222621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-413bb4e5-473f-443d-a28f-096124c8504a None None] MFA Rules not processed for user `14b35bdf82e847c690f7f8754bc4fb15`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:54.223742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-413bb4e5-473f-443d-a28f-096124c8504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:54.223901 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1276/2527] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:53 2022] DELETE /identity/v3/users/950d2e8d194040be9bc5cfd08b2e0156 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:54.372054 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1252/2528] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:54.563387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:54.564874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:54.663824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9a809-bc80-4716-b656-f05116a46dc2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:54.671993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe346b9-6fa6-42e5-b2c4-89551dbbeec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:54.673043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe346b9-6fa6-42e5-b2c4-89551dbbeec4 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:54.673501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe346b9-6fa6-42e5-b2c4-89551dbbeec4 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:54.673788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe346b9-6fa6-42e5-b2c4-89551dbbeec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:54.675810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bfe346b9-6fa6-42e5-b2c4-89551dbbeec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:54.691044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:54.754099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe346b9-6fa6-42e5-b2c4-89551dbbeec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:54.755953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe346b9-6fa6-42e5-b2c4-89551dbbeec4 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:54.767546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1277/2529] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:54.887317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca129c-42ef-4751-98f4-7cd4519bb4d5 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:54.887586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca129c-42ef-4751-98f4-7cd4519bb4d5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:54.887868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca129c-42ef-4751-98f4-7cd4519bb4d5 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:54.889238 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1253/2530] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:54.960431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687dc24-b34e-49de-be6d-08083585083a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:54.960792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687dc24-b34e-49de-be6d-08083585083a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:54.961125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687dc24-b34e-49de-be6d-08083585083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:54.992730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a687dc24-b34e-49de-be6d-08083585083a None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:54.993931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a687dc24-b34e-49de-be6d-08083585083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:55.106064 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1278/2531] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:58:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:55.157894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca129c-42ef-4751-98f4-7cd4519bb4d5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:55.160525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-45ca129c-42ef-4751-98f4-7cd4519bb4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:55.165551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687dc24-b34e-49de-be6d-08083585083a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:55.167558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a687dc24-b34e-49de-be6d-08083585083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:55.201636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca129c-42ef-4751-98f4-7cd4519bb4d5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:55.210911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:55.225912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:55.227959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:55.228738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:55.257269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:55.265246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:55.270673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:55.295806 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1254/2532] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:55.405912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687dc24-b34e-49de-be6d-08083585083a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:55.413131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:55.413755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:55.414146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:55.414424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] PATH_INFO: `/v3/users/0320d122f53c435588a69baafb9bcf98` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:55.415604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0320d122f53c435588a69baafb9bcf98)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:55.417985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:55.465842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:55.467630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:55.495510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0320d122f53c435588a69baafb9bcf98'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.495776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374d4e-8630-458e-a642-b524bad334ff service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:55.496216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0320d122f53c435588a69baafb9bcf98'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.501997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:55.507192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:55.508063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:55.508336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:55.508799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] PATH_INFO: `/v3/projects/a68f04c73d0c460386e37324fabab1e5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:55.510573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68f04c73d0c460386e37324fabab1e5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:55.512160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0320d122f53c435588a69baafb9bcf98'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.513995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:55.525312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0320d122f53c435588a69baafb9bcf98'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.526115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b79d1830-9a65-491a-a59c-ba39dce86041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0320d122f53c435588a69baafb9bcf98'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.562605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1279/2533] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:55 2022] DELETE /identity/v3/users/0320d122f53c435588a69baafb9bcf98 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:55.633724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68f04c73d0c460386e37324fabab1e5', 'request_id': 'req-d5eebb6d-65a8-4822-9d90-65e35fc777eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.648689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68f04c73d0c460386e37324fabab1e5', 'request_id': 'req-d5eebb6d-65a8-4822-9d90-65e35fc777eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.669596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68f04c73d0c460386e37324fabab1e5', 'request_id': 'req-d5eebb6d-65a8-4822-9d90-65e35fc777eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.694650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] The token cache is being invalidate because project a68f04c73d0c460386e37324fabab1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:55.694947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.700219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:55.701877 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1255/2534] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:55 2022] DELETE /identity/v3/projects/a68f04c73d0c460386e37324fabab1e5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:55.708524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2e138-2e0b-4ae2-a5a2-b66d463566ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:55.709096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2e138-2e0b-4ae2-a5a2-b66d463566ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:55.709096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2e138-2e0b-4ae2-a5a2-b66d463566ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:55.744450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-98d2e138-2e0b-4ae2-a5a2-b66d463566ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:55.746595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-98d2e138-2e0b-4ae2-a5a2-b66d463566ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:55.789280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:55.791324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:55.874098 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1280/2535] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:55.882533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6d5b8-ea4d-4f4a-adc1-6263f0cbc9df None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:55.882895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6d5b8-ea4d-4f4a-adc1-6263f0cbc9df None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:55.883168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6d5b8-ea4d-4f4a-adc1-6263f0cbc9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:55.915789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebb6d-65a8-4822-9d90-65e35fc777eb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:55.922177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:55.925241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:55.925702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:55.926139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] PATH_INFO: `/v3/users/d67531a4fd4040a5b824e805c685eab0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:55.927648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d67531a4fd4040a5b824e805c685eab0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:55.931138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:55.934497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-acc6d5b8-ea4d-4f4a-adc1-6263f0cbc9df None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:55.935737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-acc6d5b8-ea4d-4f4a-adc1-6263f0cbc9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:55.957668 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1281/2536] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:55.962331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6d5b8-ea4d-4f4a-adc1-6263f0cbc9df None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:55.984690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6d5b8-ea4d-4f4a-adc1-6263f0cbc9df None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:56.014697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:56.016289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:56.017282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:56.017806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:56.019507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:56.022527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:56.034657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:56.072160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd67531a4fd4040a5b824e805c685eab0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.073560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd67531a4fd4040a5b824e805c685eab0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.094902 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1282/2537] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:58:56.099836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:56.102563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:56.119568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd67531a4fd4040a5b824e805c685eab0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.143928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7479dc-f85e-44c2-9c72-c90df1911e46 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:56.146179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd67531a4fd4040a5b824e805c685eab0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.165494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950774b3-5e27-4c03-92d3-9c7a0b653737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:56.166443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950774b3-5e27-4c03-92d3-9c7a0b653737 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:56.168419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950774b3-5e27-4c03-92d3-9c7a0b653737 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:56.168945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950774b3-5e27-4c03-92d3-9c7a0b653737 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:56.169824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950774b3-5e27-4c03-92d3-9c7a0b653737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:56.175082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950774b3-5e27-4c03-92d3-9c7a0b653737 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:56.185379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd67531a4fd4040a5b824e805c685eab0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.196686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:56.205833 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1256/2538] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:55 2022] DELETE /identity/v3/users/d67531a4fd4040a5b824e805c685eab0 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:58:56.257488 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1283/2539] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:56 2022] POST /identity/v3/projects => generated 397 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:58:56.261170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:56.384434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12609e-bf54-4546-9e71-d0549518f0d5 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:56.384904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12609e-bf54-4546-9e71-d0549518f0d5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:56.385265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12609e-bf54-4546-9e71-d0549518f0d5 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:56.386796 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1284/2540] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 09:58:56.392621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:56.393044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:56.395462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:56.407523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184440-f0bd-471d-af91-dc1849dee979 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:56.424942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:56.433407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:56.538306 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1285/2541] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:58:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 27 09:58:56.540203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:56.544059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:56.571021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:56.571749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:56.572031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:56.574935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97906e-e52e-4c01-8f8a-5932db52da7a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:56.580741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:56.595623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:56.604646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:56.610855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:56.618372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:56.619459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:56.619459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] PATH_INFO: `/v3/projects/7476bff780f0442e9be3b2c4ff4b7e41` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:56.623968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7476bff780f0442e9be3b2c4ff4b7e41)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:56.627937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:56.727644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1257/2542] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:56 2022] POST /identity/v3/users => generated 299 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:56.730378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:56.733613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:56.734945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:56.758587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7476bff780f0442e9be3b2c4ff4b7e41', 'request_id': 'req-1183de40-ce1c-45a5-9d61-bd0b0d652a68'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.782550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c69388-f2a8-41ea-b00d-8c6ac40594aa admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:56.794025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7476bff780f0442e9be3b2c4ff4b7e41', 'request_id': 'req-1183de40-ce1c-45a5-9d61-bd0b0d652a68'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.802200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7476bff780f0442e9be3b2c4ff4b7e41', 'request_id': 'req-1183de40-ce1c-45a5-9d61-bd0b0d652a68'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.818431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] The token cache is being invalidate because project 7476bff780f0442e9be3b2c4ff4b7e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:56.819143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.820063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:56.822092 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1286/2543] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:56 2022] DELETE /identity/v3/projects/7476bff780f0442e9be3b2c4ff4b7e41 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:56.823045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:56.825177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:56.920284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216e969-7027-4c2f-a869-8a3b2df86bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:56.921374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216e969-7027-4c2f-a869-8a3b2df86bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:56.921970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216e969-7027-4c2f-a869-8a3b2df86bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:56.922697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216e969-7027-4c2f-a869-8a3b2df86bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:56.926591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2216e969-7027-4c2f-a869-8a3b2df86bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:56.972758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183de40-ce1c-45a5-9d61-bd0b0d652a68 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:56.973708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216e969-7027-4c2f-a869-8a3b2df86bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:56.981949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216e969-7027-4c2f-a869-8a3b2df86bc3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.012908 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1258/2544] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:57.017994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.018771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.022628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003f80-0a30-4bdb-adb5-06008ee846ff None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.023248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003f80-0a30-4bdb-adb5-06008ee846ff None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.023842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003f80-0a30-4bdb-adb5-06008ee846ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.024860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.025146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.028124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:57.036735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.072851 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1287/2545] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:57.075638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.079363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.098232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f2003f80-0a30-4bdb-adb5-06008ee846ff None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:57.100332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f2003f80-0a30-4bdb-adb5-06008ee846ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.118668 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1259/2546] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:57.119743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2003f80-0a30-4bdb-adb5-06008ee846ff None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.123155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0be7dc-d8ed-480a-8fcc-6cfee953441c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:57.131820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2003f80-0a30-4bdb-adb5-06008ee846ff None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:57.138337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a4ad0-4008-4835-a766-2f069cca5ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.139190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4ad0-4008-4835-a766-2f069cca5ddf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.139703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4ad0-4008-4835-a766-2f069cca5ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.140131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4ad0-4008-4835-a766-2f069cca5ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.141141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a4ad0-4008-4835-a766-2f069cca5ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:57.143260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a4ad0-4008-4835-a766-2f069cca5ddf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.147661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.152091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.152579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.153099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] PATH_INFO: `/v3/projects/b07a770a637643a0bdf3fa219a46b756` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.162013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07a770a637643a0bdf3fa219a46b756)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:57.163320 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1260/2547] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:57.165856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015b05c-4c2c-49c6-a4b9-4bb7af9917e8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.166216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015b05c-4c2c-49c6-a4b9-4bb7af9917e8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.166658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015b05c-4c2c-49c6-a4b9-4bb7af9917e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.173667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.195682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1015b05c-4c2c-49c6-a4b9-4bb7af9917e8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:57.199204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1015b05c-4c2c-49c6-a4b9-4bb7af9917e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.230487 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1261/2548] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:57.230487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015b05c-4c2c-49c6-a4b9-4bb7af9917e8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.254503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015b05c-4c2c-49c6-a4b9-4bb7af9917e8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:57.268440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.270443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.271036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.273126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] PATH_INFO: `/v3/projects/2369f5ee467643948f618bbdee837521/users/29d2f7b0e8274c769521d13930be9825/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.303082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:57.316706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2369f5ee467643948f618bbdee837521, user_id=29d2f7b0e8274c769521d13930be9825, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:57.319887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.347806 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1262/2549] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:57 2022] PUT /identity/v3/projects/2369f5ee467643948f618bbdee837521/users/29d2f7b0e8274c769521d13930be9825/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:57.349036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.392240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07a770a637643a0bdf3fa219a46b756', 'request_id': 'req-4d614ff4-c645-4b28-bce4-46926e30f9e1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.405802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07a770a637643a0bdf3fa219a46b756', 'request_id': 'req-4d614ff4-c645-4b28-bce4-46926e30f9e1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.416854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07a770a637643a0bdf3fa219a46b756', 'request_id': 'req-4d614ff4-c645-4b28-bce4-46926e30f9e1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.442109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] The token cache is being invalidate because project b07a770a637643a0bdf3fa219a46b756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:57.443074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.444296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d614ff4-c645-4b28-bce4-46926e30f9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.450322 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1288/2550] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:57 2022] DELETE /identity/v3/projects/b07a770a637643a0bdf3fa219a46b756 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:57.451823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.452106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.452381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.505260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:57.506727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.527831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadde39-f855-4c6e-aaf9-00c0375d7e3f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:57.559119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:57.623520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1289/2551] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:57.624969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.629633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.643636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.645786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.646098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.648384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.650120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:57.650363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2a8a0-c3ca-41bf-a292-0c2832e47de0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:57.650791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:57.653396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.655831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.656611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.656993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.657364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] PATH_INFO: `/v3/users/f353d5a0cabf4c24ac7ee5c9e7f44007` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.658603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f353d5a0cabf4c24ac7ee5c9e7f44007)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:57.661762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.672166 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1263/2552] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:57.672166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.672166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.738802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f353d5a0cabf4c24ac7ee5c9e7f44007'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.740102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f353d5a0cabf4c24ac7ee5c9e7f44007'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.752289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f353d5a0cabf4c24ac7ee5c9e7f44007'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.763299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f353d5a0cabf4c24ac7ee5c9e7f44007'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.764218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f353d5a0cabf4c24ac7ee5c9e7f44007'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:57.779647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b41051-aa26-4ae3-b2da-a6e1814f9bdc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:57.785009 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1290/2553] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:57 2022] DELETE /identity/v3/users/f353d5a0cabf4c24ac7ee5c9e7f44007 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:57.785941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.787423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.799490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.800123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.800400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.800703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.802818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.862552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:57.919814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a15b3a-ea98-44a9-9741-1c2edacaa216 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:57.925658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd57d4a-0668-4188-b4d2-3cb563a9483d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:57.926284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd57d4a-0668-4188-b4d2-3cb563a9483d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:57.926560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd57d4a-0668-4188-b4d2-3cb563a9483d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:57.926890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd57d4a-0668-4188-b4d2-3cb563a9483d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:57.929814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-edd57d4a-0668-4188-b4d2-3cb563a9483d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:57.953079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:57.954915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:57.969443 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1264/2554] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:57.970394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:57.972881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:58.027225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561422d-2ace-4a34-bdde-52beba24c91f service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.034425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd57d4a-0668-4188-b4d2-3cb563a9483d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.036240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd57d4a-0668-4188-b4d2-3cb563a9483d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.040634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:58.042279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.043484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.044222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.045451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.050822 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1291/2555] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:58.051353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.052246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ad8f6-0f22-4a46-b372-1f46a256ff3c None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.052689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ad8f6-0f22-4a46-b372-1f46a256ff3c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.052963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ad8f6-0f22-4a46-b372-1f46a256ff3c None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.056382 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1292/2556] 10.222.0.26 () {54 vars in 921 bytes} [Tue Sep 27 09:58:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 09:58:58.063087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dd850-c375-407b-87d9-7c53b05b0537 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.067090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dd850-c375-407b-87d9-7c53b05b0537 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.067090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dd850-c375-407b-87d9-7c53b05b0537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.082831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:58.099817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4e0dd850-c375-407b-87d9-7c53b05b0537 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:58.102140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4e0dd850-c375-407b-87d9-7c53b05b0537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:58.114869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1265/2557] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:57 2022] POST /identity/v3/projects => generated 409 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:58.121703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:58.202753 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1293/2558] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 09:58:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:58.233486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dd850-c375-407b-87d9-7c53b05b0537 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:58.256467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dd850-c375-407b-87d9-7c53b05b0537 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.326229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8cb2e-62d2-46cf-8b2d-a3746543dced admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.338116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:58.338826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.339108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.339381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.340397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.342374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.377918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc899f1-02f2-4881-a3d9-6e8faffe094d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:58.378570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc899f1-02f2-4881-a3d9-6e8faffe094d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.378852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc899f1-02f2-4881-a3d9-6e8faffe094d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.379128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc899f1-02f2-4881-a3d9-6e8faffe094d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.388971 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1266/2559] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:58 2022] POST /identity/v3/users => generated 305 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:58.390820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:58.395897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:58.426232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91483872-8909-4968-8361-2624d9dca649 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.441756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:58.442924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.442924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.443512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] PATH_INFO: `/v3/projects/00b477951d834dca9ce99392b85d9c16` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.445291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b477951d834dca9ce99392b85d9c16)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.447714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.552051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b477951d834dca9ce99392b85d9c16', 'request_id': 'req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:58.566635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b477951d834dca9ce99392b85d9c16', 'request_id': 'req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:58.570560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc899f1-02f2-4881-a3d9-6e8faffe094d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.575298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc899f1-02f2-4881-a3d9-6e8faffe094d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.584522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b477951d834dca9ce99392b85d9c16', 'request_id': 'req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:58.603712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] The token cache is being invalidate because project 00b477951d834dca9ce99392b85d9c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:58:58.604227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:58.605176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:58:58.610665 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1267/2560] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:58:58 2022] DELETE /identity/v3/projects/00b477951d834dca9ce99392b85d9c16 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:58.610665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:58.610665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:58.620695 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1294/2561] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 09:58:58.622835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae5b31-1d39-4d96-bce6-ce4919c8cb9e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.623136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae5b31-1d39-4d96-bce6-ce4919c8cb9e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.623495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae5b31-1d39-4d96-bce6-ce4919c8cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.638191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:58.657542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a9ae5b31-1d39-4d96-bce6-ce4919c8cb9e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:58.659806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a9ae5b31-1d39-4d96-bce6-ce4919c8cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:58.718240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:58.730902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b1c3e-dd6b-4cfa-8518-e31f1b3f857a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.736647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:58.737516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.737945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.738339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.739153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.741137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.754276 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1268/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:58.761011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:58.777801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f30d1-0ad2-4b31-8a59-6cdf1f0ad74a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.784976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:58.785702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.786048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.786389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.787137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.788937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.810415 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1295/2563] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:58.812170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1269/2564] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:58.813262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:58.815350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75082e-fd54-4c0a-90ec-9952883ed352 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.817469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75082e-fd54-4c0a-90ec-9952883ed352 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.818053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75082e-fd54-4c0a-90ec-9952883ed352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.827272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a057-8530-4008-951f-15a9bd2ccf3a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.834556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fac44-565f-4e0c-adcb-d196bd5ff003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:58.835346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fac44-565f-4e0c-adcb-d196bd5ff003 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.835763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fac44-565f-4e0c-adcb-d196bd5ff003 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.836016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fac44-565f-4e0c-adcb-d196bd5ff003 admin admin] PATH_INFO: `/v3/projects/2b3380f3fa2b494da649aaa809e11368/users/57365b62b3294810b8a030d2a26cc451/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.860455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fac44-565f-4e0c-adcb-d196bd5ff003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3380f3fa2b494da649aaa809e11368, user_id=57365b62b3294810b8a030d2a26cc451, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:58.861202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ce75082e-fd54-4c0a-90ec-9952883ed352 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:58.862814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ce75082e-fd54-4c0a-90ec-9952883ed352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:58.864301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fac44-565f-4e0c-adcb-d196bd5ff003 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:58.876715 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1296/2565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:58.879417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75082e-fd54-4c0a-90ec-9952883ed352 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:58.891785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75082e-fd54-4c0a-90ec-9952883ed352 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:58.898517 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1270/2566] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:58 2022] PUT /identity/v3/projects/2b3380f3fa2b494da649aaa809e11368/users/57365b62b3294810b8a030d2a26cc451/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:58.900968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:58.901399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:58.901680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:58.970454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:58.972229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:58.999053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.002145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.003527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.003813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.004487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:58:59.004824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.013108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.048161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1297/2567] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:58:59.057929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:59.071445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a140e-3c7d-4219-8e16-e1b946280c58 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:59.077699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.078320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.078595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.078863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.079685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.081912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.128710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1298/2568] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:58:59 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:59.130421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:59.134500 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1271/2569] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:59.168696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:59.170796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:59.267592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5052f56-bc2b-4387-a119-e073991e0987 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:59.277430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.278562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.279024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.279468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.280396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.283184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.318263 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1299/2570] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:58:59 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:58:59.324845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dac9c-0333-4f7b-b32b-3e98bd426770 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:59.325790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:59.340771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67551abf-15d2-4781-bec6-2f3c46f61971 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:59.354993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.355437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.355725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.355995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.357908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.361892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.388800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1300/2571] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:59.389471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:59.398561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:59.414671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fda11-c824-419e-95b2-7ac6664f142c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:59.424295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.424956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.425236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.425502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.426407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.431904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.444120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a411bc6-bbfc-4f09-96d5-ae5e9442735f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.444762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411bc6-bbfc-4f09-96d5-ae5e9442735f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.445047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411bc6-bbfc-4f09-96d5-ae5e9442735f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.445418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411bc6-bbfc-4f09-96d5-ae5e9442735f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.465889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1301/2572] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:58:59.466884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:59.473519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a411bc6-bbfc-4f09-96d5-ae5e9442735f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.475001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a411bc6-bbfc-4f09-96d5-ae5e9442735f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.490265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2e243-ab3f-4af4-8b4d-e60ae32819c8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:59.498140 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1272/2573] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:59.500507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.501644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.501644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.501850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] PATH_INFO: `/v3/projects/ec0980b5824b425fbad9b8554cf5c42c/users/dce01d3212934161a38be853b4cd93ae/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.506332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:59.526328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0980b5824b425fbad9b8554cf5c42c, user_id=dce01d3212934161a38be853b4cd93ae, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.534161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.565041 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1302/2574] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:58:59 2022] PUT /identity/v3/projects/ec0980b5824b425fbad9b8554cf5c42c/users/dce01d3212934161a38be853b4cd93ae/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:58:59.571896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.572305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.572777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.607752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:58:59.610700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:59.711473 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1273/2575] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:58:59.731924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:58:59.734276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:58:59.831532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac509984-e3a0-4ece-b995-b35d779deaf9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:58:59.837805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c88c8-6bb0-4731-b57a-67392d8e9404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:58:59.838225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c88c8-6bb0-4731-b57a-67392d8e9404 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:58:59.839724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c88c8-6bb0-4731-b57a-67392d8e9404 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:58:59.840419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c88c8-6bb0-4731-b57a-67392d8e9404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:58:59.853957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c88c8-6bb0-4731-b57a-67392d8e9404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:58:59.855627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c88c8-6bb0-4731-b57a-67392d8e9404 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:58:59.867484 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1303/2576] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:08.253592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:08.254789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:08.471133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3422eb5-2dcb-4d2f-a5c0-3b796b9ba633 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:08.501088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:08.501718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:08.502001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:08.502275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:08.506377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:08.551731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:08.553406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:08.598238 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1274/2577] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:08.800485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d39dc5-6e60-4e46-a55d-f1ed5ac6b9e4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:08.801085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d39dc5-6e60-4e46-a55d-f1ed5ac6b9e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:08.801085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d39dc5-6e60-4e46-a55d-f1ed5ac6b9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:08.897005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-54d39dc5-6e60-4e46-a55d-f1ed5ac6b9e4 None None] MFA Rules not processed for user `29d2f7b0e8274c769521d13930be9825`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:08.904550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-54d39dc5-6e60-4e46-a55d-f1ed5ac6b9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:09.086710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1304/2578] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:08 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:09.091915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:09.093828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:09.125052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d214dd-b31e-406b-8d15-7c5c9b662e6b service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:09.140365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:09.141050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:09.142290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:09.142635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:09.143293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:09.149257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:09.207845 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1275/2579] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:59:09 2022] POST /identity/v3/projects => generated 397 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:09.212221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d39dc5-6e60-4e46-a55d-f1ed5ac6b9e4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:09.228364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d39dc5-6e60-4e46-a55d-f1ed5ac6b9e4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:09.333556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:09.335173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:09.335173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:09.335173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:09.336052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:09.339436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:09.370835 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1305/2580] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:59:09 2022] POST /identity/v3/users => generated 305 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:09.374834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:09.406647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39167c01-b6a0-4295-8c18-cd9da28b1445 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:09.428094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:09.428773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:09.429851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:09.429851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:09.433843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:09.446291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:09.488825 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1276/2581] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:09.493322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:09.510433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc660375-df44-4590-a47c-8dfaf4a47f74 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:09.529776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:09.530623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:09.530997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:09.531364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] PATH_INFO: `/v3/projects/0ad373626b794b5482182e11b5d83183/users/f495627b28c44a979882d447f73d97fd/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:09.548309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:09.597713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad373626b794b5482182e11b5d83183, user_id=f495627b28c44a979882d447f73d97fd, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:09.603987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:09.639968 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1306/2582] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:09 2022] PUT /identity/v3/projects/0ad373626b794b5482182e11b5d83183/users/f495627b28c44a979882d447f73d97fd/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:09.643990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:09.664414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814e53-4670-4515-8b78-eb96b469c323 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:09.748876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:09.821087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:09.822238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:09.822532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:09.822802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:09.825148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:09.831100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:09.860142 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1277/2583] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:09.861936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:09.889952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fc922-a090-4cfc-a038-a06ac7eb2d5f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:09.900550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca1ac3-e060-4b06-9cce-9d2292d701fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:09.901385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca1ac3-e060-4b06-9cce-9d2292d701fd admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:09.901666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca1ac3-e060-4b06-9cce-9d2292d701fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:09.902984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca1ac3-e060-4b06-9cce-9d2292d701fd admin admin] PATH_INFO: `/v3/projects/0ad373626b794b5482182e11b5d83183/users/f495627b28c44a979882d447f73d97fd/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:09.905954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca1ac3-e060-4b06-9cce-9d2292d701fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad373626b794b5482182e11b5d83183, user_id=f495627b28c44a979882d447f73d97fd, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:09.908999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca1ac3-e060-4b06-9cce-9d2292d701fd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:09.933541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1307/2584] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:09 2022] PUT /identity/v3/projects/0ad373626b794b5482182e11b5d83183/users/f495627b28c44a979882d447f73d97fd/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:09.933843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:09.935702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:09.971227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293cb31-36c6-4c00-9adf-ff23fd80cda9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:10.154578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:10.155968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:10.157528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:10.158413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] PATH_INFO: `/v3/users/aa2474e7686c4a758ed93064edf47867` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:10.160474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa2474e7686c4a758ed93064edf47867)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:10.163776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:10.294375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa2474e7686c4a758ed93064edf47867'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:10.295178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa2474e7686c4a758ed93064edf47867'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:10.311493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa2474e7686c4a758ed93064edf47867'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:10.343583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa2474e7686c4a758ed93064edf47867'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:10.374891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa2474e7686c4a758ed93064edf47867'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:10.398857 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1278/2585] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:59:09 2022] DELETE /identity/v3/users/aa2474e7686c4a758ed93064edf47867 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:10.403858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:10.404497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:10.404895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:10.440190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:10.442037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:10.616458 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1308/2586] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:59:10.653622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:10.657060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:10.793611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:10.805984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e5380-37d3-40c7-a4cb-40af50f4a682 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:10.817189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4af96-1387-4118-ad01-d25cc75f7569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:10.818497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4af96-1387-4118-ad01-d25cc75f7569 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:10.818794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4af96-1387-4118-ad01-d25cc75f7569 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:10.819082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4af96-1387-4118-ad01-d25cc75f7569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:10.849899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4af96-1387-4118-ad01-d25cc75f7569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:10.852354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4af96-1387-4118-ad01-d25cc75f7569 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:10.879842 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1279/2587] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:11.461512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:11.466199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:11.506595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e3685-a409-4afe-b37d-6f37d49c11a7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:11.521131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:11.523218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:11.523540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:11.523904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] PATH_INFO: `/v3/users/17c11c82b72949498a9639115c907157` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:11.527740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c11c82b72949498a9639115c907157)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:11.532055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:11.691501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c11c82b72949498a9639115c907157'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:11.693815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c11c82b72949498a9639115c907157'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:11.724712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c11c82b72949498a9639115c907157'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:11.742716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c11c82b72949498a9639115c907157'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:11.744562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c11c82b72949498a9639115c907157'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:11.800698 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1309/2588] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:59:11 2022] DELETE /identity/v3/users/17c11c82b72949498a9639115c907157 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:11.806323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:11.808206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:11.808914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:11.838564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:11.840217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:12.018116 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1280/2589] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:12.066209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:12.068106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:12.106630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:12.108827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:12.134652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e14b4-14c9-4ae4-b653-937b88f7126c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:12.155217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:12.155947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:12.156762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:12.157949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] PATH_INFO: `/v3/projects/7e6987197b0a4c7d9b89d48402845f3f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:12.162453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6987197b0a4c7d9b89d48402845f3f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:12.166597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:12.210263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:12.238494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102e9e-7476-4664-88bc-1ac593cb9964 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:12.247297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:12.247934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:12.248323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:12.248543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:12.271030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:12.272564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:12.285076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6987197b0a4c7d9b89d48402845f3f', 'request_id': 'req-0811a21b-4f0a-4732-a54e-6dc8a10263c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.296881 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1310/2590] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:12.307276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6987197b0a4c7d9b89d48402845f3f', 'request_id': 'req-0811a21b-4f0a-4732-a54e-6dc8a10263c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.322622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6987197b0a4c7d9b89d48402845f3f', 'request_id': 'req-0811a21b-4f0a-4732-a54e-6dc8a10263c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.339832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] The token cache is being invalidate because project 7e6987197b0a4c7d9b89d48402845f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:59:12.339832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.340512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0811a21b-4f0a-4732-a54e-6dc8a10263c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.347298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:12.354260 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1281/2591] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:59:12 2022] DELETE /identity/v3/projects/7e6987197b0a4c7d9b89d48402845f3f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 09:59:12.361254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d6a01-25f2-47af-a716-277386c999a2 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:12.451322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:12.452003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:12.453037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:12.453157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] PATH_INFO: `/v3/users/b3e38e5269224ed2aed7bf6b00141a14` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:12.457014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e38e5269224ed2aed7bf6b00141a14)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:12.458534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:12.637494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e38e5269224ed2aed7bf6b00141a14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.638653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e38e5269224ed2aed7bf6b00141a14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.638890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79c515-2b5a-4e23-b3e0-f13c7c718719 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:12.639177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79c515-2b5a-4e23-b3e0-f13c7c718719 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:12.639451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79c515-2b5a-4e23-b3e0-f13c7c718719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:12.667991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cc79c515-2b5a-4e23-b3e0-f13c7c718719 None None] MFA Rules not processed for user `00423eb1e4c24763ae0a546225ac1dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:12.670929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e38e5269224ed2aed7bf6b00141a14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.677760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cc79c515-2b5a-4e23-b3e0-f13c7c718719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:12.682341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e38e5269224ed2aed7bf6b00141a14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.683038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e38e5269224ed2aed7bf6b00141a14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:12.742572 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1311/2592] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:59:12 2022] DELETE /identity/v3/users/b3e38e5269224ed2aed7bf6b00141a14 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:12.885312 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1282/2593] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:59:13.911320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:13.912902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:14.095478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfb347-3576-41cb-8d52-1e0b123c148a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:14.104134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:14.104853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:14.105136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:14.105413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] PATH_INFO: `/v3/projects/9ead2536c2b24ffda4fb8466602496f4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:14.116134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ead2536c2b24ffda4fb8466602496f4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:14.119149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:14.136403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:14.243793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ead2536c2b24ffda4fb8466602496f4', 'request_id': 'req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.262865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ead2536c2b24ffda4fb8466602496f4', 'request_id': 'req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.278160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ead2536c2b24ffda4fb8466602496f4', 'request_id': 'req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.306179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] The token cache is being invalidate because project 9ead2536c2b24ffda4fb8466602496f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:59:14.306648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.308375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.316331 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1312/2594] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:59:13 2022] DELETE /identity/v3/projects/9ead2536c2b24ffda4fb8466602496f4 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:14.317684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79c515-2b5a-4e23-b3e0-f13c7c718719 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:14.340958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79c515-2b5a-4e23-b3e0-f13c7c718719 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:14.369228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:14.440023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:14.442129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:14.442672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:14.443931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] PATH_INFO: `/v3/users/14b35bdf82e847c690f7f8754bc4fb15` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:14.446020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b35bdf82e847c690f7f8754bc4fb15)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:14.448304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:14.572276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b35bdf82e847c690f7f8754bc4fb15'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.575323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b35bdf82e847c690f7f8754bc4fb15'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.601317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b35bdf82e847c690f7f8754bc4fb15'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.624719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b35bdf82e847c690f7f8754bc4fb15'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.657857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b48b6503-7aa6-4260-8bce-3d8d83e2ec5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b35bdf82e847c690f7f8754bc4fb15'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:14.688189 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1283/2595] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:59:14 2022] DELETE /identity/v3/users/14b35bdf82e847c690f7f8754bc4fb15 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:15.178292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:15.181330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:15.207146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de21fcb-2b0d-4569-b928-03bb8ece0b7e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:15.362383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:15.363130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:15.363481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:15.363819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] PATH_INFO: `/v3/projects/4d2d9d09939f4d6fb8d0a9f4c9068813` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:15.365078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2d9d09939f4d6fb8d0a9f4c9068813)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:15.367254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:15.481335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2d9d09939f4d6fb8d0a9f4c9068813', 'request_id': 'req-6a73c932-29d2-4d53-9159-aeb3b7fdd072'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:15.497679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2d9d09939f4d6fb8d0a9f4c9068813', 'request_id': 'req-6a73c932-29d2-4d53-9159-aeb3b7fdd072'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:15.543544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2d9d09939f4d6fb8d0a9f4c9068813', 'request_id': 'req-6a73c932-29d2-4d53-9159-aeb3b7fdd072'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:15.601021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] The token cache is being invalidate because project 4d2d9d09939f4d6fb8d0a9f4c9068813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:59:15.601987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:15.603321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a73c932-29d2-4d53-9159-aeb3b7fdd072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:15.605691 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1313/2596] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:59:15 2022] DELETE /identity/v3/projects/4d2d9d09939f4d6fb8d0a9f4c9068813 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:15.610602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:15.612465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:15.612982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:15.648324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:15.651369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:15.839256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1284/2597] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:15.843084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630f72-b5d5-417d-805e-25f55996ef41 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:15.843872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630f72-b5d5-417d-805e-25f55996ef41 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:15.844625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630f72-b5d5-417d-805e-25f55996ef41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:15.885683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:15.888889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:15.925700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52154e-d6de-4c41-a970-0dcadbd9d604 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:15.927145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-89630f72-b5d5-417d-805e-25f55996ef41 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:15.947929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-89630f72-b5d5-417d-805e-25f55996ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:15.983847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:15.986060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:15.987091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:15.989474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] PATH_INFO: `/v3/projects/9867d8b59ef7448998e0e05ffc5180bf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:15.993347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9867d8b59ef7448998e0e05ffc5180bf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:16.002681 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1314/2598] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:16.004785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89630f72-b5d5-417d-805e-25f55996ef41 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.009730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:16.037349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89630f72-b5d5-417d-805e-25f55996ef41 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:16.069763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:16.072697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.073153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.074873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.075591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:59:16.080847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:16.082524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:16.150523 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1315/2599] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:59:16.151528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.156162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:16.172461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:16.182965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b27602-ef95-483e-a6a2-66ad98b8b140 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:16.188080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9867d8b59ef7448998e0e05ffc5180bf', 'request_id': 'req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:16.207535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9867d8b59ef7448998e0e05ffc5180bf', 'request_id': 'req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:16.224418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9867d8b59ef7448998e0e05ffc5180bf', 'request_id': 'req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:16.267456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] The token cache is being invalidate because project 9867d8b59ef7448998e0e05ffc5180bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:59:16.267456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:16.269347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cb2a14c-e05d-41a7-a8eb-aaab9db7d416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:16.272342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1285/2600] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:59:15 2022] DELETE /identity/v3/projects/9867d8b59ef7448998e0e05ffc5180bf => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:16.282240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a119e-ff70-40e6-8845-66bfdf72cf3f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.282673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a119e-ff70-40e6-8845-66bfdf72cf3f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.283053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a119e-ff70-40e6-8845-66bfdf72cf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.349790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ac1a119e-ff70-40e6-8845-66bfdf72cf3f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:16.353010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:16.353793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.353944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ac1a119e-ff70-40e6-8845-66bfdf72cf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:16.354207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.354539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.355493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:16.358834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:16.390006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1286/2601] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:59:16.392186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15878fcb-6f9a-45bf-a745-defa9ecbb5dd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.392487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15878fcb-6f9a-45bf-a745-defa9ecbb5dd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.392876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15878fcb-6f9a-45bf-a745-defa9ecbb5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.417783 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1316/2602] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:59:16 2022] POST /identity/v3/projects => generated 393 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:16.419742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.424458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:16.433807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-15878fcb-6f9a-45bf-a745-defa9ecbb5dd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:16.435056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-15878fcb-6f9a-45bf-a745-defa9ecbb5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:16.601132 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1287/2603] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:16.603151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15878fcb-6f9a-45bf-a745-defa9ecbb5dd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.631164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15878fcb-6f9a-45bf-a745-defa9ecbb5dd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:16.648032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d0d94-1bac-4ccb-b17f-1a1d0fd58ed7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:16.659028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:16.659833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.660121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.660400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.661100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:59:16.661407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:16.662961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:16.663138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.663254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:16.663640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.664399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.667040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:16.673324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:16.710118 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1288/2604] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 09:59:16.717999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.732736 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1317/2605] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:59:16 2022] POST /identity/v3/users => generated 297 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:16.733865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.758606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842535-32a3-4ee5-95fa-8af96e377bc3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:16.771625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29085187-afbc-43d7-bf32-d6bacbf969ad admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:16.776101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:16.777025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.777300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.778989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.795202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:16.797019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:16.798218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:16.799665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:16.804038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:16.805190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:16.805190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:16.808330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:16.869719 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1318/2606] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:16.871099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.888043 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1289/2607] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:59:16 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:16.894866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:16.922246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfc308-4308-4975-9a99-74981a0875d0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:17.097438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716a96a-7a1f-4a72-8829-e150ed796a28 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:17.110752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:17.111401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.111689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.111992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.112648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:17.121651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:17.184376 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1290/2608] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:59:16 2022] POST /identity/v3/users => generated 291 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:17.188563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:17.189688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:17.191414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.191693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.191965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.198674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:17.205223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:17.210338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3fab4-daa2-4509-bd45-5802978d5cf3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:17.233244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:17.233581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.233866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.234139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.234823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:17.240044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:17.243299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:17.247020 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1319/2609] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:17.265019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:17.272993 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1291/2610] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:17.279598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:17.282660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:17.303382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53391f4-5958-47b5-b35c-18952f737bb9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:17.316525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e5875-ba24-412c-bd0f-aceb1547be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:17.317622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e5875-ba24-412c-bd0f-aceb1547be21 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.318098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e5875-ba24-412c-bd0f-aceb1547be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.318586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e5875-ba24-412c-bd0f-aceb1547be21 admin admin] PATH_INFO: `/v3/projects/326f1e4c552b4263b54ccd2836c4f59d/users/be87e699e5eb458e94e5e9f63ddcac26/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.326037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4bd6a-d44a-4111-9a26-3bc5105b2b4d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:17.339550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:17.340205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.340490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.340864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.341532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:17.343223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:17.352121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e5875-ba24-412c-bd0f-aceb1547be21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=326f1e4c552b4263b54ccd2836c4f59d, user_id=be87e699e5eb458e94e5e9f63ddcac26, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:17.356506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e5875-ba24-412c-bd0f-aceb1547be21 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:17.381642 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1292/2611] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:17.383273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:17.411897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d88d83-87de-4b1a-a8f6-cdaaf0863382 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:17.420089 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1320/2612] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:17 2022] PUT /identity/v3/projects/326f1e4c552b4263b54ccd2836c4f59d/users/be87e699e5eb458e94e5e9f63ddcac26/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:17.432490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393b510-56d8-46da-a601-56e673c01662 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.432797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393b510-56d8-46da-a601-56e673c01662 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.433067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393b510-56d8-46da-a601-56e673c01662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.566740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813aeba1-d09d-49c5-b2f7-48c4c88fdf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:17.568144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aeba1-d09d-49c5-b2f7-48c4c88fdf45 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.568144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aeba1-d09d-49c5-b2f7-48c4c88fdf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.568507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aeba1-d09d-49c5-b2f7-48c4c88fdf45 admin admin] PATH_INFO: `/v3/projects/e9bd76ccd7824e989252951e381a17cd/users/c815e82b67e8436b9bd4a06fe2d03b49/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.620262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813aeba1-d09d-49c5-b2f7-48c4c88fdf45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bd76ccd7824e989252951e381a17cd, user_id=c815e82b67e8436b9bd4a06fe2d03b49, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:17.624841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813aeba1-d09d-49c5-b2f7-48c4c88fdf45 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:17.682671 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1293/2613] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:17 2022] PUT /identity/v3/projects/e9bd76ccd7824e989252951e381a17cd/users/c815e82b67e8436b9bd4a06fe2d03b49/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:17.684527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47059-fa82-4c77-9bcb-29cf0bb83162 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.684848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47059-fa82-4c77-9bcb-29cf0bb83162 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.685125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47059-fa82-4c77-9bcb-29cf0bb83162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.745670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8da47059-fa82-4c77-9bcb-29cf0bb83162 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:17.746936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8da47059-fa82-4c77-9bcb-29cf0bb83162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:17.783840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1393b510-56d8-46da-a601-56e673c01662 None None] MFA Rules not processed for user `be87e699e5eb458e94e5e9f63ddcac26`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:17.785650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1393b510-56d8-46da-a601-56e673c01662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:17.909981 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1294/2614] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:17.936749 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1321/2615] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:17.939227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da47059-fa82-4c77-9bcb-29cf0bb83162 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:17.952248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da47059-fa82-4c77-9bcb-29cf0bb83162 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:17.954927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393b510-56d8-46da-a601-56e673c01662 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:17.958468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1393b510-56d8-46da-a601-56e673c01662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:17.961504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:17.962252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:17.962833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:17.964394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:17.964515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:17.967278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:18.044579 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1295/2616] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:59:17 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:18.048882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:18.110971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393b510-56d8-46da-a601-56e673c01662 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:18.154125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:18.191655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8c787-1b4b-484e-a19c-ee9888d7b338 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:18.203176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766f633-b477-446c-935a-06de91b66e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:18.203784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766f633-b477-446c-935a-06de91b66e94 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.204065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766f633-b477-446c-935a-06de91b66e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.204335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766f633-b477-446c-935a-06de91b66e94 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.209019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766f633-b477-446c-935a-06de91b66e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:18.211391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766f633-b477-446c-935a-06de91b66e94 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:18.300732 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1296/2617] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:59:18 2022] POST /identity/v3/users => generated 301 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:59:18.305516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80115200-28d5-4819-958e-29b3773c9eaf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.305812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80115200-28d5-4819-958e-29b3773c9eaf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.306098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80115200-28d5-4819-958e-29b3773c9eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.334725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde0026-84be-4bb9-b520-41de11f03ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:18.335369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0026-84be-4bb9-b520-41de11f03ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.335666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0026-84be-4bb9-b520-41de11f03ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.338836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0026-84be-4bb9-b520-41de11f03ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.358178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde0026-84be-4bb9-b520-41de11f03ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:18.368878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde0026-84be-4bb9-b520-41de11f03ce8 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:18.397351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-80115200-28d5-4819-958e-29b3773c9eaf None None] MFA Rules not processed for user `57365b62b3294810b8a030d2a26cc451`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:18.400365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-80115200-28d5-4819-958e-29b3773c9eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:18.417948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1322/2618] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:18.421109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.421393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.421666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.507889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] MFA Rules not processed for user `dce01d3212934161a38be853b4cd93ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:18.508487 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1297/2619] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:18 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:18.509669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80115200-28d5-4819-958e-29b3773c9eaf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:18.511146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:18.539150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80115200-28d5-4819-958e-29b3773c9eaf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:18.545093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c43a6-4d8b-4773-b51c-e295e044df7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:18.545737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c43a6-4d8b-4773-b51c-e295e044df7f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.546248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c43a6-4d8b-4773-b51c-e295e044df7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.546427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c43a6-4d8b-4773-b51c-e295e044df7f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.547134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c43a6-4d8b-4773-b51c-e295e044df7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:18.548772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c43a6-4d8b-4773-b51c-e295e044df7f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:18.556140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:18.575916 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1298/2620] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:18.577577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff172-6f98-4ca3-b087-d894ef73cf9d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.577870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff172-6f98-4ca3-b087-d894ef73cf9d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.578147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff172-6f98-4ca3-b087-d894ef73cf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.625834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b1bff172-6f98-4ca3-b087-d894ef73cf9d None None] MFA Rules not processed for user `57365b62b3294810b8a030d2a26cc451`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:18.627254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b1bff172-6f98-4ca3-b087-d894ef73cf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:18.651052 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1299/2621] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:18 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:18.652274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bff172-6f98-4ca3-b087-d894ef73cf9d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:18.665580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bff172-6f98-4ca3-b087-d894ef73cf9d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:18.671767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:18.672698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.673149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.673574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] PATH_INFO: `/v3/projects/9f7c04508ed14a2f9b8dc8aed2745592/users/5126adbc2c534fb2bf0800ba526f3716/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.679260 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1323/2622] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:18.679260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:18.682902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:18.698736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f7c04508ed14a2f9b8dc8aed2745592, user_id=5126adbc2c534fb2bf0800ba526f3716, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:18.702690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:18.729827 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1300/2623] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:18 2022] PUT /identity/v3/projects/9f7c04508ed14a2f9b8dc8aed2745592/users/5126adbc2c534fb2bf0800ba526f3716/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:18.732554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:18.735595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:18.834324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890b9a0-f265-4009-9f6e-2eb8dd34c67b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:18.877554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b0397-3ad3-423e-8dfb-4293590b1f5c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:18.886766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:18.887413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:18.887697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:18.887970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:18.888642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:18.909778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:18.976475 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1301/2624] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:59:18 2022] POST /identity/v3/projects => generated 371 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:18.980332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:19.013386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:19.019171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.019375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.025906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.026192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.120307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:19.125186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.127641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.146517 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1324/2625] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:18 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:19.147731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:19.163078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff9cdd-8dca-414d-8181-ece159ea7ec4 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:19.171224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae9e4-9166-443c-ba55-c3ec972640bb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:19.175872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.178365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.180546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.180546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.186978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.188288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.189169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.189449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.189720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.190452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.192413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.193427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.244197 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1325/2626] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:59:19 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:19.256486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:19.267017 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1302/2627] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:19.268289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:19.272491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7aa31-050b-4da2-859d-7775a1c89447 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:19.301907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.302554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.302828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.303091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.305834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.310474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.319732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08969415-065c-4a8c-8f72-720a8c646adc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:19.332870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2979ea6-9eb6-4349-9955-c154e4311331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.336443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2979ea6-9eb6-4349-9955-c154e4311331 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.338640 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1326/2628] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:19.343412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2979ea6-9eb6-4349-9955-c154e4311331 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.344212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2979ea6-9eb6-4349-9955-c154e4311331 admin admin] PATH_INFO: `/v3/projects/9f7c04508ed14a2f9b8dc8aed2745592/users/5126adbc2c534fb2bf0800ba526f3716/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.344600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:19.349090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2979ea6-9eb6-4349-9955-c154e4311331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f7c04508ed14a2f9b8dc8aed2745592, user_id=5126adbc2c534fb2bf0800ba526f3716, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.357538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2979ea6-9eb6-4349-9955-c154e4311331 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.375024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe199e4-5fa9-4c72-9cab-b51aefd69663 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:19.406948 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1303/2629] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:19 2022] PUT /identity/v3/projects/9f7c04508ed14a2f9b8dc8aed2745592/users/5126adbc2c534fb2bf0800ba526f3716/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:19.411270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaac927-fa3c-4d4f-8c21-dd0cb1624ade None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.411615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaac927-fa3c-4d4f-8c21-dd0cb1624ade None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.411834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaac927-fa3c-4d4f-8c21-dd0cb1624ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.419408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.424150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.424592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.425010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] PATH_INFO: `/v3/projects/3edc67dee2b1455db651fa37a021e07e/users/0f8ac55ed39d4ef7bc080ec339c0aca9/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.483073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edc67dee2b1455db651fa37a021e07e, user_id=0f8ac55ed39d4ef7bc080ec339c0aca9, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.486413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.515712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9eaac927-fa3c-4d4f-8c21-dd0cb1624ade None None] MFA Rules not processed for user `5126adbc2c534fb2bf0800ba526f3716`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:19.530067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9eaac927-fa3c-4d4f-8c21-dd0cb1624ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:19.534172 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1327/2630] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:19 2022] PUT /identity/v3/projects/3edc67dee2b1455db651fa37a021e07e/users/0f8ac55ed39d4ef7bc080ec339c0aca9/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:19.537333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:19.730450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce58d8-b3c9-454e-afd8-9df7eb8a852f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:19.735988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.736913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.737461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.737907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.739060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.741219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.762682 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1328/2631] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:59:19.768690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:19.787974 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1304/2632] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:19 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:19.814460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea768038-97bb-4754-a564-9af1e90150eb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:19.839300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:19.839982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:19.840255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:19.840523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] PATH_INFO: `/v3/projects/3edc67dee2b1455db651fa37a021e07e/users/0f8ac55ed39d4ef7bc080ec339c0aca9/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:19.846571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edc67dee2b1455db651fa37a021e07e, user_id=0f8ac55ed39d4ef7bc080ec339c0aca9, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:19.849251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:19.882364 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1329/2633] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:19 2022] PUT /identity/v3/projects/3edc67dee2b1455db651fa37a021e07e/users/0f8ac55ed39d4ef7bc080ec339c0aca9/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:20.489962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaac927-fa3c-4d4f-8c21-dd0cb1624ade None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:20.662844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaac927-fa3c-4d4f-8c21-dd0cb1624ade None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:20.702740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:20.849868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:20.851623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:20.852112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:20.852706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:21.080302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:21.085762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:21.129144 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1305/2634] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:20 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:21.516148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:21.518225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:21.574932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacecce-e5b7-494c-8b3d-57352f59948d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:21.608635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5723210-51d3-4e24-a5ee-b1dad1005041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:21.609425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5723210-51d3-4e24-a5ee-b1dad1005041 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:21.609798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5723210-51d3-4e24-a5ee-b1dad1005041 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:21.610162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5723210-51d3-4e24-a5ee-b1dad1005041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:21.809546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5723210-51d3-4e24-a5ee-b1dad1005041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:21.812549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5723210-51d3-4e24-a5ee-b1dad1005041 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:21.830812 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1330/2635] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:21.869907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:22.051221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec381d4-551b-4b24-825f-0dc4516b3e25 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:22.071696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:22.072541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:22.072908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:22.074489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] PATH_INFO: `/v3/users/be87e699e5eb458e94e5e9f63ddcac26` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:22.077773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be87e699e5eb458e94e5e9f63ddcac26)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:22.089553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:22.279288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be87e699e5eb458e94e5e9f63ddcac26'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:22.284304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be87e699e5eb458e94e5e9f63ddcac26'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:22.299262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:22.306180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be87e699e5eb458e94e5e9f63ddcac26'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:22.327304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be87e699e5eb458e94e5e9f63ddcac26'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:22.366760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be87e699e5eb458e94e5e9f63ddcac26'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:22.394994 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1306/2636] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:59:21 2022] DELETE /identity/v3/users/be87e699e5eb458e94e5e9f63ddcac26 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:22.395798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:22.396167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:22.396432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:22.459868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:22.466574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:22.511373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:22.575087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1331/2637] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:22.777525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:22.779289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:23.001916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511c30b-a587-42b1-936a-da383f2d1a33 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:23.006644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:23.007593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:23.007971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:23.008343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:23.030641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:23.033373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:23.049212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1307/2638] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:25.871226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:25.873279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:25.904377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114b65d-4ec6-4b63-a71e-bf8d5f941e44 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:25.918426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:25.920002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:25.920476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:25.921968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] PATH_INFO: `/v3/projects/326f1e4c552b4263b54ccd2836c4f59d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:25.926237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=326f1e4c552b4263b54ccd2836c4f59d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:25.929975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:26.065154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326f1e4c552b4263b54ccd2836c4f59d', 'request_id': 'req-628205fd-597a-42c7-b028-cbd0ccce5284'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.081611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326f1e4c552b4263b54ccd2836c4f59d', 'request_id': 'req-628205fd-597a-42c7-b028-cbd0ccce5284'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.104239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326f1e4c552b4263b54ccd2836c4f59d', 'request_id': 'req-628205fd-597a-42c7-b028-cbd0ccce5284'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.130686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] The token cache is being invalidate because project 326f1e4c552b4263b54ccd2836c4f59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:59:26.131473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.133861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-628205fd-597a-42c7-b028-cbd0ccce5284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.138827 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1332/2639] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:59:25 2022] DELETE /identity/v3/projects/326f1e4c552b4263b54ccd2836c4f59d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:26.142761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:26.156230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e0646-735f-424b-be03-6188b550a7d3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:26.340476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:26.351841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:26.354292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:26.357038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] PATH_INFO: `/v3/users/5126adbc2c534fb2bf0800ba526f3716` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:26.365049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5126adbc2c534fb2bf0800ba526f3716)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:26.392716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:26.520168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5126adbc2c534fb2bf0800ba526f3716'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.521936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5126adbc2c534fb2bf0800ba526f3716'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.546897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5126adbc2c534fb2bf0800ba526f3716'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.567188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5126adbc2c534fb2bf0800ba526f3716'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.587428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:26.595084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-961524f7-2e88-465c-9447-c2b23c5a6059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5126adbc2c534fb2bf0800ba526f3716'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:26.625303 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1308/2640] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:59:26 2022] DELETE /identity/v3/users/5126adbc2c534fb2bf0800ba526f3716 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:26.899024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:26.899907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:26.899907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:26.996890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] MFA Rules not processed for user `f495627b28c44a979882d447f73d97fd`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:26.998055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:27.246968 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1333/2641] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:59:27.262320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:27.262937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:27.263030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:27.312119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] MFA Rules not processed for user `29d2f7b0e8274c769521d13930be9825`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:27.319798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:27.445531 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1309/2642] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:27.447759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:27.450109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:27.488495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:27.547657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5b9aa-a1cc-4989-b284-43ff048bf5be None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:27.579074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:27.580449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:27.581499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:27.582119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:27.586193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:27.622852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:27.625445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:27.651992 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1334/2643] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:28.419459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:28.422059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:28.590673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734addd-2b48-45b1-ba45-69f08b69d37c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:28.597547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500a8fd-4ccd-42d8-b5c5-c58526314a4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:28.598527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500a8fd-4ccd-42d8-b5c5-c58526314a4c service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:28.599334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500a8fd-4ccd-42d8-b5c5-c58526314a4c service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:28.599977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500a8fd-4ccd-42d8-b5c5-c58526314a4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:28.603818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4500a8fd-4ccd-42d8-b5c5-c58526314a4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:28.622186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500a8fd-4ccd-42d8-b5c5-c58526314a4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:28.628705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500a8fd-4ccd-42d8-b5c5-c58526314a4c service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:28.655985 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1310/2644] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:28.835079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:28.837482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:29.022212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885442c-b0e0-4c8b-8680-ecbaf0a77fa1 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:29.034911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:29.035605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:29.035882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:29.036156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] PATH_INFO: `/v3/projects/9f7c04508ed14a2f9b8dc8aed2745592` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:29.037409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f7c04508ed14a2f9b8dc8aed2745592)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:29.039587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:29.318735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f7c04508ed14a2f9b8dc8aed2745592', 'request_id': 'req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:29.349080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f7c04508ed14a2f9b8dc8aed2745592', 'request_id': 'req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:29.360122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f7c04508ed14a2f9b8dc8aed2745592', 'request_id': 'req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:29.390691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] The token cache is being invalidate because project 9f7c04508ed14a2f9b8dc8aed2745592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 09:59:29.391186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:29.394610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8788b4b1-c0bd-460c-a5ba-670c60fd9e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 09:59:29.401966 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1335/2645] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 09:59:28 2022] DELETE /identity/v3/projects/9f7c04508ed14a2f9b8dc8aed2745592 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:29.409140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c737af-b839-4880-b12e-db617d85dd0f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:29.409448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c737af-b839-4880-b12e-db617d85dd0f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:29.409705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c737af-b839-4880-b12e-db617d85dd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:29.455844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e8c737af-b839-4880-b12e-db617d85dd0f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:29.457336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e8c737af-b839-4880-b12e-db617d85dd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:29.648323 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1311/2646] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:29.659069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13986c38-4b38-4fd2-ba95-133bcfeb5cab None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:29.659334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13986c38-4b38-4fd2-ba95-133bcfeb5cab None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:29.659440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13986c38-4b38-4fd2-ba95-133bcfeb5cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:29.691160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-13986c38-4b38-4fd2-ba95-133bcfeb5cab None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:29.692393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-13986c38-4b38-4fd2-ba95-133bcfeb5cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:29.704652 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1336/2647] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:29.709419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c737af-b839-4880-b12e-db617d85dd0f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:29.726877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c737af-b839-4880-b12e-db617d85dd0f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:29.761597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:29.762623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:29.763082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:29.765136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:29.767274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 09:59:29.768489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:29.771706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:29.806229 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1312/2648] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 09:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 09:59:29.810804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13986c38-4b38-4fd2-ba95-133bcfeb5cab None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:29.825112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13986c38-4b38-4fd2-ba95-133bcfeb5cab None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:29.854395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:29.855334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:29.860395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:29.866013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:29.869233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:29.871945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:29.925143 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1337/2649] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:59:29 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:29.930876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:29.947826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4ced8-97ad-4a37-a556-f56943b53c62 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:29.968075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:30.341627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:30.342972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:30.344016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:30.344505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:30.360379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:30.369550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:30.413301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:30.417221 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1313/2650] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:59:29 2022] POST /identity/v3/users => generated 292 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:30.450485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346f075-2210-4f1d-9595-535f6a902044 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:30.477858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:30.478836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:30.479489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:30.479958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:30.487081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:30.499651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:30.563321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:30.570016 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1338/2651] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:30.583230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acf25e-ac22-459e-ad12-67c048adf9c4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:30.600111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74176a08-4f1b-4b00-8f05-eccc328960a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:30.601090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74176a08-4f1b-4b00-8f05-eccc328960a9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:30.601618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74176a08-4f1b-4b00-8f05-eccc328960a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:30.602111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74176a08-4f1b-4b00-8f05-eccc328960a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:30.603222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74176a08-4f1b-4b00-8f05-eccc328960a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:30.606151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74176a08-4f1b-4b00-8f05-eccc328960a9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:30.625352 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1314/2652] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:59:30.628619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:30.665952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a110322-4836-4bb7-a880-50aee3168516 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:30.699297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:30.700461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:30.701047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:30.702202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] PATH_INFO: `/v3/projects/5de10e92e2024a24880c7205697e5732/users/0d563feacff84b4c9927ec759fb365be/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:30.711319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:30.772740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de10e92e2024a24880c7205697e5732, user_id=0d563feacff84b4c9927ec759fb365be, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:30.780596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:30.813858 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1339/2653] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:30 2022] PUT /identity/v3/projects/5de10e92e2024a24880c7205697e5732/users/0d563feacff84b4c9927ec759fb365be/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:30.818592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b82d2c-8ca6-4ce7-be75-d2954c51bf67 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:30.818855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b82d2c-8ca6-4ce7-be75-d2954c51bf67 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:30.819135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b82d2c-8ca6-4ce7-be75-d2954c51bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:30.855364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-00b82d2c-8ca6-4ce7-be75-d2954c51bf67 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:30.856635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-00b82d2c-8ca6-4ce7-be75-d2954c51bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:31.160182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1315/2654] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:31.225078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:31.228634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:31.610466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc2350-8e07-4534-9af4-1ce178aad09f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:31.634834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a1251-6570-4ece-86ff-a51928a18b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:31.635484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1251-6570-4ece-86ff-a51928a18b66 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:31.635963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1251-6570-4ece-86ff-a51928a18b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:31.636086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1251-6570-4ece-86ff-a51928a18b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:31.664174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a1251-6570-4ece-86ff-a51928a18b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:31.666362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a1251-6570-4ece-86ff-a51928a18b66 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:31.709249 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1340/2655] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:38.558404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:38.559701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:38.560580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:38.667465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] MFA Rules not processed for user `c815e82b67e8436b9bd4a06fe2d03b49`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:38.669483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:38.831454 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1316/2656] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:38.921398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126967-614d-441a-85c9-55a83b7fe7b6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:38.921877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126967-614d-441a-85c9-55a83b7fe7b6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:38.921999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126967-614d-441a-85c9-55a83b7fe7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:38.972681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7c126967-614d-441a-85c9-55a83b7fe7b6 None None] MFA Rules not processed for user `c815e82b67e8436b9bd4a06fe2d03b49`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:38.976590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c126967-614d-441a-85c9-55a83b7fe7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:38.999823 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1341/2657] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:39.046971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:39.048725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:39.090027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:39.172864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ea63-0b05-40cf-b531-22e2698bfd9d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:39.182157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e034e1e-d5c2-4c22-9b7a-82c86cf6a0cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:39.185315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034e1e-d5c2-4c22-9b7a-82c86cf6a0cf service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:39.185654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034e1e-d5c2-4c22-9b7a-82c86cf6a0cf service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:39.186413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034e1e-d5c2-4c22-9b7a-82c86cf6a0cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:39.208014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e034e1e-d5c2-4c22-9b7a-82c86cf6a0cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:39.212434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e034e1e-d5c2-4c22-9b7a-82c86cf6a0cf service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:39.249402 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1317/2658] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:48.888942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:48.889802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:48.890223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:48.965930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] MFA Rules not processed for user `0f8ac55ed39d4ef7bc080ec339c0aca9`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:48.965930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:49.170916 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1342/2659] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:49.183737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62935f-cbfb-448f-badb-031d2fe450b1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:49.184775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62935f-cbfb-448f-badb-031d2fe450b1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:49.185275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62935f-cbfb-448f-badb-031d2fe450b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:49.219139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cd62935f-cbfb-448f-badb-031d2fe450b1 None None] MFA Rules not processed for user `dce01d3212934161a38be853b4cd93ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:49.221236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cd62935f-cbfb-448f-badb-031d2fe450b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:49.317306 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1318/2660] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:49.342539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:49.344687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:49.391641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07000072-b5fb-4aef-802c-cb7d23557f0e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:49.407515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:49.409112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:49.409112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:49.409341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:49.411478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:49.439637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:49.443809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:49.471824 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1343/2661] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 09:59:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 09:59:52.769419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cfff8-aefc-433f-8728-9a73c1484324 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:52.769993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cfff8-aefc-433f-8728-9a73c1484324 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:52.769993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cfff8-aefc-433f-8728-9a73c1484324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:52.865629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c02cfff8-aefc-433f-8728-9a73c1484324 None None] MFA Rules not processed for user `0d563feacff84b4c9927ec759fb365be`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:52.870461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c02cfff8-aefc-433f-8728-9a73c1484324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:53.060157 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1319/2662] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 09:59:53.063888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:53.087630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab785e-7774-4fd7-b4bf-5e2a25203d7a service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:53.093814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6bbef-847c-461c-a696-47f9f2b6fcad None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:53.094109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6bbef-847c-461c-a696-47f9f2b6fcad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:53.095604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6bbef-847c-461c-a696-47f9f2b6fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:53.103043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:53.103817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:53.104080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:53.104554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:53.105120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:53.117279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:53.162556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7cc6bbef-847c-461c-a696-47f9f2b6fcad None None] MFA Rules not processed for user `9544cdb0392f416197a586d99e8a3210`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 09:59:53.166629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7cc6bbef-847c-461c-a696-47f9f2b6fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 09:59:53.185669 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1344/2663] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 09:59:53 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 09:59:53.186883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:53.277922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:53.303986 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1320/2664] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 09:59:53.309642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:53.363582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1419d-82bf-4cfe-9422-2982f79e065f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:53.375413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:53.376139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:53.376529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:53.376924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:53.378029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:53.380792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:53.446394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6bbef-847c-461c-a696-47f9f2b6fcad None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:53.447225 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1345/2665] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 09:59:53 2022] POST /identity/v3/users => generated 298 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 09:59:53.474251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6bbef-847c-461c-a696-47f9f2b6fcad None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:53.493402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:53.494048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:53.494311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:53.494600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:53.495254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:53.496882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:53.517598 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1321/2666] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 09:59:53.521688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:53.554270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f11458-5f83-44bb-a7e9-7837a8f68ac9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:53.568708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:53.573413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:53.574132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:53.574563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] PATH_INFO: `/v3/projects/6478a4ad3f4e4c7493f011513b7bb36d/users/7d6172f063be4dc48a01a40dd71b67e1/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:53.613744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6478a4ad3f4e4c7493f011513b7bb36d, user_id=7d6172f063be4dc48a01a40dd71b67e1, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:53.622166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:53.659725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:53.661227 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1346/2667] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:53 2022] PUT /identity/v3/projects/6478a4ad3f4e4c7493f011513b7bb36d/users/7d6172f063be4dc48a01a40dd71b67e1/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 09:59:53.676638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51df7a-bda1-4aef-833a-a548bae6f549 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:53.809608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:53.811071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:53.812970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:53.813495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:53.814302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:53.819680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:53.846621 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1322/2668] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 09:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 09:59:53.851127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 09:59:53.875412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc901859-ca27-489a-a888-e4f3b9f16e60 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 09:59:53.880039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2194578-18f0-4e71-b211-ae7ff280a678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 09:59:53.880811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2194578-18f0-4e71-b211-ae7ff280a678 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 09:59:53.881092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2194578-18f0-4e71-b211-ae7ff280a678 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 09:59:53.881504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2194578-18f0-4e71-b211-ae7ff280a678 admin admin] PATH_INFO: `/v3/projects/6478a4ad3f4e4c7493f011513b7bb36d/users/7d6172f063be4dc48a01a40dd71b67e1/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 09:59:53.893639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2194578-18f0-4e71-b211-ae7ff280a678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6478a4ad3f4e4c7493f011513b7bb36d, user_id=7d6172f063be4dc48a01a40dd71b67e1, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 09:59:53.897090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2194578-18f0-4e71-b211-ae7ff280a678 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 09:59:53.940413 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1347/2669] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 09:59:53 2022] PUT /identity/v3/projects/6478a4ad3f4e4c7493f011513b7bb36d/users/7d6172f063be4dc48a01a40dd71b67e1/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:03.829905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=40 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:03.926147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=31,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:05.721760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:05.830831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2e32-721d-4596-b17e-8cf7fe7e2718 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:05.910668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:05.911371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:05.911641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:05.911911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:05.914268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:05.985967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:05.987642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:05.998456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1323/2670] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:05 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:08.939989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7a496-43ae-4665-82f1-1e1c9e7948b2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:08.940961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7a496-43ae-4665-82f1-1e1c9e7948b2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:08.941391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7a496-43ae-4665-82f1-1e1c9e7948b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:09.003775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-84c7a496-43ae-4665-82f1-1e1c9e7948b2 None None] MFA Rules not processed for user `7d6172f063be4dc48a01a40dd71b67e1`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:09.005565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84c7a496-43ae-4665-82f1-1e1c9e7948b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:09.178473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:09.181356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:09.182026 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1348/2671] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:09.240763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:09.298897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa90ab-cc04-4db3-a20f-fbcf48cad866 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:09.305269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:09.306026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:09.306521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:09.306891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:09.320521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:09.323014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:09.377401 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1324/2672] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:00:09.559695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eea8c-2528-446d-a9be-dc35f26ba8cf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:09.561172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eea8c-2528-446d-a9be-dc35f26ba8cf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:09.562708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eea8c-2528-446d-a9be-dc35f26ba8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:09.624334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a67eea8c-2528-446d-a9be-dc35f26ba8cf None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:09.632952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a67eea8c-2528-446d-a9be-dc35f26ba8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:09.658076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1349/2673] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:09.747530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:09.749391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:09.940071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15030a66-0143-4dc9-b954-c2856b7a6672 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:09.956835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3951946-786f-4a41-b9a8-314e3df17a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:09.959237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951946-786f-4a41-b9a8-314e3df17a38 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:09.959383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951946-786f-4a41-b9a8-314e3df17a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:09.959591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3951946-786f-4a41-b9a8-314e3df17a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:10.001989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3951946-786f-4a41-b9a8-314e3df17a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:10.005031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3951946-786f-4a41-b9a8-314e3df17a38 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:10.035442 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1325/2674] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:18.414217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eea8c-2528-446d-a9be-dc35f26ba8cf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:18.439750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eea8c-2528-446d-a9be-dc35f26ba8cf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:18.452682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:18.455079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:18.456093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:18.456700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:18.637591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:18.639962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:18.667642 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1350/2675] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:20.023216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=38 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:23.540429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31174087-11a7-49a9-8c1f-cf0ae5a9a1b8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:23.541480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31174087-11a7-49a9-8c1f-cf0ae5a9a1b8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:23.541904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31174087-11a7-49a9-8c1f-cf0ae5a9a1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:23.567511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-31174087-11a7-49a9-8c1f-cf0ae5a9a1b8 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:23.569285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-31174087-11a7-49a9-8c1f-cf0ae5a9a1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:23.657528 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1326/2676] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:23.717260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:23.719011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:23.825339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89d5e0-aa50-433c-a717-6b74cffae300 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:23.835702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:23.836376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:23.836867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:23.837145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:23.849273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:23.851697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:23.889304 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1351/2677] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:00:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:24.369299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:24.370044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:24.370471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:24.418229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:24.427489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:24.439536 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1327/2678] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:00:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:24.510926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:24.518187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:24.531643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421d81-e1f6-4391-a3e4-f6b9e0b85357 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:24.536022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02954386-02f1-42cf-acd2-e2a1e0aad2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:24.537078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02954386-02f1-42cf-acd2-e2a1e0aad2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:24.537569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02954386-02f1-42cf-acd2-e2a1e0aad2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:24.538036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02954386-02f1-42cf-acd2-e2a1e0aad2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:24.569390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02954386-02f1-42cf-acd2-e2a1e0aad2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:24.571250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02954386-02f1-42cf-acd2-e2a1e0aad2c8 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:24.596309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1352/2679] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:29.257668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:29.259224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:29.517460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bbb93-804b-4e15-846d-fdc73c760d57 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:29.528068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:29.529196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:29.529665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:29.530093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] PATH_INFO: `/v3/users/00423eb1e4c24763ae0a546225ac1dc7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:29.531618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00423eb1e4c24763ae0a546225ac1dc7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:29.536530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:29.647065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00423eb1e4c24763ae0a546225ac1dc7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:29.647847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00423eb1e4c24763ae0a546225ac1dc7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:29.665720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00423eb1e4c24763ae0a546225ac1dc7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:29.678692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00423eb1e4c24763ae0a546225ac1dc7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:29.709549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00423eb1e4c24763ae0a546225ac1dc7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:29.726216 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1328/2680] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:00:29 2022] DELETE /identity/v3/users/00423eb1e4c24763ae0a546225ac1dc7 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:29.731249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:29.732250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:29.732754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:29.799899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:29.801270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:29.983982 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1353/2681] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:29.995805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:29.997854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:30.031782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:30.110730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570a5e7-d3bf-441e-9450-2b47b0a2ec9f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:30.121434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:30.124932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:30.125462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:30.126184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:30.144840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:30.146944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:30.170310 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1329/2682] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:31.099742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:31.101758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:31.129395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd3c33-0082-489b-83b3-519f0b9c6eb2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:31.151515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:31.152329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:31.152612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:31.152910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] PATH_INFO: `/v3/users/c815e82b67e8436b9bd4a06fe2d03b49` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:31.157211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c815e82b67e8436b9bd4a06fe2d03b49)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:31.159813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:31.218016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:31.219567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:31.248385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517140f-2716-4e6e-ad16-3cf8f3ba96b5 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:31.261363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:31.264419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:31.265333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:31.266634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] PATH_INFO: `/v3/projects/c1ac7b64c68e4896b2c631baacdbe2be` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:31.272792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ac7b64c68e4896b2c631baacdbe2be)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:31.278529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:31.364910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c815e82b67e8436b9bd4a06fe2d03b49'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.365542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c815e82b67e8436b9bd4a06fe2d03b49'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.386085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c815e82b67e8436b9bd4a06fe2d03b49'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.408735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c815e82b67e8436b9bd4a06fe2d03b49'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.409493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c815e82b67e8436b9bd4a06fe2d03b49'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.447501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ac7b64c68e4896b2c631baacdbe2be', 'request_id': 'req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.465326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1354/2683] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:00:31 2022] DELETE /identity/v3/users/c815e82b67e8436b9bd4a06fe2d03b49 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:31.465570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ac7b64c68e4896b2c631baacdbe2be', 'request_id': 'req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.481373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ac7b64c68e4896b2c631baacdbe2be', 'request_id': 'req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.505617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] The token cache is being invalidate because project c1ac7b64c68e4896b2c631baacdbe2be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:00:31.506006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.510759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ae1cd5c-4cc4-4cde-97ae-e65627611cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.515564 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1330/2684] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:00:31 2022] DELETE /identity/v3/projects/c1ac7b64c68e4896b2c631baacdbe2be => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:31.517196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:31.519403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:31.713215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ff907-77e8-47e2-9219-340c44fa902d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:31.739804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:31.741069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:31.741558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:31.742258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] PATH_INFO: `/v3/users/82777c78616b4bd6b205a12c2416c420` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:31.755569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82777c78616b4bd6b205a12c2416c420)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:31.760157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:31.767805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:31.772526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0486-1b2b-435c-9a31-758397306015 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:31.773140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0486-1b2b-435c-9a31-758397306015 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:31.773662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0486-1b2b-435c-9a31-758397306015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:31.830039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e25d0486-1b2b-435c-9a31-758397306015 None None] MFA Rules not processed for user `29d2f7b0e8274c769521d13930be9825`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:31.833102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e25d0486-1b2b-435c-9a31-758397306015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:31.885192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82777c78616b4bd6b205a12c2416c420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.886730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82777c78616b4bd6b205a12c2416c420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.900384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82777c78616b4bd6b205a12c2416c420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.935821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82777c78616b4bd6b205a12c2416c420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:31.937675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82777c78616b4bd6b205a12c2416c420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:32.046273 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1355/2685] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:00:31 2022] DELETE /identity/v3/users/82777c78616b4bd6b205a12c2416c420 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:32.047841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:32.057790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:32.110184 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1331/2686] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:32.261632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c090e0-02b5-4981-9526-c833af68ae67 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:32.281655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:32.282611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:32.283041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:32.283499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] PATH_INFO: `/v3/projects/e9bd76ccd7824e989252951e381a17cd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:32.285192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bd76ccd7824e989252951e381a17cd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:32.287656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:32.406166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bd76ccd7824e989252951e381a17cd', 'request_id': 'req-6a0a86cd-0087-4c04-879d-045b1b37d3a4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:32.422766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bd76ccd7824e989252951e381a17cd', 'request_id': 'req-6a0a86cd-0087-4c04-879d-045b1b37d3a4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:32.447586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bd76ccd7824e989252951e381a17cd', 'request_id': 'req-6a0a86cd-0087-4c04-879d-045b1b37d3a4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:32.494405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] The token cache is being invalidate because project e9bd76ccd7824e989252951e381a17cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:00:32.494797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:32.496848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:32.502057 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1356/2687] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:00:32 2022] DELETE /identity/v3/projects/e9bd76ccd7824e989252951e381a17cd => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:32.507369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdee6a-d2c8-4ffe-96b5-6ed92d62e927 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:32.508154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdee6a-d2c8-4ffe-96b5-6ed92d62e927 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:32.508154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdee6a-d2c8-4ffe-96b5-6ed92d62e927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:32.551838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-30bdee6a-d2c8-4ffe-96b5-6ed92d62e927 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:32.553751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30bdee6a-d2c8-4ffe-96b5-6ed92d62e927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:32.637247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:32.639057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:32.685586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:32.724981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:32.765672 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1332/2688] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:32.774832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48966281-605a-43cf-89e7-46110bf4f121 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:32.775429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48966281-605a-43cf-89e7-46110bf4f121 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:32.775869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48966281-605a-43cf-89e7-46110bf4f121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:32.835851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-48966281-605a-43cf-89e7-46110bf4f121 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:32.840862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-48966281-605a-43cf-89e7-46110bf4f121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:32.862508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a86cd-0087-4c04-879d-045b1b37d3a4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:32.884910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:32.885840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:32.886264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:32.886522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] PATH_INFO: `/v3/projects/752a253dce3f4a0db3010c597e5533d7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:32.887350 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1333/2689] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:32.888708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48966281-605a-43cf-89e7-46110bf4f121 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:32.891477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=752a253dce3f4a0db3010c597e5533d7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:32.898017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:32.912592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48966281-605a-43cf-89e7-46110bf4f121 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:32.931272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:32.932233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:32.932233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:32.932447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:32.933032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:00:32.933333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:32.935331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:32.969719 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1334/2690] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:00:32.974316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:32.993720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9a5c1-c0cc-408e-9599-8af5fce60b23 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.043758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752a253dce3f4a0db3010c597e5533d7', 'request_id': 'req-42129c5c-beed-4a59-a2ab-51c07d2701a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:33.059393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752a253dce3f4a0db3010c597e5533d7', 'request_id': 'req-42129c5c-beed-4a59-a2ab-51c07d2701a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:33.072902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752a253dce3f4a0db3010c597e5533d7', 'request_id': 'req-42129c5c-beed-4a59-a2ab-51c07d2701a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:33.106131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] The token cache is being invalidate because project 752a253dce3f4a0db3010c597e5533d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:00:33.107554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:33.111104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42129c5c-beed-4a59-a2ab-51c07d2701a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:33.113729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.114397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.114671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.114932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.115579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.116256 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1357/2691] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:00:32 2022] DELETE /identity/v3/projects/752a253dce3f4a0db3010c597e5533d7 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:33.118665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.121867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43126c00-88d4-418b-8b1a-5af40da23273 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.122151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43126c00-88d4-418b-8b1a-5af40da23273 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.122411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43126c00-88d4-418b-8b1a-5af40da23273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.175367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-43126c00-88d4-418b-8b1a-5af40da23273 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:33.177434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-43126c00-88d4-418b-8b1a-5af40da23273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:33.214764 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1358/2692] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:00:33.225516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b161-608a-4480-a047-cf0642e1fb09 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.225795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b161-608a-4480-a047-cf0642e1fb09 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.226393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b161-608a-4480-a047-cf0642e1fb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.230038 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1335/2693] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:00:32 2022] POST /identity/v3/projects => generated 399 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:33.238468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.243653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:33.301798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c7e7b161-608a-4480-a047-cf0642e1fb09 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:33.303054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7e7b161-608a-4480-a047-cf0642e1fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:33.420563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4bdd3-43d0-49e0-afd1-53271219aef3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.429061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.432051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.432331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.434673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.435489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.440991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.451653 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1359/2694] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:00:33.460483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b161-608a-4480-a047-cf0642e1fb09 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.494369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b161-608a-4480-a047-cf0642e1fb09 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.502569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.503210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.504985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.504985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.504985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:00:33.504985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.508907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.511672 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1336/2695] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:00:33 2022] POST /identity/v3/users => generated 300 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:33.516872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.540573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf2522-13c5-4514-a6dd-d6cf23cd10af admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.545435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1360/2696] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:00:33.550648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.565457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.566600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.566954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.567235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.569021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.576924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd787680-657a-471f-a59f-6d744a38fb9f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.577210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.598678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.600110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.602469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.603206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.605146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.619255 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1337/2697] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:00:33.620535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.620854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.646884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ed386-2fd1-4ea5-8274-1882ee615d3f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.659917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.661223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.661779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.662323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.664160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.670718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:33.673223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.687533 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1361/2698] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:00:33 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:33.689513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.703784 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1338/2699] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:00:33.705242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.721827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8c37-9ad9-4933-8a7f-3e5d26e528e9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.753550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:33.787915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3543f06-82e0-4380-bcbd-65ee74fb3e0f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.798087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.799399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.803701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.804117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.805459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.810028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.875580 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1362/2700] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:00:33 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:33.877660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.889220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357be671-2e58-458f-a9f2-821528bdf138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.891339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357be671-2e58-458f-a9f2-821528bdf138 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.891641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357be671-2e58-458f-a9f2-821528bdf138 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.891920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357be671-2e58-458f-a9f2-821528bdf138 admin admin] PATH_INFO: `/v3/projects/e2481f3613e4485a8424a23cc716a060/users/9e9418ffc1c249fd8d854fb4dc7be36f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.903185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993eb3a-02ea-40dd-aabe-0358efb8cf28 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:33.918857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:33.919561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.919874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.920152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:33.923155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.932036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.932736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357be671-2e58-458f-a9f2-821528bdf138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2481f3613e4485a8424a23cc716a060, user_id=9e9418ffc1c249fd8d854fb4dc7be36f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:33.937180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357be671-2e58-458f-a9f2-821528bdf138 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:33.956804 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1363/2701] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:00:33.962239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:33.977759 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1339/2702] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:00:33 2022] PUT /identity/v3/projects/e2481f3613e4485a8424a23cc716a060/users/9e9418ffc1c249fd8d854fb4dc7be36f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:33.980440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:33.981289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:33.981743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:34.004066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9bdc-f10f-4368-9294-68f156e1232b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:34.051447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] MFA Rules not processed for user `9e9418ffc1c249fd8d854fb4dc7be36f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:34.052925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:34.140373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:34.141117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:34.141468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:34.141995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:34.142795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:34.150767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:34.185837 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1364/2703] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:00:34.204389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:34.219069 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1340/2704] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:34.238650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd79aed-abcb-4adf-8e8c-9b5e58e8df96 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:34.239333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:34.242902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:34.251548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e4383-4d77-43f2-a51e-2965b0255afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:34.255233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e4383-4d77-43f2-a51e-2965b0255afc admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:34.258490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e4383-4d77-43f2-a51e-2965b0255afc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:34.258855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e4383-4d77-43f2-a51e-2965b0255afc admin admin] PATH_INFO: `/v3/projects/9164898c395c4ad49cf36430163a3b72/users/07ba36721b77427d96352f54367283ab/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:34.322359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e4383-4d77-43f2-a51e-2965b0255afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9164898c395c4ad49cf36430163a3b72, user_id=07ba36721b77427d96352f54367283ab, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:34.327117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e4383-4d77-43f2-a51e-2965b0255afc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:34.378013 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1365/2705] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:00:34 2022] PUT /identity/v3/projects/9164898c395c4ad49cf36430163a3b72/users/07ba36721b77427d96352f54367283ab/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:34.391415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:34.392158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:34.394311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:34.426489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81664877-b10d-4e03-90b2-cd56399edb58 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:34.449352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] MFA Rules not processed for user `07ba36721b77427d96352f54367283ab`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:34.451194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:34.511345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:34.544069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:34.598389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fb1f7-bfc0-4bab-9e4f-6c20b73ab444 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:34.606848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1366/2706] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:34.617144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:34.619309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:34.631902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fb1f7-bfc0-4bab-9e4f-6c20b73ab444 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:34.632470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fb1f7-bfc0-4bab-9e4f-6c20b73ab444 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:34.632855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fb1f7-bfc0-4bab-9e4f-6c20b73ab444 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:34.673339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bde8f-27f2-449b-9608-48bcf524b911 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:34.691380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:34.692504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:34.693084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:34.693568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:34.716158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:34.718440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:34.732040 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1367/2707] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:34.820469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fb1f7-bfc0-4bab-9e4f-6c20b73ab444 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:34.825038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fb1f7-bfc0-4bab-9e4f-6c20b73ab444 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:34.845379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1341/2708] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:00:35.134729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:35.282490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35abef-eb2d-4ffc-a286-3cac388f9586 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:35.300291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:35.300997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:35.301276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:35.301550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] PATH_INFO: `/v3/users/9e9418ffc1c249fd8d854fb4dc7be36f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:35.303077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9418ffc1c249fd8d854fb4dc7be36f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:35.306499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:35.317814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:35.318319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:35.318799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:35.422349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:35.426919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:35.500871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9418ffc1c249fd8d854fb4dc7be36f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:35.503858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9418ffc1c249fd8d854fb4dc7be36f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:35.525608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9418ffc1c249fd8d854fb4dc7be36f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:35.545343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9418ffc1c249fd8d854fb4dc7be36f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:35.550535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30ed697f-9b08-48fc-8969-05e06bf1420e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9418ffc1c249fd8d854fb4dc7be36f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:35.605624 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1368/2709] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:00:35 2022] DELETE /identity/v3/users/9e9418ffc1c249fd8d854fb4dc7be36f => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:35.608730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:35.609026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:35.609304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:35.664504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:35.666076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:35.672625 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1342/2710] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:00:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:35.702950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:35.706313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:35.879347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:35.949030 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1369/2711] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:35.977502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:35.979453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:35.994611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203650-eef7-4e2e-9ab0-d931e5840d38 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:36.032650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:36.118781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d0b88-b293-480e-985e-4c6c9ad31e5c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:36.126366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a1f1a-6be6-446f-854e-97f80e409378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:36.126814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a1f1a-6be6-446f-854e-97f80e409378 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:36.127086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a1f1a-6be6-446f-854e-97f80e409378 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:36.127353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a1f1a-6be6-446f-854e-97f80e409378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:36.147216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a1f1a-6be6-446f-854e-97f80e409378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:36.150255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a1f1a-6be6-446f-854e-97f80e409378 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:36.154543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:36.156098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:36.156098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:36.156098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:36.174014 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1370/2712] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:00:36.255373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:36.257655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:36.278185 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1343/2713] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:00:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:36.525523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe69ad9-09bb-49ef-8c88-c6a757017aea None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:36.526145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe69ad9-09bb-49ef-8c88-c6a757017aea None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:36.526145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe69ad9-09bb-49ef-8c88-c6a757017aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:36.561724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cbe69ad9-09bb-49ef-8c88-c6a757017aea None None] MFA Rules not processed for user `07ba36721b77427d96352f54367283ab`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:36.574158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cbe69ad9-09bb-49ef-8c88-c6a757017aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:36.777096 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1371/2714] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:37.025282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:37.027659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:37.053864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a422b8-c7ea-421e-87bd-a4157c381046 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:37.063062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:37.063774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.064054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.064327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] PATH_INFO: `/v3/projects/e2481f3613e4485a8424a23cc716a060` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.066208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2481f3613e4485a8424a23cc716a060)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:37.092028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:37.205322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2481f3613e4485a8424a23cc716a060', 'request_id': 'req-73b24ba2-e18d-4dad-af7c-1eb08e14e467'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:37.221434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2481f3613e4485a8424a23cc716a060', 'request_id': 'req-73b24ba2-e18d-4dad-af7c-1eb08e14e467'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:37.236467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2481f3613e4485a8424a23cc716a060', 'request_id': 'req-73b24ba2-e18d-4dad-af7c-1eb08e14e467'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:37.263147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] The token cache is being invalidate because project e2481f3613e4485a8424a23cc716a060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:00:37.263436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:37.264101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-73b24ba2-e18d-4dad-af7c-1eb08e14e467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:37.269246 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1344/2715] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:00:37 2022] DELETE /identity/v3/projects/e2481f3613e4485a8424a23cc716a060 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:37.271744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fc494-04b9-49e3-98f7-7bff1a6b36ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.272220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fc494-04b9-49e3-98f7-7bff1a6b36ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.272599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fc494-04b9-49e3-98f7-7bff1a6b36ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.337511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-924fc494-04b9-49e3-98f7-7bff1a6b36ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:37.348597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-924fc494-04b9-49e3-98f7-7bff1a6b36ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:37.519386 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1372/2716] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:00:37.530537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56afa28-b821-4838-9d34-7549b7de18de None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.530894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56afa28-b821-4838-9d34-7549b7de18de None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.531145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56afa28-b821-4838-9d34-7549b7de18de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.555584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a56afa28-b821-4838-9d34-7549b7de18de None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:37.557503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a56afa28-b821-4838-9d34-7549b7de18de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:37.576123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001fe44-72a3-4699-9d9d-b4a6cb14aff7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.577022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001fe44-72a3-4699-9d9d-b4a6cb14aff7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.577921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001fe44-72a3-4699-9d9d-b4a6cb14aff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.590043 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1345/2717] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:37.591508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56afa28-b821-4838-9d34-7549b7de18de None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:37.606246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56afa28-b821-4838-9d34-7549b7de18de None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:37.618698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:37.619801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.620111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.620352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.621077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:00:37.621379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:37.623658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:37.628053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b001fe44-72a3-4699-9d9d-b4a6cb14aff7 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:37.630121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b001fe44-72a3-4699-9d9d-b4a6cb14aff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:37.640366 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1346/2718] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:00:37.645072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:37.657743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249aaf5-e10c-47ac-8138-1a949e8ac52c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:37.663781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:37.665322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.665665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.665971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.666579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:37.669418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:37.674777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:37.705598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:37.717427 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1347/2719] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:00:37 2022] POST /identity/v3/projects => generated 403 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:00:37.717846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1373/2720] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:00:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:37.728607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001fe44-72a3-4699-9d9d-b4a6cb14aff7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:37.737701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:37.739647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:37.746931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001fe44-72a3-4699-9d9d-b4a6cb14aff7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:37.834391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fdc82-0230-44fe-aa13-a0c1b31bd574 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:37.845854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:37.846641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.846990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.847342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.875690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:37.878347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.878891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.879384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.881209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:37.883822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:37.929030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:37.930835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:37.933810 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1374/2721] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:00:37 2022] POST /identity/v3/users => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:37.938529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:37.942094 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1348/2722] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:00:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:00:37.961849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938972a-de97-4531-82c9-13a0d85abb02 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:37.970372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:37.971435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:37.973612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:37.975153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:37.977335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:37.980966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:38.008609 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1375/2723] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:00:38.012400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:38.033446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474f20-809a-4038-a3d5-6dd39793942e service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:38.054986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:38.055579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:38.055722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:38.055994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:38.057018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:38.058696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:38.084744 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1349/2724] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:00:38.092116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:38.115387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bdc31-1afd-41bf-b2c4-3c8a47300270 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:38.127793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cad46-5123-4616-aac3-bffe6ed30cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:38.128965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cad46-5123-4616-aac3-bffe6ed30cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:38.129631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cad46-5123-4616-aac3-bffe6ed30cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:38.129925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cad46-5123-4616-aac3-bffe6ed30cc3 admin admin] PATH_INFO: `/v3/projects/e5db36951642427481c9b79db0009118/users/546c5dc3d40c4e72ac683f084429b5f8/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:38.203169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cad46-5123-4616-aac3-bffe6ed30cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5db36951642427481c9b79db0009118, user_id=546c5dc3d40c4e72ac683f084429b5f8, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:38.208686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cad46-5123-4616-aac3-bffe6ed30cc3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:38.251228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:38.253184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:38.273637 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1376/2725] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:00:38 2022] PUT /identity/v3/projects/e5db36951642427481c9b79db0009118/users/546c5dc3d40c4e72ac683f084429b5f8/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:38.277342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:38.280486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:38.280908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:38.323987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e09470-0354-45de-af00-206aa33d8ea8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:38.336987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:38.338730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:38.480583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:38.481775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:38.482058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:38.482338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] PATH_INFO: `/v3/users/07ba36721b77427d96352f54367283ab` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:38.484529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ba36721b77427d96352f54367283ab)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:38.493158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:38.515925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1377/2726] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:38.522665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:38.524537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:38.583658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:38.612482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ba36721b77427d96352f54367283ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:38.614934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ba36721b77427d96352f54367283ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:38.631103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ba36721b77427d96352f54367283ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:38.642871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:38.651287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ba36721b77427d96352f54367283ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:38.693510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c57f15-cc6d-4b2b-992e-dbda40105c69 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:38.701910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:38.703033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:38.703530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:38.704008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:38.706178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e252684-a264-4ba0-a7b0-0c3c11df6436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ba36721b77427d96352f54367283ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:38.706896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:38.714251 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1350/2727] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:00:38 2022] DELETE /identity/v3/users/07ba36721b77427d96352f54367283ab => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:38.718576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:38.718862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:38.719134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:38.748493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:38.750121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:38.754368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:38.755908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:38.787265 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1378/2728] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:38.858781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1351/2729] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:00:38.873596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:38.875368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:38.901718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe258d88-be09-410a-b63c-5cd402afcb2b service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:39.081337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9b7da-aa5b-4f3c-b713-cad42e44a16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:39.088568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9b7da-aa5b-4f3c-b713-cad42e44a16b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:39.089211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9b7da-aa5b-4f3c-b713-cad42e44a16b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:39.090097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9b7da-aa5b-4f3c-b713-cad42e44a16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:39.126005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9b7da-aa5b-4f3c-b713-cad42e44a16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:39.132454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9b7da-aa5b-4f3c-b713-cad42e44a16b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:39.163532 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1379/2730] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:40.916012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:40.918435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:41.251740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa783f9-d5bc-4651-9cff-4249795a0d35 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:41.266219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:41.270105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:41.270439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:41.270771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] PATH_INFO: `/v3/projects/9164898c395c4ad49cf36430163a3b72` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:41.278228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9164898c395c4ad49cf36430163a3b72)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:41.280316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:41.393217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:41.401126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9164898c395c4ad49cf36430163a3b72', 'request_id': 'req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:41.421618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9164898c395c4ad49cf36430163a3b72', 'request_id': 'req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:41.436963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9164898c395c4ad49cf36430163a3b72', 'request_id': 'req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:41.463434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] The token cache is being invalidate because project 9164898c395c4ad49cf36430163a3b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:00:41.463764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:41.464434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97a0cc29-21a8-4a1f-b488-2c1b9152fe29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:41.469821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1352/2731] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:00:40 2022] DELETE /identity/v3/projects/9164898c395c4ad49cf36430163a3b72 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:41.471662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b273a1-9e44-47cd-9324-1f1a56107917 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:41.472157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b273a1-9e44-47cd-9324-1f1a56107917 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:41.472256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b273a1-9e44-47cd-9324-1f1a56107917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:41.660757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-50b273a1-9e44-47cd-9324-1f1a56107917 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:41.662405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-50b273a1-9e44-47cd-9324-1f1a56107917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:41.845398 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1380/2732] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:41.847552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fa56-d146-492d-b2ad-0ea31b4dea1a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:41.848047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fa56-d146-492d-b2ad-0ea31b4dea1a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:41.848598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fa56-d146-492d-b2ad-0ea31b4dea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:41.898471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b878fa56-d146-492d-b2ad-0ea31b4dea1a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:41.903674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b878fa56-d146-492d-b2ad-0ea31b4dea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:41.935878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b273a1-9e44-47cd-9324-1f1a56107917 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:41.943836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1353/2733] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:41.965505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b273a1-9e44-47cd-9324-1f1a56107917 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:41.972484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:41.973174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:41.973540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:41.973865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:41.974602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:00:41.974910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:41.979515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:42.028442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878fa56-d146-492d-b2ad-0ea31b4dea1a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:42.032896 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1381/2734] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:00:42.039836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878fa56-d146-492d-b2ad-0ea31b4dea1a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:42.053904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:42.054678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:42.055400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:42.055678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:42.056701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:42.058676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:42.153573 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1354/2735] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:00:42 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:42.160234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:42.195278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687b898-1eb4-4d6f-b4dd-ba60b3ee38fb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:42.296581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:42.374360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:42.374979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:42.375376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:42.375712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:42.376423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:42.378477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:42.432984 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1382/2736] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:00:42 2022] POST /identity/v3/users => generated 282 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:42.435054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:42.456860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80c39-9e63-410f-ab82-c713ad18d47b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:42.473154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:42.474143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:42.477176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:42.477718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:42.478577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:42.487701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:42.533995 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1355/2737] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:00:42.536176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:42.549914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7872e6-653c-47dc-aefc-f751f7e642f0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:42.562435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade20837-9b2a-4901-ad5b-fd196d2e9ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:42.563594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade20837-9b2a-4901-ad5b-fd196d2e9ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:42.567850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade20837-9b2a-4901-ad5b-fd196d2e9ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:42.568171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade20837-9b2a-4901-ad5b-fd196d2e9ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:42.569470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade20837-9b2a-4901-ad5b-fd196d2e9ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:42.575078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade20837-9b2a-4901-ad5b-fd196d2e9ba5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:42.615803 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1383/2738] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:00:42.621688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:42.635437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc8d33-c6e6-46d9-91fa-6587a374abaf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:42.648766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:42.649557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:42.650365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:42.652311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] PATH_INFO: `/v3/projects/e5a565f65b3849a99b74454791397ca5/users/6eb3ccfcd9c344e1a7ad55f8398ec87a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:42.692357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a565f65b3849a99b74454791397ca5, user_id=6eb3ccfcd9c344e1a7ad55f8398ec87a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:42.698805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:42.750536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:42.750983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:42.751344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:42.754440 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1356/2739] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:00:42 2022] PUT /identity/v3/projects/e5a565f65b3849a99b74454791397ca5/users/6eb3ccfcd9c344e1a7ad55f8398ec87a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:42.803886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:42.809091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:42.922345 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1384/2740] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:00:43.083925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:43.087134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:43.319175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ac6cd-24c2-41ba-89f0-5fb84805cb0f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:43.333000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:43.333656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:43.333994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:43.334266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:43.355944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:43.358208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:43.393194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1357/2741] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:45.371785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:45.374441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:45.423258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8b838-4318-4c08-ab82-d8722557c91f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:45.457507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:45.462261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:45.462584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:45.469104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] PATH_INFO: `/v3/users/57365b62b3294810b8a030d2a26cc451` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:45.477899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=57365b62b3294810b8a030d2a26cc451)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:45.491939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:45.701039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57365b62b3294810b8a030d2a26cc451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:45.709376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57365b62b3294810b8a030d2a26cc451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:45.721891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57365b62b3294810b8a030d2a26cc451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:45.738266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57365b62b3294810b8a030d2a26cc451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:45.739077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57365b62b3294810b8a030d2a26cc451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:45.788251 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1385/2742] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:00:45 2022] DELETE /identity/v3/users/57365b62b3294810b8a030d2a26cc451 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:00:46.618155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:46.620114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:46.724430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:46.730030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:46.732346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:46.817734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:46.871000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f26d2-d125-4a70-89ac-ddab1d3c7b0e service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:46.892284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003ea6a-5def-4281-abae-fb3221deb61b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:46.901691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:46.904622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:46.905211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:46.905682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] PATH_INFO: `/v3/projects/2b3380f3fa2b494da649aaa809e11368` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:46.907458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3380f3fa2b494da649aaa809e11368)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:46.910011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:47.030896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbfb92-0613-45da-b09f-1b92e08a6e79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:47.031448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbfb92-0613-45da-b09f-1b92e08a6e79 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:47.031727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbfb92-0613-45da-b09f-1b92e08a6e79 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:47.032000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbfb92-0613-45da-b09f-1b92e08a6e79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:47.033118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3380f3fa2b494da649aaa809e11368', 'request_id': 'req-516d0e7a-6114-41b1-9d9a-a1237361d480'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:47.036795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3bbbfb92-0613-45da-b09f-1b92e08a6e79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:47.066369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3380f3fa2b494da649aaa809e11368', 'request_id': 'req-516d0e7a-6114-41b1-9d9a-a1237361d480'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:47.085178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3380f3fa2b494da649aaa809e11368', 'request_id': 'req-516d0e7a-6114-41b1-9d9a-a1237361d480'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:47.137906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] The token cache is being invalidate because project 2b3380f3fa2b494da649aaa809e11368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:00:47.138401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:47.141315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-516d0e7a-6114-41b1-9d9a-a1237361d480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:00:47.145433 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1386/2743] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:00:46 2022] DELETE /identity/v3/projects/2b3380f3fa2b494da649aaa809e11368 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:00:47.152017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc4972-5095-4343-bde3-da744d4f6422 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:47.152573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc4972-5095-4343-bde3-da744d4f6422 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:47.154174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc4972-5095-4343-bde3-da744d4f6422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:47.200010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbfb92-0613-45da-b09f-1b92e08a6e79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:47.202707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbfb92-0613-45da-b09f-1b92e08a6e79 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:47.216883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-05bc4972-5095-4343-bde3-da744d4f6422 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:47.218357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-05bc4972-5095-4343-bde3-da744d4f6422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:47.230588 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1358/2744] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:47.326187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1387/2745] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:47.330182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a57329-8323-412a-a2cd-704756ee0bd6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:47.330499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a57329-8323-412a-a2cd-704756ee0bd6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:47.330938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a57329-8323-412a-a2cd-704756ee0bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:47.370204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a7a57329-8323-412a-a2cd-704756ee0bd6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:47.375032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a7a57329-8323-412a-a2cd-704756ee0bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:47.403178 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1359/2746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:47.406146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc4972-5095-4343-bde3-da744d4f6422 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:47.451087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc4972-5095-4343-bde3-da744d4f6422 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:47.482834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:47.484273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:47.484936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:47.485470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:47.488942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:00:47.491315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:47.493509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:47.512247 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1388/2747] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:00:47.518603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a57329-8323-412a-a2cd-704756ee0bd6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:47.555813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a57329-8323-412a-a2cd-704756ee0bd6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:47.575586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:47.576716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:47.581235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:47.582068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:47.586282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:47.590063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:47.647302 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1360/2748] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:00:47 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:47.652475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:47.685258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503928-d0b2-439f-afde-6517e7bb6efa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:47.803079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:47.844798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:47.845452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:47.845845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:47.846127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:47.846841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:47.849017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:47.896144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1389/2749] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:00:47 2022] POST /identity/v3/users => generated 295 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:00:47.900763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:47.925878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43e7b8-48d6-4b37-8725-7db62c1fd8e4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:47.940357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:47.941044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:47.941324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:47.941600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:47.942274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:47.944354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:47.972210 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1361/2750] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:00:47.974143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:47.996646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50116c-7ac4-4143-902e-1ff42a4c46c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:48.028056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64417607-663e-448b-a43e-ee7cc7c1816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:48.028056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64417607-663e-448b-a43e-ee7cc7c1816e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:48.028404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64417607-663e-448b-a43e-ee7cc7c1816e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:48.028630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64417607-663e-448b-a43e-ee7cc7c1816e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:48.029843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64417607-663e-448b-a43e-ee7cc7c1816e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:48.031880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64417607-663e-448b-a43e-ee7cc7c1816e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:48.065541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1390/2751] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:00:48.065780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:48.085155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cc792-dd9d-4006-a977-851c59eef5bd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:48.103096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:48.104215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:48.104716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:48.105142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] PATH_INFO: `/v3/projects/52e5e5d10e5a4da28c68aab50536bc08/users/424d1e6ad1ba435180f9de10d6beea14/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:48.134277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e5e5d10e5a4da28c68aab50536bc08, user_id=424d1e6ad1ba435180f9de10d6beea14, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:48.137609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:48.150225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:48.161768 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1362/2752] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:00:48 2022] PUT /identity/v3/projects/52e5e5d10e5a4da28c68aab50536bc08/users/424d1e6ad1ba435180f9de10d6beea14/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:00:48.167591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a37862-1118-41d1-8775-419bc1639ac8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:48.167880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a37862-1118-41d1-8775-419bc1639ac8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:48.168181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a37862-1118-41d1-8775-419bc1639ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:48.197633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-33a37862-1118-41d1-8775-419bc1639ac8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:48.199845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-33a37862-1118-41d1-8775-419bc1639ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:48.363277 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1391/2753] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:48.409713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:48.412079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:48.514068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89725b03-b313-42da-a0ce-468a95bd10c9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:48.518106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064ef79-0412-4dec-b001-7dd204012e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:48.518737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064ef79-0412-4dec-b001-7dd204012e5c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:48.519012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064ef79-0412-4dec-b001-7dd204012e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:48.519276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064ef79-0412-4dec-b001-7dd204012e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:48.543600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064ef79-0412-4dec-b001-7dd204012e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:48.555900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064ef79-0412-4dec-b001-7dd204012e5c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:48.579770 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1363/2754] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:53.900520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15861cfa-e2bc-4ca2-b11e-14307eb7d40a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:53.901247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15861cfa-e2bc-4ca2-b11e-14307eb7d40a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:53.901247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15861cfa-e2bc-4ca2-b11e-14307eb7d40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:53.980431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-15861cfa-e2bc-4ca2-b11e-14307eb7d40a None None] MFA Rules not processed for user `dce01d3212934161a38be853b4cd93ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:53.981948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15861cfa-e2bc-4ca2-b11e-14307eb7d40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:54.043968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:54.044543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:54.044688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:54.074748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] MFA Rules not processed for user `0d563feacff84b4c9927ec759fb365be`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:54.078344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:54.195216 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1392/2755] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:54.284137 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1364/2756] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:57.026486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6dd98-aaf8-4c74-a4b0-be7a14d880fe None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:57.027356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6dd98-aaf8-4c74-a4b0-be7a14d880fe None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:57.027859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6dd98-aaf8-4c74-a4b0-be7a14d880fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:57.115020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f5e6dd98-aaf8-4c74-a4b0-be7a14d880fe None None] MFA Rules not processed for user `546c5dc3d40c4e72ac683f084429b5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:00:57.116299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f5e6dd98-aaf8-4c74-a4b0-be7a14d880fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:57.266979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:57.280062 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1393/2757] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:00:57 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:00:57.816301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:00:57.817764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:00:58.017186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c9a85-b124-4f87-9f81-5b45cad6ce1f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:00:58.024542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:00:58.025279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:00:58.025608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:00:58.026045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:00:58.047880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:00:58.062729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:00:58.080230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1365/2758] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:00:57 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:01.031583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b36b7e-4ac6-4ea9-b882-b183942432fd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:01.032549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b36b7e-4ac6-4ea9-b882-b183942432fd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:01.032852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b36b7e-4ac6-4ea9-b882-b183942432fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:01.150967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-90b36b7e-4ac6-4ea9-b882-b183942432fd None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:01.157380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-90b36b7e-4ac6-4ea9-b882-b183942432fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:01.292182 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1394/2759] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:01:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:01.311134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:01.312941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:01.426754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:01.522101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c73888-ec09-4343-96c0-e76eed710d9a service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:01.529668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa928740-ef54-4005-af3f-e698dbceab25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:01.530750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa928740-ef54-4005-af3f-e698dbceab25 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:01.531174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa928740-ef54-4005-af3f-e698dbceab25 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:01.531590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa928740-ef54-4005-af3f-e698dbceab25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:01.544583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa928740-ef54-4005-af3f-e698dbceab25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:01.549538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa928740-ef54-4005-af3f-e698dbceab25 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:01.579115 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1366/2760] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:02.217282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:02.218168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:02.218641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:02.333438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] MFA Rules not processed for user `6eb3ccfcd9c344e1a7ad55f8398ec87a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:02.334708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:02.543197 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1395/2761] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:02.551227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8e08-6f1d-4a14-9980-113db156a613 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:02.553036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8e08-6f1d-4a14-9980-113db156a613 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:02.554193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8e08-6f1d-4a14-9980-113db156a613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:02.606364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6d3d8e08-6f1d-4a14-9980-113db156a613 None None] MFA Rules not processed for user `6eb3ccfcd9c344e1a7ad55f8398ec87a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:02.606364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6d3d8e08-6f1d-4a14-9980-113db156a613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:02.618692 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1367/2762] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:01:04.908036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:04.909529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:04.944885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2b673-7bad-4afc-8315-f4136a13a986 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:04.956588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:04.957460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:04.957947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:04.958394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:04.974171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:04.978004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:05.011846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1396/2763] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:01:12.589765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=21 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:13.837177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:13.837808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:13.837808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:13.877404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] MFA Rules not processed for user `424d1e6ad1ba435180f9de10d6beea14`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:13.878661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:14.007066 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1368/2764] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:14.508126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:14.520987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047acca-5a58-4c75-8731-1cfb9c296739 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:14.534053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:14.534910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:14.535290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:14.535654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:14.552212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:14.555554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:14.574609 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1397/2765] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:23.232393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:23.234893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:23.260964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922d26-ae6e-42ad-8cf3-967c4e1e55b5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:23.265163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:23.266074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:23.266504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:23.266939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:23.269025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:23.299885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:23.301324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:23.336930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1369/2766] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:24.569147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=73 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:30.946114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:30.969459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22260c-23dd-443f-bf48-3031921f3d81 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:30.974625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:30.975483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:30.975986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:30.976591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:30.981751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:30.999844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:31.002029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:31.015657 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1398/2767] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:31.348846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:31.359862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b19d00-5724-498c-b7f7-781bd07caca8 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:31.366250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:31.367311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:31.367835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:31.368356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] PATH_INFO: `/v3/users/424d1e6ad1ba435180f9de10d6beea14` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:31.372865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=424d1e6ad1ba435180f9de10d6beea14)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:31.387952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:31.529711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '424d1e6ad1ba435180f9de10d6beea14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:31.530664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '424d1e6ad1ba435180f9de10d6beea14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:31.556191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '424d1e6ad1ba435180f9de10d6beea14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:31.580924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '424d1e6ad1ba435180f9de10d6beea14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:31.634570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '424d1e6ad1ba435180f9de10d6beea14'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:31.654015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1370/2768] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:31 2022] DELETE /identity/v3/users/424d1e6ad1ba435180f9de10d6beea14 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:32.159584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:32.161619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:32.176524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519598c-ecdd-43d7-9209-cbb81f80d46e service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:32.247077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:32.247863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:32.248377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:32.248938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] PATH_INFO: `/v3/users/b4cebc85e62046d28fab159c6bb13e15` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:32.250366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4cebc85e62046d28fab159c6bb13e15)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:32.252830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:32.352169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4cebc85e62046d28fab159c6bb13e15'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:32.352877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4cebc85e62046d28fab159c6bb13e15'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:32.384316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4cebc85e62046d28fab159c6bb13e15'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:32.411936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4cebc85e62046d28fab159c6bb13e15'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:32.414503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4cebc85e62046d28fab159c6bb13e15'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:32.470446 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1399/2769] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:32 2022] DELETE /identity/v3/users/b4cebc85e62046d28fab159c6bb13e15 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:33.037304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:33.038952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:33.077965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824fff4-fd4b-48f2-8e11-975708a998c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:33.160164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:33.205010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:33.208593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:33.209948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:33.211279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] PATH_INFO: `/v3/projects/52e5e5d10e5a4da28c68aab50536bc08` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:33.213137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52e5e5d10e5a4da28c68aab50536bc08)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:33.215954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:33.268305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:33.269769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:33.357516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e5e5d10e5a4da28c68aab50536bc08', 'request_id': 'req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.376142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e5e5d10e5a4da28c68aab50536bc08', 'request_id': 'req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.395802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e5e5d10e5a4da28c68aab50536bc08', 'request_id': 'req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.414255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] The token cache is being invalidate because project 52e5e5d10e5a4da28c68aab50536bc08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:33.416205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.419091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2900d15c-afb4-4e6d-9f52-b3b8d04e8829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.424992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1371/2770] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:33 2022] DELETE /identity/v3/projects/52e5e5d10e5a4da28c68aab50536bc08 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:33.429959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:33.430321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:33.430509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:33.465191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45342d8-6839-4bc8-a5c3-c2dc740845d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:33.483800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:33.485139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:33.516000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:33.580563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb561232-bd69-4c4e-be4c-53a46764c4e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:33.581700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561232-bd69-4c4e-be4c-53a46764c4e5 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:33.581941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561232-bd69-4c4e-be4c-53a46764c4e5 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:33.582266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb561232-bd69-4c4e-be4c-53a46764c4e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:33.587214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cb561232-bd69-4c4e-be4c-53a46764c4e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:33.655040 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1372/2771] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:33.665925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:33.668992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:33.710713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6925f-ab8f-4f83-9184-aec2406267f6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:33.725505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:33.726666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:33.727149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:33.727662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] PATH_INFO: `/v3/projects/1881e013b4b844cba1afce9fd21f8dd7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:33.729319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1881e013b4b844cba1afce9fd21f8dd7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:33.731437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:33.745196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb561232-bd69-4c4e-be4c-53a46764c4e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:33.748269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb561232-bd69-4c4e-be4c-53a46764c4e5 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:33.784006 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1400/2772] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:01:33.785932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296965d-109b-4e25-9ecb-77b3b4d2f59b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:33.786459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296965d-109b-4e25-9ecb-77b3b4d2f59b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:33.786938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296965d-109b-4e25-9ecb-77b3b4d2f59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:33.829001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1881e013b4b844cba1afce9fd21f8dd7', 'request_id': 'req-08224377-c83a-4a9e-b9fc-2314677f4d9a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.841762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e296965d-109b-4e25-9ecb-77b3b4d2f59b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:33.850098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e296965d-109b-4e25-9ecb-77b3b4d2f59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:33.863186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1881e013b4b844cba1afce9fd21f8dd7', 'request_id': 'req-08224377-c83a-4a9e-b9fc-2314677f4d9a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.880591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1881e013b4b844cba1afce9fd21f8dd7', 'request_id': 'req-08224377-c83a-4a9e-b9fc-2314677f4d9a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.900159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] The token cache is being invalidate because project 1881e013b4b844cba1afce9fd21f8dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:33.900159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.901208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:33.904621 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1373/2773] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:33 2022] DELETE /identity/v3/projects/1881e013b4b844cba1afce9fd21f8dd7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:33.908582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:33.912799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:33.922365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:33.993980 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1401/2774] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:33.995084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296965d-109b-4e25-9ecb-77b3b4d2f59b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:34.007839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296965d-109b-4e25-9ecb-77b3b4d2f59b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:34.013951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:34.014872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.015150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.015421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.016494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:34.016836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:34.019047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:34.048591 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1402/2775] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:34.049724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:34.056809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224377-c83a-4a9e-b9fc-2314677f4d9a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:34.062211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ac026-f71f-43a7-b15e-682ed3b7a91b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:34.066261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:34.067462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.067992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.068513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] PATH_INFO: `/v3/users/5cc03c5c36f14c389cb76fa0b4dc78e0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.069142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:34.070123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.070400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.070791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.071535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc03c5c36f14c389cb76fa0b4dc78e0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:34.071703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:34.074085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:34.074369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:34.125127 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1403/2776] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:01:34 2022] POST /identity/v3/projects => generated 401 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:34.135037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:34.185943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:34.187802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc03c5c36f14c389cb76fa0b4dc78e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:34.188599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc03c5c36f14c389cb76fa0b4dc78e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:34.204455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc03c5c36f14c389cb76fa0b4dc78e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:34.225230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc03c5c36f14c389cb76fa0b4dc78e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:34.249194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d6cccd5-be07-451b-8b75-bc531587ed94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc03c5c36f14c389cb76fa0b4dc78e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:34.282328 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1374/2777] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:33 2022] DELETE /identity/v3/users/5cc03c5c36f14c389cb76fa0b4dc78e0 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:34.283832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.284184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.284530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.296483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ea926-2c43-43be-8fb5-fff6f2419828 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:34.330401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] MFA Rules not processed for user `9544cdb0392f416197a586d99e8a3210`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:34.331775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:34.426631 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1375/2778] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:34.462014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:34.462893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.463507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.463901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.464845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:34.467348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:34.503847 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1404/2779] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:01:34 2022] POST /identity/v3/users => generated 301 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:34.507877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:34.510560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:34.537300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699124b-af69-4043-9cf8-2c84739237bd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:34.542291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:34.543741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.544027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.544299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.545039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:34.546646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:34.560717 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1376/2780] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:34.562494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:34.578296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532f654-05ff-4d8e-bcf2-fb46abb501f2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:34.587381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc42e3d-5024-4287-95bd-8125a936964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:34.588252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42e3d-5024-4287-95bd-8125a936964e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.588622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42e3d-5024-4287-95bd-8125a936964e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.589005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42e3d-5024-4287-95bd-8125a936964e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.589707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc42e3d-5024-4287-95bd-8125a936964e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:34.594047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc42e3d-5024-4287-95bd-8125a936964e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:34.625294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:34.625603 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1405/2781] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:34.641870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cf742-39b6-4e8b-ae8b-cf909ff66090 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:34.656517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:34.657330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.657787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.658062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] PATH_INFO: `/v3/projects/4d37d16753ac4c68aa1f1fac7925d857/users/6d6bc44a06384fa4bc0aba86ea50e52c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.677775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:34.682913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d37d16753ac4c68aa1f1fac7925d857, user_id=6d6bc44a06384fa4bc0aba86ea50e52c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:34.686390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:34.711667 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1377/2782] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:34 2022] PUT /identity/v3/projects/4d37d16753ac4c68aa1f1fac7925d857/users/6d6bc44a06384fa4bc0aba86ea50e52c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:34.717382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:34.717700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:34.717933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:34.762160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:34.763477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:34.928797 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1406/2783] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:34.994974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:35.000943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:35.092843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953bb8a-18ea-4686-ac58-aa8e3734961c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:35.104651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:35.106036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9e6e8-7f9d-4103-959f-86b163f48fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:35.106689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9e6e8-7f9d-4103-959f-86b163f48fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.106965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9e6e8-7f9d-4103-959f-86b163f48fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.107104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:35.107265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9e6e8-7f9d-4103-959f-86b163f48fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.127118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84b76f-343c-4c2b-a2f2-1c19aacc12d4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:35.135281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:35.136360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.136878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.137513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] PATH_INFO: `/v3/projects/4d40f3dbcee3467f92fc9bcbd4bde2eb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.138961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d40f3dbcee3467f92fc9bcbd4bde2eb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:35.143910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:35.153962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9e6e8-7f9d-4103-959f-86b163f48fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:35.155490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9e6e8-7f9d-4103-959f-86b163f48fe7 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:35.179637 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1378/2784] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:35.278155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d40f3dbcee3467f92fc9bcbd4bde2eb', 'request_id': 'req-ceb767ff-fb49-49b2-937e-f694a4369687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:35.280270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:35.302065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d40f3dbcee3467f92fc9bcbd4bde2eb', 'request_id': 'req-ceb767ff-fb49-49b2-937e-f694a4369687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:35.314051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d40f3dbcee3467f92fc9bcbd4bde2eb', 'request_id': 'req-ceb767ff-fb49-49b2-937e-f694a4369687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:35.344603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] The token cache is being invalidate because project 4d40f3dbcee3467f92fc9bcbd4bde2eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:35.344884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:35.346419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ceb767ff-fb49-49b2-937e-f694a4369687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:35.349565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1407/2785] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:35 2022] DELETE /identity/v3/projects/4d40f3dbcee3467f92fc9bcbd4bde2eb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:35.356188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.356188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.356745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.389855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:35.391452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:35.499709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1379/2786] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:35.508513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37595e-1a29-4413-84af-300802d1d3aa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.508855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37595e-1a29-4413-84af-300802d1d3aa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.509583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37595e-1a29-4413-84af-300802d1d3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.533811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:35.535247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:35.564635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1b37595e-1a29-4413-84af-300802d1d3aa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:35.568930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1b37595e-1a29-4413-84af-300802d1d3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:35.602960 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1408/2787] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:35.607648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37595e-1a29-4413-84af-300802d1d3aa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:35.623964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37595e-1a29-4413-84af-300802d1d3aa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:35.637487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:35.638296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.639174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.639682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.640294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:35.640780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:35.642812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:35.661046 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1409/2788] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:01:35.665377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:35.680590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508a4bc-28f5-4d56-a18a-91baf385712d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:35.695219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:35.695665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.695935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.696198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.696887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:35.722523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:35.735903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec6edf-2946-42e6-b33d-5cb35ce7c46c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:35.748439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:35.749252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.749542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.749819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.753260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:35.776277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:35.777610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1410/2789] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:01:35 2022] POST /identity/v3/projects => generated 395 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:35.779400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:35.782687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:35.800277 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1380/2790] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:01:35.906500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20592e7e-f602-4de2-b024-707ba50b91db admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:35.922372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:35.923020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:35.923301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:35.923561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:35.924203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:35.926192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:35.982261 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1411/2791] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:01:35 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:35.985436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:36.011999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc0ca5-4081-4e15-b2f4-bf1a5e8d9bf2 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:36.028997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:36.031096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:36.032303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:36.032812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:36.034814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:36.037521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:36.074210 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1381/2792] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:36.080453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:36.097775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a745f2-9d69-433c-b826-a189d26c5322 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:36.110330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec2ae6-1d0e-463d-9aa3-44180c578b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:36.111198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec2ae6-1d0e-463d-9aa3-44180c578b5c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:36.113973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec2ae6-1d0e-463d-9aa3-44180c578b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:36.114430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec2ae6-1d0e-463d-9aa3-44180c578b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:36.116430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec2ae6-1d0e-463d-9aa3-44180c578b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:36.121710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec2ae6-1d0e-463d-9aa3-44180c578b5c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:36.149793 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1412/2793] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:36.161489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:36.197448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b1961-9cf3-46a0-8d31-455722b44f01 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:36.209031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:36.210714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:36.212428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:36.212996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] PATH_INFO: `/v3/projects/cc9558c40f854a13a08b704a756bf926/users/9c8b2436fbb34407b9a3a0a1b3e89978/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:36.245149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:36.251565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc9558c40f854a13a08b704a756bf926, user_id=9c8b2436fbb34407b9a3a0a1b3e89978, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:36.256018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:36.284333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1382/2794] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:36 2022] PUT /identity/v3/projects/cc9558c40f854a13a08b704a756bf926/users/9c8b2436fbb34407b9a3a0a1b3e89978/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:36.298376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8915c-ee69-4e04-b901-676bd628906f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:36.298665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8915c-ee69-4e04-b901-676bd628906f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:36.298992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8915c-ee69-4e04-b901-676bd628906f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:36.354498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2fb8915c-ee69-4e04-b901-676bd628906f None None] MFA Rules not processed for user `9c8b2436fbb34407b9a3a0a1b3e89978`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:36.356189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2fb8915c-ee69-4e04-b901-676bd628906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:36.561034 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1413/2795] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:36.568098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:36.743822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5e86-de3c-4d3b-951d-955ed6f2c47f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:36.754926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:36.756068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:36.756599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:36.756959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:36.757904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:36.761332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:36.824464 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1383/2796] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:01:36 2022] POST /identity/v3/projects => generated 393 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:36.828604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8915c-ee69-4e04-b901-676bd628906f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:36.844469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:36.854929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8915c-ee69-4e04-b901-676bd628906f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:36.998779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:37.000431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:37.001127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:37.001598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:37.008244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:37.010779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:37.053326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1414/2797] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:01:36 2022] POST /identity/v3/users => generated 303 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:37.054949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:37.073035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d9df0-2d76-48a0-ac78-fe8f0fad5e37 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:37.081268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:37.082699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:37.082699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:37.082699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:37.083318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:37.085544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:37.107118 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1384/2798] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:37.108751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:37.124894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f082e76-be1a-4352-b8c7-782adac2a17b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:37.130011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:37.131320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:37.131652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:37.133482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] PATH_INFO: `/v3/projects/258f96d6863b402e8606fa5fe7f53547/users/94ef78e414e24c7a9f0907a191afd168/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:37.176534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258f96d6863b402e8606fa5fe7f53547, user_id=94ef78e414e24c7a9f0907a191afd168, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:37.181937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:37.214490 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1415/2799] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:37 2022] PUT /identity/v3/projects/258f96d6863b402e8606fa5fe7f53547/users/94ef78e414e24c7a9f0907a191afd168/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:37.219489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:37.244090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04fe43-7239-49e8-933c-065b2c597d21 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:37.395373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263577c0-1986-4607-adc0-96032de3d597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:37.397842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263577c0-1986-4607-adc0-96032de3d597 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:37.399489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263577c0-1986-4607-adc0-96032de3d597 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:37.400702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263577c0-1986-4607-adc0-96032de3d597 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:37.403536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263577c0-1986-4607-adc0-96032de3d597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:37.420705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263577c0-1986-4607-adc0-96032de3d597 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:37.442264 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1385/2800] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:37.446891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:37.479367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cd09a-06d8-4041-9d3b-f2b4d0523045 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:37.495721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae302-7f39-40f0-b674-db8dc379b28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:37.497375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae302-7f39-40f0-b674-db8dc379b28d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:37.497882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae302-7f39-40f0-b674-db8dc379b28d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:37.498239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae302-7f39-40f0-b674-db8dc379b28d admin admin] PATH_INFO: `/v3/projects/258f96d6863b402e8606fa5fe7f53547/users/94ef78e414e24c7a9f0907a191afd168/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:37.501453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bae302-7f39-40f0-b674-db8dc379b28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=258f96d6863b402e8606fa5fe7f53547, user_id=94ef78e414e24c7a9f0907a191afd168, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:37.520875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bae302-7f39-40f0-b674-db8dc379b28d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:37.543976 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1416/2801] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:37 2022] PUT /identity/v3/projects/258f96d6863b402e8606fa5fe7f53547/users/94ef78e414e24c7a9f0907a191afd168/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:37.557786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:37.558082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:37.558360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:37.623613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] MFA Rules not processed for user `94ef78e414e24c7a9f0907a191afd168`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:37.625072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:37.720078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3dd6-d24a-441a-a3b3-7633f8d95703 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:37.720686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3dd6-d24a-441a-a3b3-7633f8d95703 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:37.721171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3dd6-d24a-441a-a3b3-7633f8d95703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:37.737425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:37.758170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-db7e3dd6-d24a-441a-a3b3-7633f8d95703 None None] MFA Rules not processed for user `29d2f7b0e8274c769521d13930be9825`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:37.760554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-db7e3dd6-d24a-441a-a3b3-7633f8d95703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:37.798784 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1386/2802] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:37.815870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:37.817796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:37.911387 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1417/2803] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:37.989379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2feaf9-2c95-47c5-8151-7dce2fa3d476 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:37.999381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f172d73-cc18-42de-bb5c-33dc243d0872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:38.000000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f172d73-cc18-42de-bb5c-33dc243d0872 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:38.000332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f172d73-cc18-42de-bb5c-33dc243d0872 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:38.000718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f172d73-cc18-42de-bb5c-33dc243d0872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:38.012336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f172d73-cc18-42de-bb5c-33dc243d0872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:38.014083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f172d73-cc18-42de-bb5c-33dc243d0872 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:38.028420 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1387/2804] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:01:38.132327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3dd6-d24a-441a-a3b3-7633f8d95703 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:38.300943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3dd6-d24a-441a-a3b3-7633f8d95703 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:38.309437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:38.311330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:38.311818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:38.312286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] PATH_INFO: `/v3/users/9c8b2436fbb34407b9a3a0a1b3e89978` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:38.313877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8b2436fbb34407b9a3a0a1b3e89978)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:38.317321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:38.324441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:38.418796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181158a-99b9-4d19-865f-88e503bef97f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:38.422382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181158a-99b9-4d19-865f-88e503bef97f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:38.423443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181158a-99b9-4d19-865f-88e503bef97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:38.469592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8b2436fbb34407b9a3a0a1b3e89978'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.470593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8b2436fbb34407b9a3a0a1b3e89978'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.485805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c181158a-99b9-4d19-865f-88e503bef97f None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:38.487581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8b2436fbb34407b9a3a0a1b3e89978'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.489396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c181158a-99b9-4d19-865f-88e503bef97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:38.504914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8b2436fbb34407b9a3a0a1b3e89978'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.505877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8b2436fbb34407b9a3a0a1b3e89978'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.572774 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1418/2805] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:38 2022] DELETE /identity/v3/users/9c8b2436fbb34407b9a3a0a1b3e89978 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:38.574516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:38.576207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:38.646032 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1388/2806] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:01:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:01:38.647439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:38.647750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:38.648025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:38.693543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:38.694832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:38.696752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b333a-f129-40f9-a31b-e9004258aed9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:38.715852 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1389/2807] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:38.717248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:38.721390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:38.723691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:38.724927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:38.728019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:38.729134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] PATH_INFO: `/v3/users/546c5dc3d40c4e72ac683f084429b5f8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:38.732924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546c5dc3d40c4e72ac683f084429b5f8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:38.736200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:38.857494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546c5dc3d40c4e72ac683f084429b5f8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.859006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546c5dc3d40c4e72ac683f084429b5f8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.878156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546c5dc3d40c4e72ac683f084429b5f8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.891532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546c5dc3d40c4e72ac683f084429b5f8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.893182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546c5dc3d40c4e72ac683f084429b5f8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:38.899590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06baff0f-93ba-44f2-872f-294cbb225685 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:38.914215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:38.922055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:38.922771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:38.923114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:38.923452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:38.933521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:38.936551 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1419/2808] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:38 2022] DELETE /identity/v3/users/546c5dc3d40c4e72ac683f084429b5f8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:38.937777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:38.942952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:38.954008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:39.065772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448747c-0f9b-4df1-8c7e-6d9e8ed878d7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:39.071042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4d26c-6e77-4ffa-8db9-3677ee7255bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:39.072122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4d26c-6e77-4ffa-8db9-3677ee7255bc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:39.072920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4d26c-6e77-4ffa-8db9-3677ee7255bc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:39.073527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4d26c-6e77-4ffa-8db9-3677ee7255bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:39.078105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-21a4d26c-6e77-4ffa-8db9-3677ee7255bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:39.102344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:39.104776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:39.121319 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1390/2809] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:01:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:39.212240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4d26c-6e77-4ffa-8db9-3677ee7255bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:39.214580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4d26c-6e77-4ffa-8db9-3677ee7255bc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:39.230509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1420/2810] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:39.709687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:39.712411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:39.751286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f135f6-3658-45b1-b4c6-72fe2516f22c service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:39.769662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:39.770424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:39.770711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:39.774452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] PATH_INFO: `/v3/projects/e5db36951642427481c9b79db0009118` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:39.776820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5db36951642427481c9b79db0009118)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:39.783225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:39.910308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5db36951642427481c9b79db0009118', 'request_id': 'req-0cca8946-9293-4a5f-835e-00295b3298e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:39.952323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5db36951642427481c9b79db0009118', 'request_id': 'req-0cca8946-9293-4a5f-835e-00295b3298e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:39.975430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5db36951642427481c9b79db0009118', 'request_id': 'req-0cca8946-9293-4a5f-835e-00295b3298e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:39.996038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] The token cache is being invalidate because project e5db36951642427481c9b79db0009118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:39.996340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:39.998543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cca8946-9293-4a5f-835e-00295b3298e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:40.001415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1391/2811] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:39 2022] DELETE /identity/v3/projects/e5db36951642427481c9b79db0009118 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:40.006834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c5b0-9c4f-4a97-804b-c5e6d20e9c98 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.007360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c5b0-9c4f-4a97-804b-c5e6d20e9c98 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.007682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c5b0-9c4f-4a97-804b-c5e6d20e9c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.058339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-44d1c5b0-9c4f-4a97-804b-c5e6d20e9c98 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:40.066137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44d1c5b0-9c4f-4a97-804b-c5e6d20e9c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:40.199161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1421/2812] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:01:40.219300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8856706-5677-40be-b151-572c3701f5d8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.219650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8856706-5677-40be-b151-572c3701f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.219864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8856706-5677-40be-b151-572c3701f5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.251938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f8856706-5677-40be-b151-572c3701f5d8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:40.258699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f8856706-5677-40be-b151-572c3701f5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:40.278932 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1392/2813] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:40.289674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1c5b0-9c4f-4a97-804b-c5e6d20e9c98 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:40.320345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1c5b0-9c4f-4a97-804b-c5e6d20e9c98 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:40.339076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:40.339824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.340101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.340368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.341052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:40.342989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:40.344674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:40.385997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8856706-5677-40be-b151-572c3701f5d8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:40.396097 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1422/2814] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:40.399178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8856706-5677-40be-b151-572c3701f5d8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:40.411587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:40.412477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.413183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.413183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.415675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:40.417900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:40.496770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:40.505520 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1393/2815] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:01:40 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:01:40.514530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282dc70-ad0c-4cf1-90ca-a51ef5bbc200 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:40.646718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:40.695853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:40.698639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.699508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.699978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.704775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:40.705633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:40.765826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:40.776543 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1423/2816] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:01:40 2022] POST /identity/v3/users => generated 296 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:40.781806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff53a94-adf8-422c-a70f-c5dcc275deed admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:40.798565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:40.800524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.801716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.802169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.803872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:40.808556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:40.832575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:40.844414 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1394/2817] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:40.849202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40625b9b-6e90-4801-82b7-876420175e02 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:40.854623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784eebc-ec7b-4d8e-8811-d1e7e6dee5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:40.855433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784eebc-ec7b-4d8e-8811-d1e7e6dee5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.855723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784eebc-ec7b-4d8e-8811-d1e7e6dee5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.856001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784eebc-ec7b-4d8e-8811-d1e7e6dee5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.856930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784eebc-ec7b-4d8e-8811-d1e7e6dee5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:40.858745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784eebc-ec7b-4d8e-8811-d1e7e6dee5a3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:40.886746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:40.900146 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1424/2818] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:40.903427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3badd-912d-453d-9057-09e99fb08782 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:40.909436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:40.910531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:40.910964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:40.911392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] PATH_INFO: `/v3/projects/2750135860944b03816344b2e7e58f30/users/b31b5932bb8e4edcadc6954c46c2c386/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:40.974774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2750135860944b03816344b2e7e58f30, user_id=b31b5932bb8e4edcadc6954c46c2c386, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:41.004676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:41.082640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:41.083250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:41.083250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:41.094785 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1395/2819] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:40 2022] PUT /identity/v3/projects/2750135860944b03816344b2e7e58f30/users/b31b5932bb8e4edcadc6954c46c2c386/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:41.150390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:41.158432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:41.445365 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1425/2820] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:01:41.457222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:41.459178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:41.561631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:41.683486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c044fa0-d4f9-43af-ad60-f1460b053a6a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:41.711506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:41.712828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:41.712969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:41.713672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:41.739910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:41.749682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:41.801133 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1396/2821] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:42.237021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:42.239708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:42.257439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f30e0-6387-4d59-b3f2-23273424f66e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:42.263503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:42.264629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:42.265136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:42.265431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] PATH_INFO: `/v3/projects/cc9558c40f854a13a08b704a756bf926` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:42.267663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc9558c40f854a13a08b704a756bf926)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:42.270058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:42.374823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc9558c40f854a13a08b704a756bf926', 'request_id': 'req-87820e0b-b445-4317-9e23-476a1d81d0a8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.406895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc9558c40f854a13a08b704a756bf926', 'request_id': 'req-87820e0b-b445-4317-9e23-476a1d81d0a8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.422110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc9558c40f854a13a08b704a756bf926', 'request_id': 'req-87820e0b-b445-4317-9e23-476a1d81d0a8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.457340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] The token cache is being invalidate because project cc9558c40f854a13a08b704a756bf926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:42.457938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.459059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.469005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1426/2822] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:42 2022] DELETE /identity/v3/projects/cc9558c40f854a13a08b704a756bf926 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:42.473145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:42.492847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497d37-5684-4f2e-a181-f1d1b7f132e4 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:42.592159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:42.593177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:42.594137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:42.595229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] PATH_INFO: `/v3/users/94ef78e414e24c7a9f0907a191afd168` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:42.597905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ef78e414e24c7a9f0907a191afd168)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:42.602397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:42.762320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ef78e414e24c7a9f0907a191afd168'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.763469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ef78e414e24c7a9f0907a191afd168'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.784303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ef78e414e24c7a9f0907a191afd168'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.814768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ef78e414e24c7a9f0907a191afd168'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.840277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-011e8939-92b3-49a1-8642-769359eb5e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ef78e414e24c7a9f0907a191afd168'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:42.860743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1397/2823] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:42 2022] DELETE /identity/v3/users/94ef78e414e24c7a9f0907a191afd168 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:44.810204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:44.814826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:44.855290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87820e0b-b445-4317-9e23-476a1d81d0a8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:44.984581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:45.035033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:45.035423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:45.035952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:45.035952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] PATH_INFO: `/v3/projects/258f96d6863b402e8606fa5fe7f53547` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:45.037807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258f96d6863b402e8606fa5fe7f53547)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:45.042240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:45.175916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258f96d6863b402e8606fa5fe7f53547', 'request_id': 'req-3a4025f9-049d-4899-b22e-6687b37f1980'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:45.201990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258f96d6863b402e8606fa5fe7f53547', 'request_id': 'req-3a4025f9-049d-4899-b22e-6687b37f1980'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:45.208765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258f96d6863b402e8606fa5fe7f53547', 'request_id': 'req-3a4025f9-049d-4899-b22e-6687b37f1980'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:45.235019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] The token cache is being invalidate because project 258f96d6863b402e8606fa5fe7f53547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:45.235290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:45.237226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a4025f9-049d-4899-b22e-6687b37f1980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:45.246403 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1427/2824] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:44 2022] DELETE /identity/v3/projects/258f96d6863b402e8606fa5fe7f53547 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:45.249647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73977e52-8cf1-45ab-a928-ff9d785a5a17 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:45.250175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73977e52-8cf1-45ab-a928-ff9d785a5a17 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:45.250528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73977e52-8cf1-45ab-a928-ff9d785a5a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:45.348622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-73977e52-8cf1-45ab-a928-ff9d785a5a17 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:45.350493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-73977e52-8cf1-45ab-a928-ff9d785a5a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:45.475135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:45.541949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a19c73-2858-458b-8ba3-9ce528b75908 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:45.542503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a19c73-2858-458b-8ba3-9ce528b75908 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:45.542777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a19c73-2858-458b-8ba3-9ce528b75908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:45.552261 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1398/2825] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:45.624958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a6a19c73-2858-458b-8ba3-9ce528b75908 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:45.626212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a6a19c73-2858-458b-8ba3-9ce528b75908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:45.663733 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1428/2826] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:01:45.668863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73977e52-8cf1-45ab-a928-ff9d785a5a17 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:45.682132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73977e52-8cf1-45ab-a928-ff9d785a5a17 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:45.703483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:45.704129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:45.704410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:45.704702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:45.705380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:45.705683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:45.707311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:45.772314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a19c73-2858-458b-8ba3-9ce528b75908 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:45.790445 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1399/2827] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:45.794661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a19c73-2858-458b-8ba3-9ce528b75908 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:45.804312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:45.804989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:45.805486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:45.806398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:45.807073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:45.809605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:45.876746 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1429/2828] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:01:45 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:01:45.879079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:45.897154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792afa6c-3b09-4d1a-99a4-85b1fb7b0e58 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:46.039452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:46.040340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:46.041059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:46.041777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:46.042618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:46.048377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:46.117374 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1400/2829] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:01:45 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:46.122820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:46.139393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011b051-8900-41be-a71c-4a1b4625b237 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:46.153563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:46.154873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:46.155786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:46.156230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:46.157056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:46.161114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:46.195970 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1430/2830] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:46.200408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:46.214676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5ff0e-42b7-49ec-b212-bb894e96d191 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:46.232768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e617e-e601-48f0-8271-f61887cd5e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:46.233630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e617e-e601-48f0-8271-f61887cd5e69 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:46.234009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e617e-e601-48f0-8271-f61887cd5e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:46.234412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e617e-e601-48f0-8271-f61887cd5e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:46.235328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e617e-e601-48f0-8271-f61887cd5e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:46.240336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e617e-e601-48f0-8271-f61887cd5e69 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:46.268713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1401/2831] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:46.273631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:46.294903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edbfc1-0056-42ae-bdfe-55a85b2cc168 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:46.302171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:46.303424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:46.303963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:46.304561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] PATH_INFO: `/v3/projects/8b7d2980f191475181a1c8a2f4f353ef/users/a121629fa5e3450f8b7774f04fc4b96a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:46.337586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7d2980f191475181a1c8a2f4f353ef, user_id=a121629fa5e3450f8b7774f04fc4b96a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:46.340640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:46.384957 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1431/2832] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:46 2022] PUT /identity/v3/projects/8b7d2980f191475181a1c8a2f4f353ef/users/a121629fa5e3450f8b7774f04fc4b96a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:46.385921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c2e4f-e37a-4671-9b62-40fa17dd3fb3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:46.386210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c2e4f-e37a-4671-9b62-40fa17dd3fb3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:46.386483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c2e4f-e37a-4671-9b62-40fa17dd3fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:46.474215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-493c2e4f-e37a-4671-9b62-40fa17dd3fb3 None None] MFA Rules not processed for user `a121629fa5e3450f8b7774f04fc4b96a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:46.478355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-493c2e4f-e37a-4671-9b62-40fa17dd3fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:46.630317 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1402/2833] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:46.636194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:46.891752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe62587-13b2-4164-9eae-d5112186155e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:46.899279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:46.901129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:46.901428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:46.901701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:46.902900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:46.905093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:46.925556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:46.979178 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1432/2834] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:01:46 2022] POST /identity/v3/projects => generated 373 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:46.982513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c2e4f-e37a-4671-9b62-40fa17dd3fb3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:47.001950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c2e4f-e37a-4671-9b62-40fa17dd3fb3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:47.097741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:47.220305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:47.221034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:47.221408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:47.221680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:47.222393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:47.224508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:47.292538 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1403/2835] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:01:46 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:47.300002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:47.323946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df4fe3-23a0-4975-9ed3-eba21dcbe814 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:47.337111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:47.337676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:47.337950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:47.338348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:47.343114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:47.345514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:47.366957 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1433/2836] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:47.370744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:47.391697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435bba-96e6-45a9-921d-c7aa95d1216d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:47.420428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:47.422116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:47.423293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:47.423819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] PATH_INFO: `/v3/projects/cfa5a4ed25004e1aa167a140aa2f2324/users/2f9f0464f2d04778a6f42bea6226e811/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:47.472147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa5a4ed25004e1aa167a140aa2f2324, user_id=2f9f0464f2d04778a6f42bea6226e811, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:47.485698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:47.530124 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1404/2837] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:47 2022] PUT /identity/v3/projects/cfa5a4ed25004e1aa167a140aa2f2324/users/2f9f0464f2d04778a6f42bea6226e811/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:47.534117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:47.550991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e52ac-dd01-48ec-9e16-87fe6ab0ebcf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:47.693504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1430c-e902-4978-85c2-f8099dab7039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:47.694559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1430c-e902-4978-85c2-f8099dab7039 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:47.695077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1430c-e902-4978-85c2-f8099dab7039 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:47.696116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1430c-e902-4978-85c2-f8099dab7039 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:47.697394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1430c-e902-4978-85c2-f8099dab7039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:47.699433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1430c-e902-4978-85c2-f8099dab7039 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:47.726028 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1434/2838] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:01:47.731103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:47.757919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed3dbd-b6a5-4425-b5aa-fceb07b8a20c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:47.764877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:47.766402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:47.766850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:47.767295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] PATH_INFO: `/v3/projects/cfa5a4ed25004e1aa167a140aa2f2324/users/2f9f0464f2d04778a6f42bea6226e811/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:47.769773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa5a4ed25004e1aa167a140aa2f2324, user_id=2f9f0464f2d04778a6f42bea6226e811, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:47.774144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:47.790774 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1405/2839] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:01:47 2022] PUT /identity/v3/projects/cfa5a4ed25004e1aa167a140aa2f2324/users/2f9f0464f2d04778a6f42bea6226e811/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:47.803699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3bdea-d17d-4978-a36d-abcee849705f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:47.804313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3bdea-d17d-4978-a36d-abcee849705f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:47.804313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3bdea-d17d-4978-a36d-abcee849705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:47.880115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bbf3bdea-d17d-4978-a36d-abcee849705f None None] MFA Rules not processed for user `2f9f0464f2d04778a6f42bea6226e811`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:47.882750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bbf3bdea-d17d-4978-a36d-abcee849705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:48.119697 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1435/2840] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:48.126781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:48.146568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366f1f6-abbe-4929-be26-e4211b818f30 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:48.154295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:48.155073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:48.155445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:48.157033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:48.158107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:48.158519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:48.161304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:48.233542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1406/2841] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:48.237913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3bdea-d17d-4978-a36d-abcee849705f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:48.263510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3bdea-d17d-4978-a36d-abcee849705f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:48.277412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:48.278397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:48.278787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:48.279313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:48.280083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:48.282779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:48.390704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1436/2842] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:01:48 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:48.392425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:48.423251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be331c-ada4-4815-8f7f-74d25ed80104 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:48.626233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059342e-a2b1-43b8-9bb3-b5d3c38d590b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:48.626877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059342e-a2b1-43b8-9bb3-b5d3c38d590b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:48.627155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059342e-a2b1-43b8-9bb3-b5d3c38d590b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:48.627426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059342e-a2b1-43b8-9bb3-b5d3c38d590b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:48.628082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059342e-a2b1-43b8-9bb3-b5d3c38d590b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:48.628384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059342e-a2b1-43b8-9bb3-b5d3c38d590b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:48.634183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059342e-a2b1-43b8-9bb3-b5d3c38d590b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:48.654028 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1407/2843] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:48.668826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:48.671196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:48.810900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:48.863750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a8a03-2696-4b45-ac9a-d186e4f916f5 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:48.875452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:48.876029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:48.876305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:48.876574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:48.889423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:48.894008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:48.931587 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1437/2844] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:48.949764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442861e9-3328-42c2-95c3-f68079477b15 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:48.950062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442861e9-3328-42c2-95c3-f68079477b15 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:48.950314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442861e9-3328-42c2-95c3-f68079477b15 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:48.951729 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1408/2845] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 10:01:48.965602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:01:48.983977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:49.000615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64069364-4ad0-4cf3-ae29-adcc947576ec service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:49.012021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:49.012718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:49.013720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:49.014003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/projects/e558ccda20e64593971db841548cc33d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:49.017445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:get_project(project_id=e558ccda20e64593971db841548cc33d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:49.019023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:49.022986 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1438/2846] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:01:48 2022] GET /identity/v3/projects/e558ccda20e64593971db841548cc33d => generated 367 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:50.155386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bc85-f730-4663-9882-f7737f4d216a None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:50.156038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bc85-f730-4663-9882-f7737f4d216a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:50.156038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bc85-f730-4663-9882-f7737f4d216a None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:50.157763 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1409/2847] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:01:50.169673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:01:50.190754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:50.215457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12310e3b-f879-4d95-b89b-e993eef4e8a7 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:50.220941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:50.221709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:50.222084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:50.225068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/projects/e558ccda20e64593971db841548cc33d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:50.227070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:get_project(project_id=e558ccda20e64593971db841548cc33d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:50.237076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:50.244149 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1439/2848] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:01:50 2022] GET /identity/v3/projects/e558ccda20e64593971db841548cc33d => generated 367 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:50.317718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8bc85-f730-4663-9882-f7737f4d216a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:50.346329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8bc85-f730-4663-9882-f7737f4d216a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:50.356739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:50.358372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:50.359085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:50.360996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:50.362474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:50.367111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:50.371426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:50.432836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1410/2849] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:50.437084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:50.451700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb261ed8-c242-4522-82e4-1e4aeeec2a2e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:50.465474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:50.466525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:50.467024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:50.467504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:50.468386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:50.472878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:50.523309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1440/2850] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:01:50 2022] POST /identity/v3/users => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:01:50.527939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:50.549025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e360-42a6-4075-8e04-a2b3948bbf3e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:50.579265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d2eca-faf6-44d0-a50f-3772fa7e60d1 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:50.580476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2eca-faf6-44d0-a50f-3772fa7e60d1 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:50.580997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2eca-faf6-44d0-a50f-3772fa7e60d1 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:50.581491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2eca-faf6-44d0-a50f-3772fa7e60d1 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:50.582378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d2eca-faf6-44d0-a50f-3772fa7e60d1 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:50.582931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d2eca-faf6-44d0-a50f-3772fa7e60d1 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:50.585482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d2eca-faf6-44d0-a50f-3772fa7e60d1 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:50.622857 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1411/2851] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:50.706535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:50.708313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:50.746772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c88a1b-7141-4356-872c-896de603a96b tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:50.781115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:50.781583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:50.781869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:50.782124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:50.783924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:50.810162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:50.811834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:50.832526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1441/2852] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:51.115701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f790f-a4c2-4781-92be-5b9370e32969 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:51.115982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f790f-a4c2-4781-92be-5b9370e32969 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:51.116226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f790f-a4c2-4781-92be-5b9370e32969 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:51.117862 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1412/2853] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:01:51.126103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:01:51.155539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:51.182943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a03220-eb0f-4560-9602-c8c81856413f service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:51.198629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:51.203360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:51.203360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:51.205741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/projects/e558ccda20e64593971db841548cc33d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:51.212929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:get_project(project_id=e558ccda20e64593971db841548cc33d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:51.215594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:51.225330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1442/2854] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:01:51 2022] GET /identity/v3/projects/e558ccda20e64593971db841548cc33d => generated 367 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:51.647689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ca05d-f73c-4744-bc42-845bdd19dc58 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:51.650014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ca05d-f73c-4744-bc42-845bdd19dc58 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:51.650504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ca05d-f73c-4744-bc42-845bdd19dc58 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:51.652901 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1413/2855] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:01:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:01:51.659277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:01:51.684633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:51.723276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b61e9-1ac6-4a84-a54a-cc283c2137d2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:51.735339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:51.737455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:51.737929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:51.740291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/projects/e558ccda20e64593971db841548cc33d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:51.743143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:get_project(project_id=e558ccda20e64593971db841548cc33d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:51.749743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:51.767515 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1443/2856] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:01:51 2022] GET /identity/v3/projects/e558ccda20e64593971db841548cc33d => generated 367 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:01:51.817647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ca05d-f73c-4744-bc42-845bdd19dc58 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:51.848205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ca05d-f73c-4744-bc42-845bdd19dc58 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:51.869457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:51.871000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:51.874819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:51.878281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/users/6aeeba5aa2744622abfbe9e2e234f580` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:51.924768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6aeeba5aa2744622abfbe9e2e234f580)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:51.930096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:52.028652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:52.051590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aeeba5aa2744622abfbe9e2e234f580'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.052300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aeeba5aa2744622abfbe9e2e234f580'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.076814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aeeba5aa2744622abfbe9e2e234f580'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.103725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aeeba5aa2744622abfbe9e2e234f580'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.172740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aeeba5aa2744622abfbe9e2e234f580'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.191790 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1414/2857] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:51 2022] DELETE /identity/v3/users/6aeeba5aa2744622abfbe9e2e234f580 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:52.193986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:52.196250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:52.230214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e076944-e29c-447f-add7-ab804ae618a3 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:52.406172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f9f0464f2d04778a6f42bea6226e811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfa5a4ed25004e1aa167a140aa2f2324', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:52.408278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:52.409741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:52.410287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] PATH_INFO: `/v3/projects/e558ccda20e64593971db841548cc33d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:52.413767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e558ccda20e64593971db841548cc33d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:52.418744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:52.565179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e558ccda20e64593971db841548cc33d', 'request_id': 'req-eb083f35-c596-4868-b980-ee7907627e19'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.581996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e558ccda20e64593971db841548cc33d', 'request_id': 'req-eb083f35-c596-4868-b980-ee7907627e19'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.589349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:52.593946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e558ccda20e64593971db841548cc33d', 'request_id': 'req-eb083f35-c596-4868-b980-ee7907627e19'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.639090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] The token cache is being invalidate because project e558ccda20e64593971db841548cc33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:52.641990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.647482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb083f35-c596-4868-b980-ee7907627e19 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:52.655844 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1444/2858] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:52 2022] DELETE /identity/v3/projects/e558ccda20e64593971db841548cc33d => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:52.656530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:52.658064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:52.887321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f406a3-29b4-48e4-a358-3fcaf6889eff tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:53.054263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:53.054949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:53.055231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:53.055505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] PATH_INFO: `/v3/users/a121629fa5e3450f8b7774f04fc4b96a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:53.056864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a121629fa5e3450f8b7774f04fc4b96a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:53.059282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:53.170351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a121629fa5e3450f8b7774f04fc4b96a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:53.171090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a121629fa5e3450f8b7774f04fc4b96a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:53.190349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a121629fa5e3450f8b7774f04fc4b96a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:53.212331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a121629fa5e3450f8b7774f04fc4b96a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:53.215139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:53.241484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a121629fa5e3450f8b7774f04fc4b96a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:53.259961 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1415/2859] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:52 2022] DELETE /identity/v3/users/a121629fa5e3450f8b7774f04fc4b96a => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:01:53.262461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d89fe9-b773-4d1f-aedf-d05626b067d7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:53.263332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d89fe9-b773-4d1f-aedf-d05626b067d7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:53.263816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d89fe9-b773-4d1f-aedf-d05626b067d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:53.308076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-92d89fe9-b773-4d1f-aedf-d05626b067d7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:53.314051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-92d89fe9-b773-4d1f-aedf-d05626b067d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:53.492546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1445/2860] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:53.536095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:53.537891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:53.729441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39e73d-40f7-4881-b942-5e7e87b08387 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:53.753903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:53.754804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:53.755915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:53.756911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:53.804553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:53.809711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:53.854428 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1416/2861] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:55.084285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:55.085744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:55.086271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:55.151074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] MFA Rules not processed for user `6d6bc44a06384fa4bc0aba86ea50e52c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:55.152773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:55.334535 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1446/2862] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:55.346366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:55.348579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:55.599874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75a9bd-560d-4a9b-adee-a8bc11f72e66 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:55.616283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2c408-a6b8-4ca0-80dc-40152e0d8e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:55.619784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c408-a6b8-4ca0-80dc-40152e0d8e70 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:55.620273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c408-a6b8-4ca0-80dc-40152e0d8e70 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:55.620775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c408-a6b8-4ca0-80dc-40152e0d8e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:55.669642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2c408-a6b8-4ca0-80dc-40152e0d8e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:55.672111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2c408-a6b8-4ca0-80dc-40152e0d8e70 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:55.689875 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1417/2863] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:55.772599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:55.772599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:55.775081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298286f-58de-4ec5-889d-6a05e7839781 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:55.775390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298286f-58de-4ec5-889d-6a05e7839781 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:55.775628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298286f-58de-4ec5-889d-6a05e7839781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:55.802255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26670bc-2dfb-42bd-8924-865aa1811b77 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:55.818890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:55.820084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:55.820435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:55.820811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] PATH_INFO: `/v3/projects/8b7d2980f191475181a1c8a2f4f353ef` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:55.822215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7d2980f191475181a1c8a2f4f353ef)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:55.824360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:55.914163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5298286f-58de-4ec5-889d-6a05e7839781 None None] MFA Rules not processed for user `b31b5932bb8e4edcadc6954c46c2c386`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:55.915635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5298286f-58de-4ec5-889d-6a05e7839781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:55.964534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7d2980f191475181a1c8a2f4f353ef', 'request_id': 'req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:55.982656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7d2980f191475181a1c8a2f4f353ef', 'request_id': 'req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.001564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7d2980f191475181a1c8a2f4f353ef', 'request_id': 'req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.047296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] The token cache is being invalidate because project 8b7d2980f191475181a1c8a2f4f353ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:56.048117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.049812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.054541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1447/2864] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:55 2022] DELETE /identity/v3/projects/8b7d2980f191475181a1c8a2f4f353ef => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:56.061792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:56.069309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:56.091956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:56.115326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:56.138704 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1418/2865] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:56.149726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:56.150276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:56.150720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:56.185308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] MFA Rules not processed for user `b31b5932bb8e4edcadc6954c46c2c386`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:56.188590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:56.320554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4a3c-7ffa-46f9-bd3e-177aaa916f0b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:56.362958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:56.363638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:56.363920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:56.364199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] PATH_INFO: `/v3/users/2f9f0464f2d04778a6f42bea6226e811` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:56.365477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9f0464f2d04778a6f42bea6226e811)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:56.368585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:56.431269 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1419/2866] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:01:56.449399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:56.451179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:56.549342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9f0464f2d04778a6f42bea6226e811'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.564681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9f0464f2d04778a6f42bea6226e811'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.628221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9f0464f2d04778a6f42bea6226e811'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.657349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9f0464f2d04778a6f42bea6226e811'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.658751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae9edb9e-f9c8-4735-8402-7fcb2e97dd1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9f0464f2d04778a6f42bea6226e811'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:56.687434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be48f0b-20d9-4dd6-b6ae-77453f93c100 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:56.713201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:56.713870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:56.714164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:56.714445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:56.721159 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1448/2867] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:01:56 2022] DELETE /identity/v3/users/2f9f0464f2d04778a6f42bea6226e811 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:56.724776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:56.774401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:56.783924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:56.838072 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1420/2868] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:01:57.774704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:57.775363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:57.775562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:57.821751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] MFA Rules not processed for user `dce01d3212934161a38be853b4cd93ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:57.824361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:58.011887 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1449/2869] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:58.804835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:58.808532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:58.975110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f7f0c-001b-44b0-a0a7-c7b4d89c3cd0 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:59.001299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:59.003459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:59.098980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:59.145740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:59.167078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dcdfc-542b-4276-836f-87c4befb2de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:59.169957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dcdfc-542b-4276-836f-87c4befb2de5 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:59.171347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dcdfc-542b-4276-836f-87c4befb2de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:59.171729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dcdfc-542b-4276-836f-87c4befb2de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:59.176170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-015dcdfc-542b-4276-836f-87c4befb2de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:59.201571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfc13-6c49-420d-8935-c94e0885064a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:59.202832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015dcdfc-542b-4276-836f-87c4befb2de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:59.204443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015dcdfc-542b-4276-836f-87c4befb2de5 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:59.217027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:59.217027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:59.217027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:59.217027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] PATH_INFO: `/v3/projects/cfa5a4ed25004e1aa167a140aa2f2324` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:59.225176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa5a4ed25004e1aa167a140aa2f2324)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:59.231911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:01:59.245393 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1421/2870] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:01:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:59.359380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa5a4ed25004e1aa167a140aa2f2324', 'request_id': 'req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:59.394041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa5a4ed25004e1aa167a140aa2f2324', 'request_id': 'req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:59.410438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa5a4ed25004e1aa167a140aa2f2324', 'request_id': 'req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:59.454550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] The token cache is being invalidate because project cfa5a4ed25004e1aa167a140aa2f2324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:01:59.455099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:59.457413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a8a14033-47a1-4e7e-a5a6-9cc44418ae14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:01:59.465730 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1450/2871] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:59 2022] DELETE /identity/v3/projects/cfa5a4ed25004e1aa167a140aa2f2324 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:01:59.473130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2f8ff-7d13-45bf-a0d4-c93c492b17a7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:59.473673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2f8ff-7d13-45bf-a0d4-c93c492b17a7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:59.474108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2f8ff-7d13-45bf-a0d4-c93c492b17a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:59.528611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-04b2f8ff-7d13-45bf-a0d4-c93c492b17a7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:59.532809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04b2f8ff-7d13-45bf-a0d4-c93c492b17a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:59.751802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1422/2872] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:59.759969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ecc2a-0dad-43cf-9f72-99025fc74795 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:59.760247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ecc2a-0dad-43cf-9f72-99025fc74795 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:59.760520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ecc2a-0dad-43cf-9f72-99025fc74795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:59.823530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0a3ecc2a-0dad-43cf-9f72-99025fc74795 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:01:59.827612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0a3ecc2a-0dad-43cf-9f72-99025fc74795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:01:59.869823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2f8ff-7d13-45bf-a0d4-c93c492b17a7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:01:59.879936 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1451/2873] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:01:59.883093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2f8ff-7d13-45bf-a0d4-c93c492b17a7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:01:59.907078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:01:59.907726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:01:59.908005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:01:59.908280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:01:59.908977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:01:59.918619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:01:59.920291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:00.003438 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1423/2874] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:02:00.007838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ecc2a-0dad-43cf-9f72-99025fc74795 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:00.018762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ecc2a-0dad-43cf-9f72-99025fc74795 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:00.065241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:00.065905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:00.066178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:00.066444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:00.067095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:00.097413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:00.237722 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1452/2875] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:00 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:00.242241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:00.262909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037f6e1-9d3e-4d6e-b156-729a81eeb8bd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:00.503985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:00.505693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:00.510932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:00.511603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:00.512530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:00.529884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:00.591642 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1424/2876] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:00 2022] POST /identity/v3/users => generated 290 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:00.596817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:00.618391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735936b2-ca2c-4fa8-805c-3cad2ded5ade admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:00.646189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:00.646828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:00.647109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:00.647382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:00.648075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:00.649727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:00.710354 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1453/2877] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:00.719260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:00.742390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511fbc4-869e-4290-8a06-21dced02f55d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:00.752145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562e9f-7ded-47a3-b6aa-954be0c27a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:00.753386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562e9f-7ded-47a3-b6aa-954be0c27a38 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:00.753857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562e9f-7ded-47a3-b6aa-954be0c27a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:00.754368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562e9f-7ded-47a3-b6aa-954be0c27a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:00.755212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562e9f-7ded-47a3-b6aa-954be0c27a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:00.757200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562e9f-7ded-47a3-b6aa-954be0c27a38 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:00.783469 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1425/2878] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:00.787660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:00.820077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77374f13-ed73-43c5-bea4-cf488926ac63 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:00.834891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:00.836192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:00.836397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:00.836959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] PATH_INFO: `/v3/projects/ae126572a1384be3b08a8af5c3c18f74/users/c37aa9fdaad649fc8f1c60eacdc0f841/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:00.897946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae126572a1384be3b08a8af5c3c18f74, user_id=c37aa9fdaad649fc8f1c60eacdc0f841, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:00.907332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:00.974181 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1454/2879] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:00 2022] PUT /identity/v3/projects/ae126572a1384be3b08a8af5c3c18f74/users/c37aa9fdaad649fc8f1c60eacdc0f841/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:02:00.974895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ad72-d2d0-4263-8eff-5e5c2df69e15 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:00.975358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ad72-d2d0-4263-8eff-5e5c2df69e15 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:00.975486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ad72-d2d0-4263-8eff-5e5c2df69e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:01.063245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f166ad72-d2d0-4263-8eff-5e5c2df69e15 None None] MFA Rules not processed for user `c37aa9fdaad649fc8f1c60eacdc0f841`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:01.071073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f166ad72-d2d0-4263-8eff-5e5c2df69e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:01.229756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:01.289252 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1426/2880] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:01.293851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:01.494616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:01.579513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5a87b-9bd5-44cd-ac2a-e010a22cdf05 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:01.630332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:01.633659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:01.636288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:01.641956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:01.643042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:01.647220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:01.784667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166ad72-d2d0-4263-8eff-5e5c2df69e15 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:01.784918 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1455/2881] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:01 2022] POST /identity/v3/projects => generated 379 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:01.838314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166ad72-d2d0-4263-8eff-5e5c2df69e15 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:02.049410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:02.053422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:02.054871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:02.058146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:02.063394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:02.073589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:02.147418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:02.167861 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1427/2882] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:01 2022] POST /identity/v3/users => generated 296 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:02.189019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9b19-be73-46d0-b0c7-c3c0906c0a81 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:02.203986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:02.204990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:02.205374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:02.205732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:02.206507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:02.210808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:02.244176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1456/2883] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:02:02.249769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:02.295778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851631b7-1427-4fa4-b9c3-35d31aed0884 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:02.301694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:02.305352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:02.339329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:02.343464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:02.344547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:02.347055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] PATH_INFO: `/v3/projects/874e05177b424dffa1a328f2752b4a59/users/c87781c440114a578b67c9e60ca6f8b6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:02.439373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874e05177b424dffa1a328f2752b4a59, user_id=c87781c440114a578b67c9e60ca6f8b6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:02.448857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:02.503933 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1428/2884] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:02 2022] PUT /identity/v3/projects/874e05177b424dffa1a328f2752b4a59/users/c87781c440114a578b67c9e60ca6f8b6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:02.512798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:02.525981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d76c2-8e5d-4824-8202-6246825d8bd9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:02.697210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09928aa9-3395-441b-bb84-d650386f0193 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:02.714750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:02.715758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:02.715758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:02.716077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:02.716626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:02.719162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862313bc-2f7e-4111-a012-2ada7cd867b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:02.719344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:02.719466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862313bc-2f7e-4111-a012-2ada7cd867b2 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:02.719889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862313bc-2f7e-4111-a012-2ada7cd867b2 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:02.720254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862313bc-2f7e-4111-a012-2ada7cd867b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:02.722985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-862313bc-2f7e-4111-a012-2ada7cd867b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:02.753285 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1429/2885] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:02.754386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:02.756817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862313bc-2f7e-4111-a012-2ada7cd867b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:02.759428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862313bc-2f7e-4111-a012-2ada7cd867b2 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:02.781774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dc552-869a-42af-ac9d-25634a62d09e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:02.789512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1457/2886] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:02.805342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:02.806104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:02.806387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:02.806662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] PATH_INFO: `/v3/projects/874e05177b424dffa1a328f2752b4a59/users/c87781c440114a578b67c9e60ca6f8b6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:02.812429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874e05177b424dffa1a328f2752b4a59, user_id=c87781c440114a578b67c9e60ca6f8b6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:02.817215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:02.847022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1430/2887] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:02 2022] PUT /identity/v3/projects/874e05177b424dffa1a328f2752b4a59/users/c87781c440114a578b67c9e60ca6f8b6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:02.862540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876d5b-4cd1-42d7-aae9-82d5c5b9f73e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:02.862815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876d5b-4cd1-42d7-aae9-82d5c5b9f73e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:02.863091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876d5b-4cd1-42d7-aae9-82d5c5b9f73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:02.966521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-94876d5b-4cd1-42d7-aae9-82d5c5b9f73e None None] MFA Rules not processed for user `c87781c440114a578b67c9e60ca6f8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:02.968014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-94876d5b-4cd1-42d7-aae9-82d5c5b9f73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:03.095628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:03.211116 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1458/2888] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:02:03.713540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:03.831372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:03.961772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cf28a-6693-4d2c-812f-4d1d3c923670 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:03.971883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bccfc-3719-4b01-be47-602448498adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:03.973691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bccfc-3719-4b01-be47-602448498adc service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:03.974285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bccfc-3719-4b01-be47-602448498adc service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:03.974826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bccfc-3719-4b01-be47-602448498adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:04.021173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bccfc-3719-4b01-be47-602448498adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:04.032399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bccfc-3719-4b01-be47-602448498adc service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:04.117760 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1431/2889] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:04.419945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876d5b-4cd1-42d7-aae9-82d5c5b9f73e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:04.574497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876d5b-4cd1-42d7-aae9-82d5c5b9f73e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:04.581734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:04.582420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:04.582698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:04.582966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] PATH_INFO: `/v3/users/c37aa9fdaad649fc8f1c60eacdc0f841` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:04.584278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37aa9fdaad649fc8f1c60eacdc0f841)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:04.587489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:04.735875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37aa9fdaad649fc8f1c60eacdc0f841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:04.737037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37aa9fdaad649fc8f1c60eacdc0f841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:04.764924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37aa9fdaad649fc8f1c60eacdc0f841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:04.785515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37aa9fdaad649fc8f1c60eacdc0f841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:04.786704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37aa9fdaad649fc8f1c60eacdc0f841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:04.830176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1459/2890] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:02:04 2022] DELETE /identity/v3/users/c37aa9fdaad649fc8f1c60eacdc0f841 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:04.832189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9866e-a274-470c-9378-09fc3e743b4c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:04.832682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9866e-a274-470c-9378-09fc3e743b4c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:04.832794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9866e-a274-470c-9378-09fc3e743b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:04.870576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-63f9866e-a274-470c-9378-09fc3e743b4c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:04.871836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-63f9866e-a274-470c-9378-09fc3e743b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:05.080917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1432/2891] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:05.137477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:05.139384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:05.403181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198675b-a47f-4f3b-95b9-1048731e386e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:05.420715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:05.421419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:05.421638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:05.422273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:05.443001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:05.444567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:05.456230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:05.481049 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1460/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:06.118872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9866e-a274-470c-9378-09fc3e743b4c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:06.137265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9866e-a274-470c-9378-09fc3e743b4c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:06.158294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:06.159192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:06.159467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:06.161904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:06.169117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:06.237909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:06.239802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:06.256220 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1433/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:07.336148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:07.338549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:07.386672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe3421-bfa8-4be6-a9a4-43f8382d28f6 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:07.397643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:07.398275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:07.398741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:07.399220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] PATH_INFO: `/v3/projects/ae126572a1384be3b08a8af5c3c18f74` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:07.402254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae126572a1384be3b08a8af5c3c18f74)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:07.405527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:07.567679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae126572a1384be3b08a8af5c3c18f74', 'request_id': 'req-b78a329f-dbc8-40ef-8b26-d99829c44573'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:07.609600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae126572a1384be3b08a8af5c3c18f74', 'request_id': 'req-b78a329f-dbc8-40ef-8b26-d99829c44573'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:07.623655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae126572a1384be3b08a8af5c3c18f74', 'request_id': 'req-b78a329f-dbc8-40ef-8b26-d99829c44573'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:07.680809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:07.682144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] The token cache is being invalidate because project ae126572a1384be3b08a8af5c3c18f74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:02:07.683231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:07.688503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:07.689762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:07.694583 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1461/2894] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:07 2022] DELETE /identity/v3/projects/ae126572a1384be3b08a8af5c3c18f74 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:07.700034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:07.702060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:07.804084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bb02b-fea0-4d0b-b66b-ae4fdcbf1358 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:07.861783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:07.885551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67ac8-7986-465a-8ba4-de030a275c53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:07.886361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67ac8-7986-465a-8ba4-de030a275c53 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:07.886831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67ac8-7986-465a-8ba4-de030a275c53 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:07.887255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67ac8-7986-465a-8ba4-de030a275c53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:07.889432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-40c67ac8-7986-465a-8ba4-de030a275c53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:07.909143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67ac8-7986-465a-8ba4-de030a275c53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:07.911038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67ac8-7986-465a-8ba4-de030a275c53 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:07.927626 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1434/2895] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:02:07.953064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a329f-dbc8-40ef-8b26-d99829c44573 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:07.965588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:07.966072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:07.966403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:07.966837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] PATH_INFO: `/v3/users/c87781c440114a578b67c9e60ca6f8b6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:07.968996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87781c440114a578b67c9e60ca6f8b6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:07.975531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:08.131073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87781c440114a578b67c9e60ca6f8b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:08.132405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87781c440114a578b67c9e60ca6f8b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:08.154770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87781c440114a578b67c9e60ca6f8b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:08.166864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87781c440114a578b67c9e60ca6f8b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:08.167744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87781c440114a578b67c9e60ca6f8b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:08.189848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:08.196362 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1462/2896] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:02:07 2022] DELETE /identity/v3/users/c87781c440114a578b67c9e60ca6f8b6 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:02:08.471874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:08.472388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:08.472814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:08.504561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:08.506457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:08.641889 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1435/2897] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:08.655456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:08.657221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:08.805511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064eeb-49fd-4c53-bd97-92afff85984a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:08.812246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:08.812911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:08.813417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:08.813686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:08.824521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:08.826330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:08.839802 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1463/2898] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:02:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:02:09.496402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:09.498515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:09.682542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:09.685291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:09.724243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315c422-7fee-47c8-8936-855efd02bffd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:09.733268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:09.733935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:09.734208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:09.734475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] PATH_INFO: `/v3/users/6d6bc44a06384fa4bc0aba86ea50e52c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:09.738048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6bc44a06384fa4bc0aba86ea50e52c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:09.741693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:09.846359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6bc44a06384fa4bc0aba86ea50e52c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:09.848128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6bc44a06384fa4bc0aba86ea50e52c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:09.866446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6bc44a06384fa4bc0aba86ea50e52c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:09.872371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74123135-0e4c-4ef8-bc7e-74eb6f1b2892 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:09.889682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:09.893509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:09.894102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6bc44a06384fa4bc0aba86ea50e52c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:09.897141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:09.897815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] PATH_INFO: `/v3/projects/874e05177b424dffa1a328f2752b4a59` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:09.899518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=874e05177b424dffa1a328f2752b4a59)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:09.902568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:09.925297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6bc44a06384fa4bc0aba86ea50e52c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:09.932091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:09.944354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1436/2899] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:02:09 2022] DELETE /identity/v3/users/6d6bc44a06384fa4bc0aba86ea50e52c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:09.945731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:09.952797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:09.977298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874e05177b424dffa1a328f2752b4a59', 'request_id': 'req-371026ae-699d-4b46-8362-73f18b31e9bd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:09.989394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874e05177b424dffa1a328f2752b4a59', 'request_id': 'req-371026ae-699d-4b46-8362-73f18b31e9bd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:10.000041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874e05177b424dffa1a328f2752b4a59', 'request_id': 'req-371026ae-699d-4b46-8362-73f18b31e9bd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:10.022115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] The token cache is being invalidate because project 874e05177b424dffa1a328f2752b4a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:02:10.025440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:10.031298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-371026ae-699d-4b46-8362-73f18b31e9bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:10.034273 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1464/2900] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:09 2022] DELETE /identity/v3/projects/874e05177b424dffa1a328f2752b4a59 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:10.040452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2496af6-d735-47b8-92d9-86d313335098 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.041128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2496af6-d735-47b8-92d9-86d313335098 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.042160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2496af6-d735-47b8-92d9-86d313335098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.097739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c2496af6-d735-47b8-92d9-86d313335098 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:10.099406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c2496af6-d735-47b8-92d9-86d313335098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:10.119947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae570d13-0b53-496d-abfd-b16a17197bb5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.138174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:10.212244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c770e-6e2c-4bf4-822e-5ed369aff033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.212927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c770e-6e2c-4bf4-822e-5ed369aff033 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.213208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c770e-6e2c-4bf4-822e-5ed369aff033 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.213482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c770e-6e2c-4bf4-822e-5ed369aff033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.215486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-505c770e-6e2c-4bf4-822e-5ed369aff033 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:10.230259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c770e-6e2c-4bf4-822e-5ed369aff033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.233341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c770e-6e2c-4bf4-822e-5ed369aff033 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.245816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1465/2901] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:10.249409 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1437/2902] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:10.253976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f02c7-efb5-4d93-8d6f-4dcf663c5b80 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.254335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f02c7-efb5-4d93-8d6f-4dcf663c5b80 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.254547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f02c7-efb5-4d93-8d6f-4dcf663c5b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.265115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28e5bc-b547-44fa-ba6e-dab088fb966a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.265527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28e5bc-b547-44fa-ba6e-dab088fb966a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.265762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28e5bc-b547-44fa-ba6e-dab088fb966a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.296912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e77f02c7-efb5-4d93-8d6f-4dcf663c5b80 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:10.298425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e77f02c7-efb5-4d93-8d6f-4dcf663c5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:10.311048 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1438/2903] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:10.316181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f02c7-efb5-4d93-8d6f-4dcf663c5b80 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.317172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4c28e5bc-b547-44fa-ba6e-dab088fb966a None None] MFA Rules not processed for user `6eb3ccfcd9c344e1a7ad55f8398ec87a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:10.318416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c28e5bc-b547-44fa-ba6e-dab088fb966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:10.326987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f02c7-efb5-4d93-8d6f-4dcf663c5b80 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.331948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.332695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.333056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.333379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.334125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:02:10.334491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.336697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.353724 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1439/2904] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:02:10.356032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.372860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b149b6a-deee-4cf9-85db-457d43555474 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.379510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.380438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.380909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.381353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.382285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.385976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.414898 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1466/2905] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:10.429054 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1440/2906] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:10 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:10.433315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28e5bc-b547-44fa-ba6e-dab088fb966a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.450049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28e5bc-b547-44fa-ba6e-dab088fb966a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.533644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.534417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.534696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.535002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.535662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.538127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.576889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1467/2907] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:10 2022] POST /identity/v3/users => generated 302 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:02:10.579105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.592652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e584e9-eff8-4120-a26c-10ac45f6d1b5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.610192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.612012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.612618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.615399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.616686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.618405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.630555 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1441/2908] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:10.635862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.648116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de87565-5020-4350-af61-298bd62f7b0e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.653575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c174b3-386d-41da-8743-d158543fa6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.654171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c174b3-386d-41da-8743-d158543fa6fb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.654630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c174b3-386d-41da-8743-d158543fa6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.654872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c174b3-386d-41da-8743-d158543fa6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.655554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c174b3-386d-41da-8743-d158543fa6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.657222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c174b3-386d-41da-8743-d158543fa6fb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.668773 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1468/2909] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:10.673143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.684375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c854-5fdc-4ba4-8b92-f45fd05f4e33 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.689792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.690605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.690935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.691443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] PATH_INFO: `/v3/projects/9f3336f27cfd4866b6e527676bb6acb0/users/7d51e5e0cb8f4e8e84ce6e63fe212acc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.709936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3336f27cfd4866b6e527676bb6acb0, user_id=7d51e5e0cb8f4e8e84ce6e63fe212acc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.713224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.737259 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1442/2910] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:10 2022] PUT /identity/v3/projects/9f3336f27cfd4866b6e527676bb6acb0/users/7d51e5e0cb8f4e8e84ce6e63fe212acc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:10.751382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f36113-ce57-4678-8924-a67229d4ccb3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.751939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f36113-ce57-4678-8924-a67229d4ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.752861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f36113-ce57-4678-8924-a67229d4ccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.777850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.780087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:10.794930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:10.829955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d7f36113-ce57-4678-8924-a67229d4ccb3 None None] MFA Rules not processed for user `7d51e5e0cb8f4e8e84ce6e63fe212acc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:10.831650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d7f36113-ce57-4678-8924-a67229d4ccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:10.881401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:10.888582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cecf9-99c8-46d6-b32b-852fba28e7e5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.898332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:10.900166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:10.900531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:10.900881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] PATH_INFO: `/v3/projects/4d37d16753ac4c68aa1f1fac7925d857` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:10.903288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d37d16753ac4c68aa1f1fac7925d857)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:10.905792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:10.955196 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1469/2911] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:10.964517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f36113-ce57-4678-8924-a67229d4ccb3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:10.983383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d37d16753ac4c68aa1f1fac7925d857', 'request_id': 'req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:10.987085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f36113-ce57-4678-8924-a67229d4ccb3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:10.996801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d37d16753ac4c68aa1f1fac7925d857', 'request_id': 'req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:11.011559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d37d16753ac4c68aa1f1fac7925d857', 'request_id': 'req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:11.101766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] The token cache is being invalidate because project 4d37d16753ac4c68aa1f1fac7925d857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:02:11.106110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:11.109337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-574f49e7-9a0a-4aac-9cc2-6c76efd2efaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:11.113683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.114563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.114854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.115130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.116690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.117474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1443/2912] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:10 2022] DELETE /identity/v3/projects/4d37d16753ac4c68aa1f1fac7925d857 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:11.123090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.124830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ee062-222d-443f-ad7f-e73069ccb569 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.125238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ee062-222d-443f-ad7f-e73069ccb569 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.125623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ee062-222d-443f-ad7f-e73069ccb569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.175439 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1470/2913] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:10 2022] POST /identity/v3/projects => generated 401 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:11.176716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.182513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:11.221687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c39ee062-222d-443f-ad7f-e73069ccb569 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:11.228071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c39ee062-222d-443f-ad7f-e73069ccb569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:11.316470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c394f2-d807-433e-a85a-7fcc052ce88b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.321529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.322149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.322426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.322885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.323521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.325492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.360558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1471/2914] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:11 2022] POST /identity/v3/users => generated 307 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:11.363881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.387269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887a1a4-4ad9-46a3-b2bb-e207ffdd72f7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.401643 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1444/2915] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:11.403174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05559d3b-b11e-49e1-bd6f-7ed9fc70df4e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.403450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05559d3b-b11e-49e1-bd6f-7ed9fc70df4e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.403722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05559d3b-b11e-49e1-bd6f-7ed9fc70df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.411041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.411700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.412258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.412258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.412926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.415099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.432779 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1472/2916] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:11.433407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-05559d3b-b11e-49e1-bd6f-7ed9fc70df4e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:11.433567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.434424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-05559d3b-b11e-49e1-bd6f-7ed9fc70df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:11.446658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92601e73-0d5b-48c4-9bbe-a955cbdd7e5c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.459673 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1445/2917] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:11.459673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05559d3b-b11e-49e1-bd6f-7ed9fc70df4e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.467701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.467701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.467701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.468119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] PATH_INFO: `/v3/projects/e65d33280d6e4c3289089cd018e5bba5/users/9d6be144314e42d4bb25f70ef6ed3a43/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.477911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05559d3b-b11e-49e1-bd6f-7ed9fc70df4e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.483082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.483835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.484041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.484267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.485043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:02:11.485350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.487723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.510628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65d33280d6e4c3289089cd018e5bba5, user_id=9d6be144314e42d4bb25f70ef6ed3a43, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.513696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.515662 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1446/2918] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:02:11.521501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.539447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038890e6-c0ae-4b94-814a-fb7749f9a71b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.541158 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1473/2919] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:11 2022] PUT /identity/v3/projects/e65d33280d6e4c3289089cd018e5bba5/users/9d6be144314e42d4bb25f70ef6ed3a43/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:11.546114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.580183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:11.588879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:11.668845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.669916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.671072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.672489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.678168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.681233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.689659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bccebf-c989-41b0-9402-fc9ce417e463 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.699433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.700507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.701218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.701712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.703123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.706360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.729228 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1474/2920] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:11.733932 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1447/2921] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:11 2022] POST /identity/v3/projects => generated 393 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:11.734162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.734890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:11.737594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:11.863255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4af28-3875-47a1-b07f-3e44e275780d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.896609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a284dde-4fa9-4001-92bf-e523f2b3fc48 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:11.905505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:11.906102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:11.906373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:11.906704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] PATH_INFO: `/v3/projects/e65d33280d6e4c3289089cd018e5bba5/users/9d6be144314e42d4bb25f70ef6ed3a43/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:11.912338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65d33280d6e4c3289089cd018e5bba5, user_id=9d6be144314e42d4bb25f70ef6ed3a43, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:11.919410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:11.952693 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1448/2922] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:11 2022] PUT /identity/v3/projects/e65d33280d6e4c3289089cd018e5bba5/users/9d6be144314e42d4bb25f70ef6ed3a43/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:11.952693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:12.011385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc20e6c-b6f4-4e76-a1b0-670f113e29b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:12.012546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20e6c-b6f4-4e76-a1b0-670f113e29b0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.012924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20e6c-b6f4-4e76-a1b0-670f113e29b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.013197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20e6c-b6f4-4e76-a1b0-670f113e29b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.016254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2bc20e6c-b6f4-4e76-a1b0-670f113e29b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:12.035005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc20e6c-b6f4-4e76-a1b0-670f113e29b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:12.037125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc20e6c-b6f4-4e76-a1b0-670f113e29b0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:12.049189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07271fe8-0239-4ec8-9de5-a4294cea890b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:12.056905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:12.058267 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1475/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:12.059196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.059573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.060033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.060354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.060939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.061361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.061936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:12.064915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:12.099719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] MFA Rules not processed for user `9d6be144314e42d4bb25f70ef6ed3a43`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:12.102322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:12.139880 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1449/2924] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:11 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:12.143324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:12.201247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2625287-488c-4555-9f96-9d0891d2968a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:12.226684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:12.228243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.228920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.229959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.235982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:12.243854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:12.253974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:12.268038 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1476/2925] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:12.287988 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1450/2926] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:12.292575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:12.294471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:12.294662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:12.320728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ac5fc-af01-4ac4-8eba-a0f81fcdb47f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:12.331455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:12.332333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.332866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.334081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.338278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:12.344498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:12.375333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1451/2927] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:12.381492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:12.396592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:12.419588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aace7-3823-4f63-8c8e-49179c188f20 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:12.435132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:12.439037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.439568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.441201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] PATH_INFO: `/v3/projects/644bbf0f519f4cd6a02e998859ec00e2/users/36deeaadd647404e877c2b07ff9a4fbc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.486253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4845-3970-4d84-b854-a87d24692a32 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:12.515665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dff74-a118-4237-bfa9-c5398e4e1868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:12.516909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dff74-a118-4237-bfa9-c5398e4e1868 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.517377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dff74-a118-4237-bfa9-c5398e4e1868 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.517840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dff74-a118-4237-bfa9-c5398e4e1868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.532311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] RBAC: Authorizing `identity:create_grant(project_id=644bbf0f519f4cd6a02e998859ec00e2, user_id=36deeaadd647404e877c2b07ff9a4fbc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:12.538438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:12.541684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dff74-a118-4237-bfa9-c5398e4e1868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:12.544587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dff74-a118-4237-bfa9-c5398e4e1868 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:12.565725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1477/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:12 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:12.588973 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1452/2929] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:12 2022] PUT /identity/v3/projects/644bbf0f519f4cd6a02e998859ec00e2/users/36deeaadd647404e877c2b07ff9a4fbc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:02:12.702256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25959ce1-74f1-45c7-bd28-a65bfce3a4dd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:12.702731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25959ce1-74f1-45c7-bd28-a65bfce3a4dd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:12.703185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25959ce1-74f1-45c7-bd28-a65bfce3a4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:12.762112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-25959ce1-74f1-45c7-bd28-a65bfce3a4dd None None] MFA Rules not processed for user `36deeaadd647404e877c2b07ff9a4fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:12.766049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-25959ce1-74f1-45c7-bd28-a65bfce3a4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:13.064863 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1478/2930] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:13.290948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:13.295446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:13.518951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b8d0d-de10-4d84-9547-28843557d55b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:13.537416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:13.550238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:13.552287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:13.555990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:13.580911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:13.585872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:13.610362 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1453/2931] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:13.908649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25959ce1-74f1-45c7-bd28-a65bfce3a4dd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:13.930387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25959ce1-74f1-45c7-bd28-a65bfce3a4dd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:13.939086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c949da9-b5e0-4480-b4f3-02bba637b2b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:13.940326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c949da9-b5e0-4480-b4f3-02bba637b2b5 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:13.940326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c949da9-b5e0-4480-b4f3-02bba637b2b5 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:13.940742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c949da9-b5e0-4480-b4f3-02bba637b2b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:13.942942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c949da9-b5e0-4480-b4f3-02bba637b2b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:14.076523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c949da9-b5e0-4480-b4f3-02bba637b2b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:14.078888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c949da9-b5e0-4480-b4f3-02bba637b2b5 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:14.103405 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1479/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:13 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:14.134350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:14.265885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0a298-2003-4fdf-98fa-7876d97a1842 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:14.269442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:14.270299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:14.270645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:14.271027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] PATH_INFO: `/v3/users/7d51e5e0cb8f4e8e84ce6e63fe212acc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:14.272805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d51e5e0cb8f4e8e84ce6e63fe212acc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:14.278670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:14.394613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d51e5e0cb8f4e8e84ce6e63fe212acc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:14.400851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d51e5e0cb8f4e8e84ce6e63fe212acc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:14.428766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d51e5e0cb8f4e8e84ce6e63fe212acc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:14.452985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d51e5e0cb8f4e8e84ce6e63fe212acc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:14.469935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:14.498345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d51e5e0cb8f4e8e84ce6e63fe212acc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:14.510909 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1454/2933] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:02:14 2022] DELETE /identity/v3/users/7d51e5e0cb8f4e8e84ce6e63fe212acc => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:14.516043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:14.516378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:14.516779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:14.542773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:14.554124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:14.558591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:14.687326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1480/2934] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:02:14.740396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:14.745043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:14.855963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2962594-b75a-48ef-8f34-ff6a59e50ac6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:14.861310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e9723-5035-47d2-bd26-17d6b8d4470c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:14.862329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9723-5035-47d2-bd26-17d6b8d4470c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:14.862848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9723-5035-47d2-bd26-17d6b8d4470c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:14.863449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9723-5035-47d2-bd26-17d6b8d4470c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:14.884027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e9723-5035-47d2-bd26-17d6b8d4470c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:14.887817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e9723-5035-47d2-bd26-17d6b8d4470c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:14.905739 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1455/2935] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:15.225006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:15.226756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:15.253470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573c58-dbc1-4631-a400-91d414166b66 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:15.264937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:15.266570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:15.267364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:15.268138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] PATH_INFO: `/v3/users/36deeaadd647404e877c2b07ff9a4fbc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:15.271517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36deeaadd647404e877c2b07ff9a4fbc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:15.279479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:15.399499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36deeaadd647404e877c2b07ff9a4fbc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:15.400719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36deeaadd647404e877c2b07ff9a4fbc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:15.415919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36deeaadd647404e877c2b07ff9a4fbc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:15.432418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36deeaadd647404e877c2b07ff9a4fbc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:15.435007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36deeaadd647404e877c2b07ff9a4fbc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:15.488924 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1481/2936] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:02:15 2022] DELETE /identity/v3/users/36deeaadd647404e877c2b07ff9a4fbc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:15.494285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:15.495628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:15.496433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:15.522940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:15.523900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:15.629289 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1456/2937] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:15.648966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:15.651984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:15.764111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15def133-94d0-438b-95bd-735db8b14b96 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:15.770010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:15.771701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:15.771701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:15.771701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:15.786089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:15.789050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:15.806251 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1482/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:16.438760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:16.445406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:16.464727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c23e7d-0d6b-4083-8171-99856111138a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:16.480495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:16.485243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:16.485243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:16.485243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] PATH_INFO: `/v3/projects/9f3336f27cfd4866b6e527676bb6acb0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:16.496856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3336f27cfd4866b6e527676bb6acb0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:16.496856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:16.655080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3336f27cfd4866b6e527676bb6acb0', 'request_id': 'req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:16.662885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3336f27cfd4866b6e527676bb6acb0', 'request_id': 'req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:16.676482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3336f27cfd4866b6e527676bb6acb0', 'request_id': 'req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:16.710510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] The token cache is being invalidate because project 9f3336f27cfd4866b6e527676bb6acb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:02:16.713220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:16.715562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:16.718960 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1457/2939] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:16 2022] DELETE /identity/v3/projects/9f3336f27cfd4866b6e527676bb6acb0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:16.723396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:16.741919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36ad4d-85e8-445a-a618-718c42538f36 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:16.792202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:16.917929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:16.918627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:16.918904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:16.919181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] PATH_INFO: `/v3/users/9d6be144314e42d4bb25f70ef6ed3a43` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:16.924683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:16.926275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:16.939319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d6be144314e42d4bb25f70ef6ed3a43)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:16.941433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:16.953023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13d95c-7b45-43c3-ad6c-1aa5ee9e08bc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:16.959465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:16.960645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:16.960645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:16.961157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] PATH_INFO: `/v3/projects/644bbf0f519f4cd6a02e998859ec00e2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:16.963601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] RBAC: Authorizing `identity:delete_project(project_id=644bbf0f519f4cd6a02e998859ec00e2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:16.967811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:16.977968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:17.056351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644bbf0f519f4cd6a02e998859ec00e2', 'request_id': 'req-552522f0-038f-483b-b3ad-e1ff9cd1018d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.059831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d6be144314e42d4bb25f70ef6ed3a43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.062014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d6be144314e42d4bb25f70ef6ed3a43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.067061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644bbf0f519f4cd6a02e998859ec00e2', 'request_id': 'req-552522f0-038f-483b-b3ad-e1ff9cd1018d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.076091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d6be144314e42d4bb25f70ef6ed3a43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.082863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644bbf0f519f4cd6a02e998859ec00e2', 'request_id': 'req-552522f0-038f-483b-b3ad-e1ff9cd1018d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.097500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d6be144314e42d4bb25f70ef6ed3a43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.100394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c469600f-0eec-402b-854d-23e7d3987f8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d6be144314e42d4bb25f70ef6ed3a43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.106002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] The token cache is being invalidate because project 644bbf0f519f4cd6a02e998859ec00e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:02:17.106431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.107326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-552522f0-038f-483b-b3ad-e1ff9cd1018d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:17.109640 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1458/2940] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:16 2022] DELETE /identity/v3/projects/644bbf0f519f4cd6a02e998859ec00e2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:17.119303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81747b7-7e33-4cc9-9d0a-75aa59626ce0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:17.119774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81747b7-7e33-4cc9-9d0a-75aa59626ce0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:17.120101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81747b7-7e33-4cc9-9d0a-75aa59626ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:17.148536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b81747b7-7e33-4cc9-9d0a-75aa59626ce0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:17.150119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b81747b7-7e33-4cc9-9d0a-75aa59626ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:17.173193 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1483/2941] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:02:16 2022] DELETE /identity/v3/users/9d6be144314e42d4bb25f70ef6ed3a43 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:02:17.305831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0984b-47dd-404d-82d1-eef9f235006a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:17.306036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0984b-47dd-404d-82d1-eef9f235006a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:17.306306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0984b-47dd-404d-82d1-eef9f235006a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:17.306520 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1459/2942] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:17.348047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-44a0984b-47dd-404d-82d1-eef9f235006a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:17.349696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44a0984b-47dd-404d-82d1-eef9f235006a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:17.463310 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1484/2943] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:17.470784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81747b7-7e33-4cc9-9d0a-75aa59626ce0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:17.492611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81747b7-7e33-4cc9-9d0a-75aa59626ce0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:17.506879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:17.508053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:17.517075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:17.517696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:17.518708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:02:17.521222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:17.522990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:17.561577 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1460/2944] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:02:17.565940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0984b-47dd-404d-82d1-eef9f235006a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:17.576729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0984b-47dd-404d-82d1-eef9f235006a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:17.582723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:17.583466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:17.583673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:17.583889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:17.584548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:17.586749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:17.660196 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1485/2945] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:17 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:17.664337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:17.677555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bec71d-ec98-4864-a258-6d8b5f8f2f04 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:17.727401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:17.729896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:17.782661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:17.813146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:17.813879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:17.814159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:17.814432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:17.815390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:17.817874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:17.856460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1461/2946] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:17 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:02:17.863438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:17.904416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034db106-93ed-473c-8aa7-b514d2288663 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:17.912137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:17.912633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:17.913289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:17.913814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:17.927623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:17.932031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730ac63-2081-4d5b-8ab2-21d3e8df1a6d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:17.932825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:17.957068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:17.957068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:17.957068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:17.957068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] PATH_INFO: `/v3/projects/e65d33280d6e4c3289089cd018e5bba5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:17.957733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:17.957833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e65d33280d6e4c3289089cd018e5bba5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:17.957833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:17.966689 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1462/2947] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:02:17.967900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.002065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a23f2-2de0-4f88-b64c-3fcba4c4bd56 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.038126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.041931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.042213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.042492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.043172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.044859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.065407 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1463/2948] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:18.066559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.075281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e65d33280d6e4c3289089cd018e5bba5', 'request_id': 'req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:18.078895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dd5fb-9ec5-4365-ae83-7be82870cb43 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.089263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e65d33280d6e4c3289089cd018e5bba5', 'request_id': 'req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:18.101653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e65d33280d6e4c3289089cd018e5bba5', 'request_id': 'req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:18.120274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] The token cache is being invalidate because project e65d33280d6e4c3289089cd018e5bba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:02:18.121615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:18.123062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c78d9f80-ba35-4bbf-b334-95ff1a00cac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:18.124971 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1486/2949] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:17 2022] DELETE /identity/v3/projects/e65d33280d6e4c3289089cd018e5bba5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:02:18.130475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616199a0-ebdd-4c82-9d90-feff55b95f2c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.131110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616199a0-ebdd-4c82-9d90-feff55b95f2c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.131833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616199a0-ebdd-4c82-9d90-feff55b95f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.149347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e71f1d-891b-4a64-8fc1-c4c38305bc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.150302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71f1d-891b-4a64-8fc1-c4c38305bc48 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.150746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71f1d-891b-4a64-8fc1-c4c38305bc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.151092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71f1d-891b-4a64-8fc1-c4c38305bc48 admin admin] PATH_INFO: `/v3/projects/183cb066566c4510945d6eb0d41a2853/users/06dc3376ada04ffe922c9023f4182259/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.163303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-616199a0-ebdd-4c82-9d90-feff55b95f2c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:18.165219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-616199a0-ebdd-4c82-9d90-feff55b95f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:18.170972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e71f1d-891b-4a64-8fc1-c4c38305bc48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=183cb066566c4510945d6eb0d41a2853, user_id=06dc3376ada04ffe922c9023f4182259, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.173813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e71f1d-891b-4a64-8fc1-c4c38305bc48 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.185197 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1487/2950] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:18.197153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25cf8-bcc0-4f80-8f9f-d9d0d0bb2c95 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.198338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25cf8-bcc0-4f80-8f9f-d9d0d0bb2c95 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.198911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25cf8-bcc0-4f80-8f9f-d9d0d0bb2c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.215283 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1464/2951] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:18 2022] PUT /identity/v3/projects/183cb066566c4510945d6eb0d41a2853/users/06dc3376ada04ffe922c9023f4182259/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:18.222533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c180-4aec-4171-bb3f-3ea065ceb79f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.222799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c180-4aec-4171-bb3f-3ea065ceb79f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.223084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c180-4aec-4171-bb3f-3ea065ceb79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.241348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-80f25cf8-bcc0-4f80-8f9f-d9d0d0bb2c95 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:18.243066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-80f25cf8-bcc0-4f80-8f9f-d9d0d0bb2c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:18.273532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3be9c180-4aec-4171-bb3f-3ea065ceb79f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:18.275015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3be9c180-4aec-4171-bb3f-3ea065ceb79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:18.378525 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1488/2952] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:18.380416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f25cf8-bcc0-4f80-8f9f-d9d0d0bb2c95 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.398528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f25cf8-bcc0-4f80-8f9f-d9d0d0bb2c95 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.407819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.408932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.409420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.409882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.410979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:02:18.411516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.414757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.424413 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1465/2953] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:18.436274 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1489/2954] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:02:18.440927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c180-4aec-4171-bb3f-3ea065ceb79f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.446163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.450013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:18.452530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c180-4aec-4171-bb3f-3ea065ceb79f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.460409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.461536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.461823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.462092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.462745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.465134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.510406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:18.512292 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1466/2955] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:18 2022] POST /identity/v3/projects => generated 395 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:18.517475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.579764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f02ce-8a48-4506-872e-875dd56c87d3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.626264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:18.646741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baef4b-b57c-4aeb-90b3-4e2c070032a5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.652484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.653215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.653498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.653832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.654971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.659674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.676198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.677168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.677635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.678036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.690469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.692700 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1467/2956] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:18 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:18.692865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.697265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.705984 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1490/2957] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:18.714847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d117d1-f947-41f6-822d-920d8de59cd0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.720005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.720891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.721679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.721679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.722098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.723732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.735699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1468/2958] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:02:18.739552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.750558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3248e-1520-4895-964e-a73976f09ceb service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.756189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093e260-57c4-4aae-995b-11bcbfac2263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.756963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093e260-57c4-4aae-995b-11bcbfac2263 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.757263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093e260-57c4-4aae-995b-11bcbfac2263 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.757633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093e260-57c4-4aae-995b-11bcbfac2263 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.757633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093e260-57c4-4aae-995b-11bcbfac2263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.759833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093e260-57c4-4aae-995b-11bcbfac2263 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.779247 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1491/2959] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:18.780693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.792213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793c0a-6303-471e-aedd-15e72978c0f6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:18.797354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:18.798027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.798314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.798613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] PATH_INFO: `/v3/projects/de87d5984a9a4d4481ab4921133841d6/users/c7468cc8bc7d4dde8e2a5eba1fa42fd8/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.815455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de87d5984a9a4d4481ab4921133841d6, user_id=c7468cc8bc7d4dde8e2a5eba1fa42fd8, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:18.818216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:18.841674 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1469/2960] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:18 2022] PUT /identity/v3/projects/de87d5984a9a4d4481ab4921133841d6/users/c7468cc8bc7d4dde8e2a5eba1fa42fd8/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:18.846623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2a3a5-06a0-43c4-be94-55aea5a23a0b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:18.846943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2a3a5-06a0-43c4-be94-55aea5a23a0b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:18.847667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2a3a5-06a0-43c4-be94-55aea5a23a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:18.869739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-42f2a3a5-06a0-43c4-be94-55aea5a23a0b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:18.869976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-42f2a3a5-06a0-43c4-be94-55aea5a23a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:18.952468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1492/2961] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:02:18.960008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:18.961938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:19.112315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3b433-6c52-4f80-935b-21f1397209f2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:19.129006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4286-ca5d-4a86-82ba-9b110299fce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:19.129785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4286-ca5d-4a86-82ba-9b110299fce1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:19.130409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4286-ca5d-4a86-82ba-9b110299fce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:19.132202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4286-ca5d-4a86-82ba-9b110299fce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:19.158125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c4286-ca5d-4a86-82ba-9b110299fce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:19.160509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c4286-ca5d-4a86-82ba-9b110299fce1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:19.181624 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1470/2962] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:24.422863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2a3a5-06a0-43c4-be94-55aea5a23a0b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:24.436702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2a3a5-06a0-43c4-be94-55aea5a23a0b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:24.446240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:24.447081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:24.447514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:24.447939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:24.452352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:24.596560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:24.598079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:24.623460 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1493/2963] 10.222.0.26 () {60 vars in 1396 bytes} [Tue Sep 27 10:02:24 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:27.851198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e87b2-36b0-4656-a637-f3bf69cf04a3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:27.851780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e87b2-36b0-4656-a637-f3bf69cf04a3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:27.851780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e87b2-36b0-4656-a637-f3bf69cf04a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:27.912056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1e2e87b2-36b0-4656-a637-f3bf69cf04a3 None None] MFA Rules not processed for user `06dc3376ada04ffe922c9023f4182259`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:27.914707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1e2e87b2-36b0-4656-a637-f3bf69cf04a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:28.112839 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1471/2964] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:28.195826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:28.212725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:28.253168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:28.396081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510bfae-e5ae-4dad-ae3a-e9463c104dbd service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:28.406377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c3c1e-a298-4257-9850-8b313c8a22a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:28.407296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c3c1e-a298-4257-9850-8b313c8a22a6 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:28.407577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c3c1e-a298-4257-9850-8b313c8a22a6 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:28.407849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c3c1e-a298-4257-9850-8b313c8a22a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:28.429277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c3c1e-a298-4257-9850-8b313c8a22a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:28.431080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c3c1e-a298-4257-9850-8b313c8a22a6 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:28.444220 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1494/2965] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:02:28.501191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47de28-9d84-4056-bf61-35f0eb79f4a9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:28.501502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47de28-9d84-4056-bf61-35f0eb79f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:28.502041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47de28-9d84-4056-bf61-35f0eb79f4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:28.548229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8c47de28-9d84-4056-bf61-35f0eb79f4a9 None None] MFA Rules not processed for user `c7468cc8bc7d4dde8e2a5eba1fa42fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:28.549499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8c47de28-9d84-4056-bf61-35f0eb79f4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:28.645642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:28.800744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b17e9-9b3c-483b-b1eb-27e6993a34a8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:28.801035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b17e9-9b3c-483b-b1eb-27e6993a34a8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:28.801311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b17e9-9b3c-483b-b1eb-27e6993a34a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:28.804635 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1472/2966] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:28.854134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6e6b17e9-9b3c-483b-b1eb-27e6993a34a8 None None] MFA Rules not processed for user `c7468cc8bc7d4dde8e2a5eba1fa42fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:28.857046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e6b17e9-9b3c-483b-b1eb-27e6993a34a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:28.885327 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1495/2967] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:28.926449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47de28-9d84-4056-bf61-35f0eb79f4a9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:28.954549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47de28-9d84-4056-bf61-35f0eb79f4a9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:28.970853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:28.971549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:28.974027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:28.974511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:28.998905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:29.000599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:29.022866 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1473/2968] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:28 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:31.714136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787edb60-6206-4232-8eca-f264f5c12cd4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:31.715017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787edb60-6206-4232-8eca-f264f5c12cd4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:31.715485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787edb60-6206-4232-8eca-f264f5c12cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:31.770640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-787edb60-6206-4232-8eca-f264f5c12cd4 None None] MFA Rules not processed for user `06dc3376ada04ffe922c9023f4182259`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:31.787698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-787edb60-6206-4232-8eca-f264f5c12cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:31.802935 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1496/2969] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:36.428362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:36.430662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:36.450830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccc314-1411-4c7d-a3b1-d612d4277841 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:36.460322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:36.461317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:36.461515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:36.461790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] PATH_INFO: `/v3/users/b31b5932bb8e4edcadc6954c46c2c386` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:36.462961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31b5932bb8e4edcadc6954c46c2c386)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:36.464942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:36.548852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31b5932bb8e4edcadc6954c46c2c386'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:36.548852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31b5932bb8e4edcadc6954c46c2c386'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:36.560324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31b5932bb8e4edcadc6954c46c2c386'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:36.572456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31b5932bb8e4edcadc6954c46c2c386'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:36.618529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c44e3fec-0f82-47f2-8699-26be2367e7a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31b5932bb8e4edcadc6954c46c2c386'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:36.638132 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1474/2970] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:02:36 2022] DELETE /identity/v3/users/b31b5932bb8e4edcadc6954c46c2c386 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:37.101756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787edb60-6206-4232-8eca-f264f5c12cd4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:37.125873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787edb60-6206-4232-8eca-f264f5c12cd4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:37.241561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:37.242555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.243101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.243531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] PATH_INFO: `/v3/projects/2750135860944b03816344b2e7e58f30` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.245266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2750135860944b03816344b2e7e58f30)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:37.247739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:37.356190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2750135860944b03816344b2e7e58f30', 'request_id': 'req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:37.375581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2750135860944b03816344b2e7e58f30', 'request_id': 'req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:37.384964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2750135860944b03816344b2e7e58f30', 'request_id': 'req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:37.416954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] The token cache is being invalidate because project 2750135860944b03816344b2e7e58f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:02:37.418123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:37.423638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3bf93d3-ef2f-4593-8a8e-06896ea76f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:02:37.428435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1497/2971] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:37 2022] DELETE /identity/v3/projects/2750135860944b03816344b2e7e58f30 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:37.435208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78765e-b702-4547-8a46-cf76c5c89280 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.436144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78765e-b702-4547-8a46-cf76c5c89280 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.436736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78765e-b702-4547-8a46-cf76c5c89280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.496857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0c78765e-b702-4547-8a46-cf76c5c89280 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:37.499325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0c78765e-b702-4547-8a46-cf76c5c89280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:37.587047 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1475/2972] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:37.597329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283b581-c7d7-4eb6-a14a-f2d15ad525eb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.597910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283b581-c7d7-4eb6-a14a-f2d15ad525eb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.598354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283b581-c7d7-4eb6-a14a-f2d15ad525eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.626443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5283b581-c7d7-4eb6-a14a-f2d15ad525eb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:37.629265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5283b581-c7d7-4eb6-a14a-f2d15ad525eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:37.647856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1498/2973] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:37.651866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78765e-b702-4547-8a46-cf76c5c89280 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:37.663653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78765e-b702-4547-8a46-cf76c5c89280 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:37.668917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:37.669733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.670012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.670280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.670949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:02:37.671249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:37.672938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:37.691681 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1476/2974] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:02:37.699333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283b581-c7d7-4eb6-a14a-f2d15ad525eb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:37.710923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283b581-c7d7-4eb6-a14a-f2d15ad525eb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:37.721288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:37.721925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.722306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.722585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.723227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:37.727060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:37.756264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:37.791409 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1499/2975] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:02:37 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:37.792413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:37.812863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b09b26-be4c-43dc-959b-4c598384ec7f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:37.829593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:37.892516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:37.893173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.893595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.893878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.894521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:37.896753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:37.925813 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1477/2976] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:02:37 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:02:37.930311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:37.942445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562e42-65cb-4979-9eb9-8b9148e70456 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:37.948100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:37.948758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.949030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.949295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.949935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:37.951531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:37.968709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1500/2977] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:02:37.970706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:37.982609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beab34-6c01-469c-bce5-d253d85acfe3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:37.988789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26370c-0823-4e46-a09b-0d45e2bc3efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:37.989349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26370c-0823-4e46-a09b-0d45e2bc3efe admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:37.989629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26370c-0823-4e46-a09b-0d45e2bc3efe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:37.989903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26370c-0823-4e46-a09b-0d45e2bc3efe admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:37.990558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26370c-0823-4e46-a09b-0d45e2bc3efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:37.992292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26370c-0823-4e46-a09b-0d45e2bc3efe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:38.004312 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1478/2978] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:02:38.009054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:38.019852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c014cec-1178-4b04-b871-00e9d1d17913 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:38.025682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:38.026344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:38.026615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:38.026891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] PATH_INFO: `/v3/projects/f48a592eb8fa4221a56eac1a50152c91/users/ac98fc0d0296493999906e8e3607c7f3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:38.042889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48a592eb8fa4221a56eac1a50152c91, user_id=ac98fc0d0296493999906e8e3607c7f3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:38.045644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:38.061631 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1501/2979] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:02:38 2022] PUT /identity/v3/projects/f48a592eb8fa4221a56eac1a50152c91/users/ac98fc0d0296493999906e8e3607c7f3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:02:38.066947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2d87c-e7d6-48eb-bda1-a6b15e492d69 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:38.067259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2d87c-e7d6-48eb-bda1-a6b15e492d69 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:38.067853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2d87c-e7d6-48eb-bda1-a6b15e492d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:38.090690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c5d2d87c-e7d6-48eb-bda1-a6b15e492d69 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:38.092371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c5d2d87c-e7d6-48eb-bda1-a6b15e492d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:38.169899 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1479/2980] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:38.179923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:38.182838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:38.261830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e69f5-f4ce-43e1-9e19-02f9919729ec admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:38.267007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c3dce-0276-4b9e-846d-e6c0e33305a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:38.267690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c3dce-0276-4b9e-846d-e6c0e33305a4 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:38.267966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c3dce-0276-4b9e-846d-e6c0e33305a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:38.268231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c3dce-0276-4b9e-846d-e6c0e33305a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:38.279605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c3dce-0276-4b9e-846d-e6c0e33305a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:38.282064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c3dce-0276-4b9e-846d-e6c0e33305a4 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:38.301029 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1502/2981] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:39.541358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff8b3-119c-4c61-96ab-cbba86b134c3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:39.542472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff8b3-119c-4c61-96ab-cbba86b134c3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:39.542913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff8b3-119c-4c61-96ab-cbba86b134c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:39.589892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-607ff8b3-119c-4c61-96ab-cbba86b134c3 None None] MFA Rules not processed for user `ac98fc0d0296493999906e8e3607c7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:39.591002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-607ff8b3-119c-4c61-96ab-cbba86b134c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:39.698064 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1480/2982] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:39.714116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:39.714714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:39.714845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:39.754957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] MFA Rules not processed for user `ac98fc0d0296493999906e8e3607c7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:39.757019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:39.777065 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1503/2983] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:39.781926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3fdc8-6047-46d8-860f-ebdc7846ec38 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:39.782518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3fdc8-6047-46d8-860f-ebdc7846ec38 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:39.783569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3fdc8-6047-46d8-860f-ebdc7846ec38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:39.826681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-03b3fdc8-6047-46d8-860f-ebdc7846ec38 None None] MFA Rules not processed for user `ac98fc0d0296493999906e8e3607c7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:39.827925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-03b3fdc8-6047-46d8-860f-ebdc7846ec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:39.836816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1481/2984] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:39.852356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:39.854590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:39.998942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5fa3a-ca5c-4ce7-a928-ed511cfcf483 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:40.010019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec54fbb-1318-49ac-ae9c-abd7666b1abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:40.010801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54fbb-1318-49ac-ae9c-abd7666b1abd service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:40.010925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54fbb-1318-49ac-ae9c-abd7666b1abd service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:40.011200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54fbb-1318-49ac-ae9c-abd7666b1abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:40.041745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec54fbb-1318-49ac-ae9c-abd7666b1abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:40.043378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec54fbb-1318-49ac-ae9c-abd7666b1abd service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:40.067226 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1504/2985] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:02:40.833509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f63c-207e-4725-9e66-fbbf11eeb767 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:40.834505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f63c-207e-4725-9e66-fbbf11eeb767 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:40.835001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f63c-207e-4725-9e66-fbbf11eeb767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:40.862132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1e34f63c-207e-4725-9e66-fbbf11eeb767 None None] MFA Rules not processed for user `ac98fc0d0296493999906e8e3607c7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:40.863421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1e34f63c-207e-4725-9e66-fbbf11eeb767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:40.873336 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1482/2986] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:41.988765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b743d-b97b-473d-8887-656654663a8b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:41.989698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b743d-b97b-473d-8887-656654663a8b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:41.990171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b743d-b97b-473d-8887-656654663a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:42.048855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-574b743d-b97b-473d-8887-656654663a8b None None] MFA Rules not processed for user `ac98fc0d0296493999906e8e3607c7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:42.050111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-574b743d-b97b-473d-8887-656654663a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:42.086455 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1505/2987] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:49.911476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc013f87-3880-4e4c-8e2e-6d7737f3ca33 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:49.912141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc013f87-3880-4e4c-8e2e-6d7737f3ca33 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:49.912141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc013f87-3880-4e4c-8e2e-6d7737f3ca33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:49.949833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dc013f87-3880-4e4c-8e2e-6d7737f3ca33 None None] MFA Rules not processed for user `c7468cc8bc7d4dde8e2a5eba1fa42fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:02:49.954840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dc013f87-3880-4e4c-8e2e-6d7737f3ca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:49.971791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:50.080102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1483/2988] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:02:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:02:50.528940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b743d-b97b-473d-8887-656654663a8b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:02:50.560177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b743d-b97b-473d-8887-656654663a8b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:02:50.567991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:02:50.568935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:02:50.569472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:02:50.570007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:02:50.574261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:02:50.595076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:02:50.597024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:02:50.627786 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1506/2989] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:02:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:00.058165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=21 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:00.624610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:01.975549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc013f87-3880-4e4c-8e2e-6d7737f3ca33 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:01.993327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc013f87-3880-4e4c-8e2e-6d7737f3ca33 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:01.999214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:01.999842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:02.000119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:02.000391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:02.002563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:02.071105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:02.073131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:02.085127 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1484/2990] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:08.684743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:08.686781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:08.788138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a4d4-d534-4ed7-8f8f-5feeaff9dabc service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:08.794493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6bb3be-59bf-480e-a2a9-51b183c0c8ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:08.795116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bb3be-59bf-480e-a2a9-51b183c0c8ff service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:08.795393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bb3be-59bf-480e-a2a9-51b183c0c8ff service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:08.795661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bb3be-59bf-480e-a2a9-51b183c0c8ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:08.797648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3d6bb3be-59bf-480e-a2a9-51b183c0c8ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:08.815647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6bb3be-59bf-480e-a2a9-51b183c0c8ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:08.817332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6bb3be-59bf-480e-a2a9-51b183c0c8ff service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:08.831378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1507/2991] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:12.080570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=30 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:13.587371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:13.589339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:13.613103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09e5af-2bb5-4a11-b030-6ac174c48d31 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:13.618355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:13.619258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:13.619867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:13.620473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:13.622981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:13.638695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:13.640563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:13.662110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1485/2992] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:17.614842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446c43-21de-487c-84ef-881846c171c3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:17.615347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446c43-21de-487c-84ef-881846c171c3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:17.620330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446c43-21de-487c-84ef-881846c171c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:17.677065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-81446c43-21de-487c-84ef-881846c171c3 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:17.679039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-81446c43-21de-487c-84ef-881846c171c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:17.861690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1508/2993] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:03:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:17.944478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:17.947305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:18.041365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50685618-21e7-45b9-bdee-da0acff0f4a3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:18.047815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:18.048834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:18.049112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:18.049384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:18.059064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:18.061106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:18.070749 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1486/2994] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:03:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:20.348091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:20.348623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:20.348764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:20.397892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:20.401246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:20.415816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1509/2995] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:03:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:03:20.431536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:20.446195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bac79f-a76c-4cd3-a7f6-a47b0154d6a6 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:20.452040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b1713-02e7-40d8-a8d6-cb6d08497e5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:20.452941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b1713-02e7-40d8-a8d6-cb6d08497e5d service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:20.453404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b1713-02e7-40d8-a8d6-cb6d08497e5d service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:20.453838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b1713-02e7-40d8-a8d6-cb6d08497e5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:20.464232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b1713-02e7-40d8-a8d6-cb6d08497e5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:20.466336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b1713-02e7-40d8-a8d6-cb6d08497e5d service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:20.476751 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1487/2996] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:03:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:03:30.380634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=60 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:30.472554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=44 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:34.504648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:34.507547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:34.526219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641f1c6-6fe7-4fa2-90f7-a151101fe865 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:34.531694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:34.532622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:34.533026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:34.533415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] PATH_INFO: `/v3/users/9544cdb0392f416197a586d99e8a3210` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:34.534649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9544cdb0392f416197a586d99e8a3210)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:34.537762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:34.640948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9544cdb0392f416197a586d99e8a3210'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:34.642015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9544cdb0392f416197a586d99e8a3210'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:34.653551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9544cdb0392f416197a586d99e8a3210'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:34.674669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9544cdb0392f416197a586d99e8a3210'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:34.679293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9544cdb0392f416197a586d99e8a3210'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:34.736870 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1510/2997] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:03:34 2022] DELETE /identity/v3/users/9544cdb0392f416197a586d99e8a3210 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:34.742040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:34.744594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:34.745544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:34.790901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:34.793864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:34.989303 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1488/2998] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:35.078030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:35.079914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:35.242867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa8bd-b988-48e5-a8d4-e4abc26c3fb8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:35.253915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:35.255244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:35.257164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:35.259913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:35.284415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:35.288567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:35.318557 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1511/2999] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:36.305281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:36.305685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:36.346892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc02c-8981-409c-806d-4f5054c189ac None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:36.360865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:36.361987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:36.362455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:36.363440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] PATH_INFO: `/v3/projects/e79eae6310354ca58046e59dfac23009` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:36.365285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e79eae6310354ca58046e59dfac23009)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:36.367752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:36.509018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e79eae6310354ca58046e59dfac23009', 'request_id': 'req-71eba5e9-3da3-418f-8d72-c0cfacae5457'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.539002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e79eae6310354ca58046e59dfac23009', 'request_id': 'req-71eba5e9-3da3-418f-8d72-c0cfacae5457'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.565285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e79eae6310354ca58046e59dfac23009', 'request_id': 'req-71eba5e9-3da3-418f-8d72-c0cfacae5457'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.587282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] The token cache is being invalidate because project e79eae6310354ca58046e59dfac23009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:03:36.587648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.589105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.595734 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1489/3000] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:36 2022] DELETE /identity/v3/projects/e79eae6310354ca58046e59dfac23009 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:36.599678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:36.618179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c3d05-eeb1-494d-947d-253f833ad574 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:36.720936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:36.722264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:36.722719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:36.723296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] PATH_INFO: `/v3/users/f74e77337957458794bd6c942c97b344` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:36.724713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f74e77337957458794bd6c942c97b344)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:36.727720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:36.758275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:36.824728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74e77337957458794bd6c942c97b344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.827372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74e77337957458794bd6c942c97b344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.845367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74e77337957458794bd6c942c97b344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.866973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74e77337957458794bd6c942c97b344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.868687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ae12072-b11b-4922-806d-abc70c6e7569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74e77337957458794bd6c942c97b344'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:36.902243 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1512/3001] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:03:36 2022] DELETE /identity/v3/users/f74e77337957458794bd6c942c97b344 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:03:38.250928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:38.253072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:38.272558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eba5e9-3da3-418f-8d72-c0cfacae5457 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:38.357366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:38.358156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:38.358434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:38.358707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] PATH_INFO: `/v3/projects/a3199e83259042efada3f2a565c624ff` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:38.360248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3199e83259042efada3f2a565c624ff)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:38.362610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:38.408059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:38.435107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3199e83259042efada3f2a565c624ff', 'request_id': 'req-6930b174-e763-4674-9124-d79944eb1d36'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:38.442508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3199e83259042efada3f2a565c624ff', 'request_id': 'req-6930b174-e763-4674-9124-d79944eb1d36'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:38.455478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3199e83259042efada3f2a565c624ff', 'request_id': 'req-6930b174-e763-4674-9124-d79944eb1d36'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:38.469711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] The token cache is being invalidate because project a3199e83259042efada3f2a565c624ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:03:38.470986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:38.471720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6930b174-e763-4674-9124-d79944eb1d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:38.474502 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1490/3002] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:38 2022] DELETE /identity/v3/projects/a3199e83259042efada3f2a565c624ff => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:38.479409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285da03-820a-4250-bb68-feb0fb7dd855 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:38.481107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285da03-820a-4250-bb68-feb0fb7dd855 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:38.481443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285da03-820a-4250-bb68-feb0fb7dd855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:38.506368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2285da03-820a-4250-bb68-feb0fb7dd855 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:38.507560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2285da03-820a-4250-bb68-feb0fb7dd855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:38.638886 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1513/3003] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:38.646265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84f530-d021-49c3-8761-1b83d8abba78 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:38.646954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84f530-d021-49c3-8761-1b83d8abba78 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:38.646954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84f530-d021-49c3-8761-1b83d8abba78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:38.707053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8c84f530-d021-49c3-8761-1b83d8abba78 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:38.708719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8c84f530-d021-49c3-8761-1b83d8abba78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:38.719696 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1491/3004] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:38.723783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285da03-820a-4250-bb68-feb0fb7dd855 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:38.741020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285da03-820a-4250-bb68-feb0fb7dd855 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:38.747153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:38.747978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:38.748464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:38.749105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:38.749941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:03:38.750566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:38.751333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:38.751618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:38.752147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:38.752271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:38.773746 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1514/3005] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:03:38.775733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:38.780191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:38.781809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:38.787596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256433-4e0c-493b-bcf3-206ab65e3541 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:38.794645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:38.795442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:38.795727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:38.795993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:38.796636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:38.799385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:38.851557 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1515/3006] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:03:38 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:38.856485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:38.865495 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1492/3007] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:03:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:03:38.882985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:38.885729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:38.950069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48167b06-761c-4dbf-ac02-135b85c5d544 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:38.955014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:38.955633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:38.955928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:38.956177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:38.956965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:38.959047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:38.968931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:38.973090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc4800-2998-4fd5-bafe-329c55b2b749 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:38.986503 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1516/3008] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:03:38 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:38.990380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.001424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b286db-267d-41c8-881b-b10d8884edac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:39.006876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.007502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.007781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.008055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.009095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.010849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.025292 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1517/3009] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:03:39.026291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.039417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65997370-7c1d-46cc-9bab-7ed9975b4a85 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:39.045616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.046277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.046546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.046819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.047463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.047724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:39.049123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.064176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1518/3010] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:03:39.068386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.069781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.070937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.071515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.072076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.083395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f36b9-5b0f-4336-8c1e-9857707f54ce admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:39.089559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9bb21-1d68-464c-b88d-ed9989b50fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.090577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9bb21-1d68-464c-b88d-ed9989b50fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.090873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9bb21-1d68-464c-b88d-ed9989b50fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.091134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9bb21-1d68-464c-b88d-ed9989b50fd5 admin admin] PATH_INFO: `/v3/projects/8b87f09e1bf645a4b71d8b31ee512ec0/users/b7892230632b490c8f110206461a87c6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.111618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9bb21-1d68-464c-b88d-ed9989b50fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b87f09e1bf645a4b71d8b31ee512ec0, user_id=b7892230632b490c8f110206461a87c6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.116851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9bb21-1d68-464c-b88d-ed9989b50fd5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.141748 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1519/3011] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:03:39 2022] PUT /identity/v3/projects/8b87f09e1bf645a4b71d8b31ee512ec0/users/b7892230632b490c8f110206461a87c6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:03:39.143500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.145309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.153706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c9f9-ecb4-493a-8c77-940263ec818a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.154678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c9f9-ecb4-493a-8c77-940263ec818a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.154678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c9f9-ecb4-493a-8c77-940263ec818a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.169365 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1493/3012] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:03:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:03:39.190150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e611c9f9-ecb4-493a-8c77-940263ec818a None None] MFA Rules not processed for user `b7892230632b490c8f110206461a87c6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:39.191323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e611c9f9-ecb4-493a-8c77-940263ec818a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:39.310373 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1520/3013] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:39.315520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.333967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a32e53-a959-4b74-8b14-90ff32caf5e3 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:39.459952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.461143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.461778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.462382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.463254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.465413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.537156 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1494/3014] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:03:39 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:39.544059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611c9f9-ecb4-493a-8c77-940263ec818a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.569461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611c9f9-ecb4-493a-8c77-940263ec818a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:39.719513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.720246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.720599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.721045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.721765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.723829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.756710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1521/3015] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:03:39 2022] POST /identity/v3/users => generated 295 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:39.762092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.781763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88376-0a12-4b3e-a318-7fe22cee57b3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:39.791264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.792916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.793340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.798720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.808246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.815683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.852811 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1495/3016] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:03:39.854191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.866406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292250d-8535-464c-8f22-9eea39b12964 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:39.874080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:39.874860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:39.875228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:39.875571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] PATH_INFO: `/v3/projects/6e1a0a1845644719b8c9a7c7fcbd6f2b/users/5fdeecb899e949f998041edf57ae4548/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:39.897674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1a0a1845644719b8c9a7c7fcbd6f2b, user_id=5fdeecb899e949f998041edf57ae4548, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:39.901292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:39.928953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:39.941521 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1522/3017] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:03:39 2022] PUT /identity/v3/projects/6e1a0a1845644719b8c9a7c7fcbd6f2b/users/5fdeecb899e949f998041edf57ae4548/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:39.945945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:39.962297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72751-55d8-4cc1-b489-d12ce9368eb9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:40.053718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d397eb-e4be-47ec-bb58-c7fefc6fe967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:40.054720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d397eb-e4be-47ec-bb58-c7fefc6fe967 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:40.055027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d397eb-e4be-47ec-bb58-c7fefc6fe967 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:40.055476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d397eb-e4be-47ec-bb58-c7fefc6fe967 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:40.055974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d397eb-e4be-47ec-bb58-c7fefc6fe967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:40.058444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d397eb-e4be-47ec-bb58-c7fefc6fe967 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:40.078077 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1496/3018] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:03:40.079893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:40.108355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1c655-72cc-44ee-9744-49a6d90862a0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:40.119363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030c51e-6c03-4db1-89e3-67aec1e226f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:40.124498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030c51e-6c03-4db1-89e3-67aec1e226f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:40.124798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030c51e-6c03-4db1-89e3-67aec1e226f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:40.125078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030c51e-6c03-4db1-89e3-67aec1e226f6 admin admin] PATH_INFO: `/v3/projects/6e1a0a1845644719b8c9a7c7fcbd6f2b/users/5fdeecb899e949f998041edf57ae4548/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:40.130779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030c51e-6c03-4db1-89e3-67aec1e226f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1a0a1845644719b8c9a7c7fcbd6f2b, user_id=5fdeecb899e949f998041edf57ae4548, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:40.136228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030c51e-6c03-4db1-89e3-67aec1e226f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:40.151624 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1523/3019] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:03:40 2022] PUT /identity/v3/projects/6e1a0a1845644719b8c9a7c7fcbd6f2b/users/5fdeecb899e949f998041edf57ae4548/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:40.162325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529645-1cdc-4ce7-9643-459876d58ace None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:40.162756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529645-1cdc-4ce7-9643-459876d58ace None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:40.162876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529645-1cdc-4ce7-9643-459876d58ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:40.179674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:40.232436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ea529645-1cdc-4ce7-9643-459876d58ace None None] MFA Rules not processed for user `5fdeecb899e949f998041edf57ae4548`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:40.237141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ea529645-1cdc-4ce7-9643-459876d58ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:40.408821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1497/3020] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:40.421031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:40.421301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:40.421571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:40.475208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] MFA Rules not processed for user `b7892230632b490c8f110206461a87c6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:40.490627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:40.593342 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1524/3021] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:41.209246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:41.209548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:41.210568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:41.235235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:41.236883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:41.312485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] MFA Rules not processed for user `6eb3ccfcd9c344e1a7ad55f8398ec87a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:41.315351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:41.489992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1498/3022] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:41.497740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecf82e-52d6-4e8f-863c-71b8890fd97b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:41.504885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:41.505552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:41.505840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:41.506112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:41.516794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:41.519423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:41.531512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1525/3023] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:44.648295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:44.652536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:44.857013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189434cc-2add-4745-a83a-c298fdbfd9fa None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:44.863004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:44.863767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:44.864036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:44.864359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] PATH_INFO: `/v3/users/0d563feacff84b4c9927ec759fb365be` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:44.868279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d563feacff84b4c9927ec759fb365be)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:44.869152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:44.983377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:45.001933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d563feacff84b4c9927ec759fb365be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.002955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d563feacff84b4c9927ec759fb365be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.022222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d563feacff84b4c9927ec759fb365be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.038773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d563feacff84b4c9927ec759fb365be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.066749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d563feacff84b4c9927ec759fb365be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.091015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1499/3024] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:03:44 2022] DELETE /identity/v3/users/0d563feacff84b4c9927ec759fb365be => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:45.533823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:45.579116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137091fb-3649-4a18-8d1f-9eeb21c08e42 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:45.741214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:45.742712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:45.743779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:45.744504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] PATH_INFO: `/v3/projects/5de10e92e2024a24880c7205697e5732` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:45.747503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de10e92e2024a24880c7205697e5732)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:45.752683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:45.804622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:45.882948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de10e92e2024a24880c7205697e5732', 'request_id': 'req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.897575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de10e92e2024a24880c7205697e5732', 'request_id': 'req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.908868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de10e92e2024a24880c7205697e5732', 'request_id': 'req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.928508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] The token cache is being invalidate because project 5de10e92e2024a24880c7205697e5732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:03:45.928883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.929622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:45.954805 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1526/3025] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:45 2022] DELETE /identity/v3/projects/5de10e92e2024a24880c7205697e5732 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:45.959983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:45.966543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:45.984707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1fe1a-4ddc-4690-9b8a-3a15d2d7d831 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:46.112931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:46.113598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:46.113881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:46.114160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] PATH_INFO: `/v3/users/7d6172f063be4dc48a01a40dd71b67e1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:46.115496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6172f063be4dc48a01a40dd71b67e1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:46.117966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:46.189700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6172f063be4dc48a01a40dd71b67e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.190497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6172f063be4dc48a01a40dd71b67e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.201765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6172f063be4dc48a01a40dd71b67e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.214692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6172f063be4dc48a01a40dd71b67e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.258085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdda952f-0208-485c-8638-ab141c3a9963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6172f063be4dc48a01a40dd71b67e1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.272030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1500/3026] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:03:45 2022] DELETE /identity/v3/users/7d6172f063be4dc48a01a40dd71b67e1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:46.684138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:46.686041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:46.710562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa94c0-2b98-4c24-a62f-0d5ea9e6b4b7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:46.809312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:46.810336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:46.810903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:46.811411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] PATH_INFO: `/v3/projects/6478a4ad3f4e4c7493f011513b7bb36d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:46.813362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=6478a4ad3f4e4c7493f011513b7bb36d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:46.815580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:46.903505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6478a4ad3f4e4c7493f011513b7bb36d', 'request_id': 'req-e707a145-77c5-462d-8138-95384cf987ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.916484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6478a4ad3f4e4c7493f011513b7bb36d', 'request_id': 'req-e707a145-77c5-462d-8138-95384cf987ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.929611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6478a4ad3f4e4c7493f011513b7bb36d', 'request_id': 'req-e707a145-77c5-462d-8138-95384cf987ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.949217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] The token cache is being invalidate because project 6478a4ad3f4e4c7493f011513b7bb36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:03:46.949784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.950630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e707a145-77c5-462d-8138-95384cf987ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:46.953376 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1527/3027] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:46 2022] DELETE /identity/v3/projects/6478a4ad3f4e4c7493f011513b7bb36d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:46.958341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe3706-fdff-4da3-b68b-2cf4a83dc519 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:46.958906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe3706-fdff-4da3-b68b-2cf4a83dc519 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:46.959343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe3706-fdff-4da3-b68b-2cf4a83dc519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:46.996322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1dfe3706-fdff-4da3-b68b-2cf4a83dc519 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:46.998380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1dfe3706-fdff-4da3-b68b-2cf4a83dc519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:47.063820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:47.141569 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1501/3028] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:47.149411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e100d-672d-4039-8581-84b6740dad6d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:47.150004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e100d-672d-4039-8581-84b6740dad6d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:47.150486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e100d-672d-4039-8581-84b6740dad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:47.179242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e21e100d-672d-4039-8581-84b6740dad6d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:47.182672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e21e100d-672d-4039-8581-84b6740dad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:47.211167 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1528/3029] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:47.215588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe3706-fdff-4da3-b68b-2cf4a83dc519 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:47.227273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe3706-fdff-4da3-b68b-2cf4a83dc519 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:47.243141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:47.244285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:47.244847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:47.245828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:47.246908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:03:47.247518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:47.253214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:47.333854 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1502/3030] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:03:47.340810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e100d-672d-4039-8581-84b6740dad6d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:47.355018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e100d-672d-4039-8581-84b6740dad6d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:47.372445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:47.373972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:47.376092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:47.376937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:47.378185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:47.388220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:47.490859 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1529/3031] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:03:47 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:47.496738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:47.516510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3df18d-4a3c-4fe0-b34e-940aa8789b70 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:47.697135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:47.698713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:47.699168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:47.703907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:47.706944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:47.710008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:47.800703 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1503/3032] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:03:47 2022] POST /identity/v3/users => generated 286 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:47.804882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:47.832463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbe74-3e49-4e40-9e98-575816309ee3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:47.842814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:47.843766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:47.844272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:47.844778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:47.845822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:47.856419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:47.861814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:47.885818 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1530/3033] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:03:47.892185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:47.909748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c23ec-fa75-4828-9a9d-c3de783d02da admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:47.927893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57027-33a8-4860-b063-424eb2adf57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:47.928752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57027-33a8-4860-b063-424eb2adf57a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:47.929119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57027-33a8-4860-b063-424eb2adf57a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:47.929491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57027-33a8-4860-b063-424eb2adf57a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:47.930488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be57027-33a8-4860-b063-424eb2adf57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:47.932290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be57027-33a8-4860-b063-424eb2adf57a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:47.957355 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1504/3034] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:03:47.962543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:47.980864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716508c-4068-4123-baf0-1a7bdd742afc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:47.999598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e431a3-8fd1-4c26-afff-77e9b4b16144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:47.999853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e431a3-8fd1-4c26-afff-77e9b4b16144 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:48.000649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e431a3-8fd1-4c26-afff-77e9b4b16144 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:48.000953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e431a3-8fd1-4c26-afff-77e9b4b16144 admin admin] PATH_INFO: `/v3/projects/8af81d3bf64548369ac9758cfee75c97/users/d59cddfd22844651995126a5d5d3d235/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:48.037245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e431a3-8fd1-4c26-afff-77e9b4b16144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af81d3bf64548369ac9758cfee75c97, user_id=d59cddfd22844651995126a5d5d3d235, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:48.045715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e431a3-8fd1-4c26-afff-77e9b4b16144 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:48.091258 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1531/3035] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:03:47 2022] PUT /identity/v3/projects/8af81d3bf64548369ac9758cfee75c97/users/d59cddfd22844651995126a5d5d3d235/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:48.094405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:48.095369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:48.095915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:48.157956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] MFA Rules not processed for user `d59cddfd22844651995126a5d5d3d235`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:48.160041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:48.337361 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1505/3036] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:48.717898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145dafc-9b18-4f10-ae15-b7f5641a974f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:48.718255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145dafc-9b18-4f10-ae15-b7f5641a974f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:48.718595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145dafc-9b18-4f10-ae15-b7f5641a974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:48.743047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6145dafc-9b18-4f10-ae15-b7f5641a974f None None] MFA Rules not processed for user `dce01d3212934161a38be853b4cd93ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:48.744302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6145dafc-9b18-4f10-ae15-b7f5641a974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:48.819155 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1532/3037] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:51.241605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:51.244094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:51.297671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:51.362320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a609b-4e39-4e97-bc76-b50b5eeccd65 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:51.372298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d37efe-43c9-467c-819c-ca73ed74bfea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:51.373081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d37efe-43c9-467c-819c-ca73ed74bfea service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:51.373503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d37efe-43c9-467c-819c-ca73ed74bfea service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:51.374289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d37efe-43c9-467c-819c-ca73ed74bfea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:51.385520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d37efe-43c9-467c-819c-ca73ed74bfea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:51.387373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d37efe-43c9-467c-819c-ca73ed74bfea service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:51.400774 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1506/3038] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:55.597626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145dafc-9b18-4f10-ae15-b7f5641a974f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:55.679016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145dafc-9b18-4f10-ae15-b7f5641a974f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:55.683900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:55.684563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:55.685636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:55.685636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] PATH_INFO: `/v3/users/d59cddfd22844651995126a5d5d3d235` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:55.686925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d59cddfd22844651995126a5d5d3d235)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:55.689212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:55.752832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd59cddfd22844651995126a5d5d3d235'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:55.753595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd59cddfd22844651995126a5d5d3d235'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:55.763375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd59cddfd22844651995126a5d5d3d235'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:55.771819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd59cddfd22844651995126a5d5d3d235'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:55.772451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd59cddfd22844651995126a5d5d3d235'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:55.786166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:55.796277 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1533/3039] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:03:55 2022] DELETE /identity/v3/users/d59cddfd22844651995126a5d5d3d235 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:55.801854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:55.802412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:55.802550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:55.827221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:55.828805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:55.912462 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1507/3040] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:55.926447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:55.928577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:56.014647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1303a-b94a-44bc-91fc-6ae4d7de7899 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:56.018669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54ff49-3880-4e06-8a6b-be67abbd1784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:56.019638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ff49-3880-4e06-8a6b-be67abbd1784 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:56.019777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ff49-3880-4e06-8a6b-be67abbd1784 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:56.020012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ff49-3880-4e06-8a6b-be67abbd1784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:56.031183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54ff49-3880-4e06-8a6b-be67abbd1784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:56.032797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54ff49-3880-4e06-8a6b-be67abbd1784 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:56.044605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1534/3041] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:56.785938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:56.787605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:56.804719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff737-9573-49d0-8022-8aaa05871332 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:56.824340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:56.825508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:56.826125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:56.827185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] PATH_INFO: `/v3/projects/8af81d3bf64548369ac9758cfee75c97` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:56.831796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af81d3bf64548369ac9758cfee75c97)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:56.834616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:56.921925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af81d3bf64548369ac9758cfee75c97', 'request_id': 'req-078fc62c-d69f-4bc5-b7d7-e245f799d80a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:56.932359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af81d3bf64548369ac9758cfee75c97', 'request_id': 'req-078fc62c-d69f-4bc5-b7d7-e245f799d80a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:56.943195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af81d3bf64548369ac9758cfee75c97', 'request_id': 'req-078fc62c-d69f-4bc5-b7d7-e245f799d80a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:56.963833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] The token cache is being invalidate because project 8af81d3bf64548369ac9758cfee75c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:03:56.964647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:56.965959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-078fc62c-d69f-4bc5-b7d7-e245f799d80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:56.968309 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1508/3042] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:56 2022] DELETE /identity/v3/projects/8af81d3bf64548369ac9758cfee75c97 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:56.973309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fe1de-c03f-4d9a-a9a2-7540b5869afa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:56.973790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fe1de-c03f-4d9a-a9a2-7540b5869afa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:56.973992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fe1de-c03f-4d9a-a9a2-7540b5869afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.012052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7e9fe1de-c03f-4d9a-a9a2-7540b5869afa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:57.013712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e9fe1de-c03f-4d9a-a9a2-7540b5869afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:57.114942 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1535/3043] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:57.124159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed725-67fe-4bc3-9815-17f493fdc37c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.124506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed725-67fe-4bc3-9815-17f493fdc37c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.125011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed725-67fe-4bc3-9815-17f493fdc37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.149476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d33ed725-67fe-4bc3-9815-17f493fdc37c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:57.149476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d33ed725-67fe-4bc3-9815-17f493fdc37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:57.162613 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1509/3044] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:57.163425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fe1de-c03f-4d9a-a9a2-7540b5869afa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:57.175481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fe1de-c03f-4d9a-a9a2-7540b5869afa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:57.185341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:57.186529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.186923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.187325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.188330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:03:57.189659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:57.193194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:57.212526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1536/3045] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:03:57.217380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed725-67fe-4bc3-9815-17f493fdc37c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:57.227474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed725-67fe-4bc3-9815-17f493fdc37c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:57.232478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:57.233120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.233397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.233660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.234357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:57.236241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:57.267649 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1510/3046] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:03:57 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:57.272477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:57.284867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6da8-e6fc-4c61-b691-0ce135d22344 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:57.407064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:57.407780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.408140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.408734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.409425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:57.412009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:57.445866 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1537/3047] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:03:57 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:03:57.447346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:57.459948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f389b-e94b-4c50-88e0-d84df87888f8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:57.467200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:57.467943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.468345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.468624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.469340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:57.470965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:57.481859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:57.485596 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1511/3048] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:03:57.490585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:57.504226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9dd41-d328-45fa-aaa6-83c677ef5e46 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:57.513211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba714b-d33d-4333-a954-cf21175e6ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:57.514091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba714b-d33d-4333-a954-cf21175e6ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.514378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba714b-d33d-4333-a954-cf21175e6ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.514672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba714b-d33d-4333-a954-cf21175e6ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.515477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba714b-d33d-4333-a954-cf21175e6ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:57.517149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba714b-d33d-4333-a954-cf21175e6ea8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:57.532132 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1538/3049] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:03:57.536053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:57.549396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab421f0a-bfac-402f-971b-528d7fc03817 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:57.556877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:57.557536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.557886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.558170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] PATH_INFO: `/v3/projects/c568082c4b344fbdb179d91ffab1c5bb/users/b4fbf4eb1ec34c7d89f30efe61c0959a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.611426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c568082c4b344fbdb179d91ffab1c5bb, user_id=b4fbf4eb1ec34c7d89f30efe61c0959a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:57.617253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:57.639378 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1512/3050] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:03:57 2022] PUT /identity/v3/projects/c568082c4b344fbdb179d91ffab1c5bb/users/b4fbf4eb1ec34c7d89f30efe61c0959a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:57.651324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6013-23fa-4dce-a635-00206da0ee39 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.652000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6013-23fa-4dce-a635-00206da0ee39 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.652263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6013-23fa-4dce-a635-00206da0ee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.661594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:57.686204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-273e6013-23fa-4dce-a635-00206da0ee39 None None] MFA Rules not processed for user `b4fbf4eb1ec34c7d89f30efe61c0959a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:57.687629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-273e6013-23fa-4dce-a635-00206da0ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:57.762998 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1539/3051] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:57.775289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:57.777228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:57.871259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6a4ae-258d-4bad-bffa-b0d160b2ac03 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:57.877101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36df29-49c0-473d-bce6-6a8625ba3152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:57.877726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36df29-49c0-473d-bce6-6a8625ba3152 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:57.878133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36df29-49c0-473d-bce6-6a8625ba3152 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:57.878688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36df29-49c0-473d-bce6-6a8625ba3152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:57.892858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36df29-49c0-473d-bce6-6a8625ba3152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:57.894625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36df29-49c0-473d-bce6-6a8625ba3152 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:57.907507 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1513/3052] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:03:58.511439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e6013-23fa-4dce-a635-00206da0ee39 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:58.670681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e6013-23fa-4dce-a635-00206da0ee39 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:58.679409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:58.681452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:58.683839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:58.683839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] PATH_INFO: `/v3/users/b4fbf4eb1ec34c7d89f30efe61c0959a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:58.686053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fbf4eb1ec34c7d89f30efe61c0959a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:58.688860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:58.797937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fbf4eb1ec34c7d89f30efe61c0959a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:58.799772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fbf4eb1ec34c7d89f30efe61c0959a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:58.815108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fbf4eb1ec34c7d89f30efe61c0959a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:58.825330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fbf4eb1ec34c7d89f30efe61c0959a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:58.826324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fbf4eb1ec34c7d89f30efe61c0959a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:03:58.896768 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1540/3053] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:03:58 2022] DELETE /identity/v3/users/b4fbf4eb1ec34c7d89f30efe61c0959a => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:03:58.901789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:58.902350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:58.902486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:58.964228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:03:58.969112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:59.134203 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1514/3054] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:03:59.332344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:03:59.333953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:03:59.379592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:59.579279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d4f74-ef8f-4515-b0d8-375cfb6e1e0b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:03:59.598304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d398a1-9634-40f6-938f-afa95d2751e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:03:59.602419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d398a1-9634-40f6-938f-afa95d2751e0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:03:59.604585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d398a1-9634-40f6-938f-afa95d2751e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:03:59.605243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d398a1-9634-40f6-938f-afa95d2751e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:03:59.643133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d398a1-9634-40f6-938f-afa95d2751e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:03:59.645583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d398a1-9634-40f6-938f-afa95d2751e0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:03:59.664628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1541/3055] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:00.816575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:00.818614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:00.858061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472acc83-f1aa-4d0a-b828-62b9f6492065 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:00.891498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:00.893848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:00.894926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:00.895970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] PATH_INFO: `/v3/projects/c568082c4b344fbdb179d91ffab1c5bb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:00.898377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c568082c4b344fbdb179d91ffab1c5bb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:00.906080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:01.096784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c568082c4b344fbdb179d91ffab1c5bb', 'request_id': 'req-17ca966f-c5e8-495f-840c-0d54ae284146'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:01.106024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c568082c4b344fbdb179d91ffab1c5bb', 'request_id': 'req-17ca966f-c5e8-495f-840c-0d54ae284146'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:01.119516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:01.132550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c568082c4b344fbdb179d91ffab1c5bb', 'request_id': 'req-17ca966f-c5e8-495f-840c-0d54ae284146'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:01.150117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] The token cache is being invalidate because project c568082c4b344fbdb179d91ffab1c5bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:04:01.150433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:01.156857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17ca966f-c5e8-495f-840c-0d54ae284146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:01.158567 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1515/3056] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:00 2022] DELETE /identity/v3/projects/c568082c4b344fbdb179d91ffab1c5bb => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:01.165623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4159-51e4-44ec-981c-c13776c1e1b5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:01.166580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4159-51e4-44ec-981c-c13776c1e1b5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:01.167145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4159-51e4-44ec-981c-c13776c1e1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:01.252912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c05e4159-51e4-44ec-981c-c13776c1e1b5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:01.256431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c05e4159-51e4-44ec-981c-c13776c1e1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:01.405522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1542/3057] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:04:01.405724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ab403-5881-442a-81c2-a96a1484131d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:01.406020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ab403-5881-442a-81c2-a96a1484131d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:01.406313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ab403-5881-442a-81c2-a96a1484131d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:01.472637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-331ab403-5881-442a-81c2-a96a1484131d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:01.479250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-331ab403-5881-442a-81c2-a96a1484131d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:01.500630 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1516/3058] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:04:01.504975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4159-51e4-44ec-981c-c13776c1e1b5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:01.538528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4159-51e4-44ec-981c-c13776c1e1b5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:01.556782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:01.557629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:01.558079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:01.558525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:01.559357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:04:01.559816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:01.562189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:01.590319 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1543/3059] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:04:01.596135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ab403-5881-442a-81c2-a96a1484131d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:01.614611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ab403-5881-442a-81c2-a96a1484131d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:01.633563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:01.635768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:01.636053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:01.638520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:01.639222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:01.647118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:01.735922 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1517/3060] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:04:01 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:01.739756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:01.756209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7793b-a1e2-4ab6-90ee-e09bd9a0349b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:01.923048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:01.924539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:01.926045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:01.926502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:01.927376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:01.932306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:01.998459 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1544/3061] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:04:01 2022] POST /identity/v3/users => generated 291 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:02.003199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:02.041257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a1ff9-412b-4cad-9f26-dd806fe396bf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:02.055476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:02.061940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:02.062435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:02.062680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:02.067280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:02.080091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:02.111036 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1518/3062] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:02.116471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:02.139580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af68a03-1018-4d17-b8bb-1c424cca4386 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:02.156110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9e9ca-7a08-4180-ab95-8d5a435cf66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:02.156589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9e9ca-7a08-4180-ab95-8d5a435cf66d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:02.156911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9e9ca-7a08-4180-ab95-8d5a435cf66d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:02.157159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9e9ca-7a08-4180-ab95-8d5a435cf66d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:02.157815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9e9ca-7a08-4180-ab95-8d5a435cf66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:02.165209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9e9ca-7a08-4180-ab95-8d5a435cf66d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:02.213583 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1545/3063] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:02.217758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:02.241054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a071f7-d7ef-4fb9-a146-c1ff89c6e0b0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:02.249230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88713b98-5176-4eaf-a196-53c8189db3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:02.250247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88713b98-5176-4eaf-a196-53c8189db3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:02.250696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88713b98-5176-4eaf-a196-53c8189db3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:02.251138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88713b98-5176-4eaf-a196-53c8189db3ca admin admin] PATH_INFO: `/v3/projects/58c33b046dc7494c970f26eb5900b3f3/users/a10600a9277a436396fc70971dd27893/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:02.283046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88713b98-5176-4eaf-a196-53c8189db3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c33b046dc7494c970f26eb5900b3f3, user_id=a10600a9277a436396fc70971dd27893, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:02.289849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88713b98-5176-4eaf-a196-53c8189db3ca admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:02.348910 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1519/3064] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:04:02 2022] PUT /identity/v3/projects/58c33b046dc7494c970f26eb5900b3f3/users/a10600a9277a436396fc70971dd27893/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:02.351052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:02.354688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:02.355071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:02.403337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] MFA Rules not processed for user `a10600a9277a436396fc70971dd27893`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:02.406202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:02.431201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:02.524780 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1546/3065] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:02.544337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:02.544999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:02.545967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:02.624846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] MFA Rules not processed for user `a10600a9277a436396fc70971dd27893`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:02.629708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:02.659649 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1520/3066] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:02.900626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:02.902145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:03.018905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf7a8-3cd2-40bb-8f59-e6cabfe5dff3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:03.029662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:03.030306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:03.030636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:03.030973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:03.041028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:03.043002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:03.060899 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1547/3067] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:05.405949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:05.409843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:05.433387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bf934-a134-490d-97c7-ec3f93ce845d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:05.443207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:05.443966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:05.444369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:05.444785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:05.447706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:05.548258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:05.550041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:05.564384 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1521/3068] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:08.451012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:08.452478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:08.566021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:08.566021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:08.712760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc2d74-ca70-4ffb-a778-8c7f4b027439 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:08.723198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb436b51-013b-48e4-9080-8eb2e91c8763 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:08.736030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:08.737047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:08.737381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:08.737850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:08.747717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:08.755703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:08.758364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:08.760713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:08.761111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] PATH_INFO: `/v3/users/c7468cc8bc7d4dde8e2a5eba1fa42fd8` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:08.763789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7468cc8bc7d4dde8e2a5eba1fa42fd8)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:08.774102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:08.784894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:08.793013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:08.807425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:08.846788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1548/3069] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:08.911697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:08.912982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:08.939216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:08.955410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:08.978033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3c7ab4e6-d574-40fb-ab12-c7e81f2079d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:08.992023 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1522/3070] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:04:08 2022] DELETE /identity/v3/users/c7468cc8bc7d4dde8e2a5eba1fa42fd8 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:09.741811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:09.757878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b7c0d-e43b-44d7-878c-4a4ab0187fcb service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:09.764926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:09.955103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:09.955902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:09.956512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:09.960300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] PATH_INFO: `/v3/projects/de87d5984a9a4d4481ab4921133841d6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:09.962430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] RBAC: Authorizing `identity:delete_project(project_id=de87d5984a9a4d4481ab4921133841d6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:09.970331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:10.115762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de87d5984a9a4d4481ab4921133841d6', 'request_id': 'req-cf45f8f4-003d-4edd-b216-3f8602d008da'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:10.143789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de87d5984a9a4d4481ab4921133841d6', 'request_id': 'req-cf45f8f4-003d-4edd-b216-3f8602d008da'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:10.153276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de87d5984a9a4d4481ab4921133841d6', 'request_id': 'req-cf45f8f4-003d-4edd-b216-3f8602d008da'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:10.181150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] The token cache is being invalidate because project de87d5984a9a4d4481ab4921133841d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:04:10.182672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:10.184987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cf45f8f4-003d-4edd-b216-3f8602d008da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:10.188037 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1549/3071] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:09 2022] DELETE /identity/v3/projects/de87d5984a9a4d4481ab4921133841d6 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:10.193744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304a33-3f6d-4e69-8fdb-6f2f152f3c9b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.196015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304a33-3f6d-4e69-8fdb-6f2f152f3c9b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.197942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304a33-3f6d-4e69-8fdb-6f2f152f3c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:10.245854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-65304a33-3f6d-4e69-8fdb-6f2f152f3c9b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:10.247422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-65304a33-3f6d-4e69-8fdb-6f2f152f3c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:10.364622 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1523/3072] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:10.375052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298cd79-8085-4816-a561-c7cef0ea068d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.375052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298cd79-8085-4816-a561-c7cef0ea068d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.375658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298cd79-8085-4816-a561-c7cef0ea068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:10.406306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4298cd79-8085-4816-a561-c7cef0ea068d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:10.407788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4298cd79-8085-4816-a561-c7cef0ea068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:10.437100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1550/3073] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:10.441450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304a33-3f6d-4e69-8fdb-6f2f152f3c9b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:10.462012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304a33-3f6d-4e69-8fdb-6f2f152f3c9b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:10.470395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:10.471283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.471728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.472152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:10.473232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:04:10.473694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:10.475946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:10.500582 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1524/3074] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:04:10.505273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298cd79-8085-4816-a561-c7cef0ea068d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:10.519487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298cd79-8085-4816-a561-c7cef0ea068d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:10.527249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:10.528173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.528450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.528758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:10.529491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:10.531648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:10.579406 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1551/3075] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:04:10 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:10.585350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:10.597527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb4cdb-b4b9-4dc4-b70f-c6cbe088526f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:10.743916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:10.744851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.745513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.745748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:10.746581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:10.752582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:10.816370 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1525/3076] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:04:10 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:10.819493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:10.837323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da6509-3e8b-4516-880e-ee71d10a1b16 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:10.847126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:10.849390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.849985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.850422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:10.851333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:10.860418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:10.880276 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1552/3077] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:10.884587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:10.897186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:10.905084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617cdbe-9588-4bd6-b4f3-6f984bfe2d32 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:10.916711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f070a6a-6779-4fd2-b467-007c60ac7251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:10.917709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f070a6a-6779-4fd2-b467-007c60ac7251 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.918655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f070a6a-6779-4fd2-b467-007c60ac7251 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.918931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f070a6a-6779-4fd2-b467-007c60ac7251 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:10.919669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f070a6a-6779-4fd2-b467-007c60ac7251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:10.922274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f070a6a-6779-4fd2-b467-007c60ac7251 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:10.940104 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1526/3078] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:10.941676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:10.955052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8aebc-8440-48a6-bf57-c03aae1878c4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:10.985905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:10.989228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:10.989656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:10.990033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] PATH_INFO: `/v3/projects/b932820432be493b8a85520f58f49606/users/032de99964be4424ba019adf2eedf680/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:11.020562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b932820432be493b8a85520f58f49606, user_id=032de99964be4424ba019adf2eedf680, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:11.025028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:11.067737 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1553/3079] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:04:10 2022] PUT /identity/v3/projects/b932820432be493b8a85520f58f49606/users/032de99964be4424ba019adf2eedf680/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:04:11.080074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:11.081615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:11.081615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:11.147149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] MFA Rules not processed for user `032de99964be4424ba019adf2eedf680`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:11.148596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:11.270342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1527/3080] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:11.287608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:11.289562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:11.350980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:11.389259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fac19-dca8-4a24-81f6-78fdef2b37fa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:11.394912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:11.395909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:11.396306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:11.396646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:11.415860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:11.417492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:11.429978 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1554/3081] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:12.034044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:12.035779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:12.044640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:12.048133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:12.193073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a0137-a868-4d23-8ae9-0099c642577f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:12.197632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58173e3-3d99-4d6a-a260-341036f8d8f0 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:12.203510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:12.205472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0aec14-307c-4033-af94-8d901894ad5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:12.205472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aec14-307c-4033-af94-8d901894ad5e service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:12.205819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:12.205929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aec14-307c-4033-af94-8d901894ad5e service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:12.206039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:12.206159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aec14-307c-4033-af94-8d901894ad5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:12.206270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] PATH_INFO: `/v3/users/b7892230632b490c8f110206461a87c6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:12.207135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7892230632b490c8f110206461a87c6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:12.207991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe0aec14-307c-4033-af94-8d901894ad5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:12.209608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:12.231058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0aec14-307c-4033-af94-8d901894ad5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:12.233520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0aec14-307c-4033-af94-8d901894ad5e service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:12.253310 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1528/3082] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:12.310920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7892230632b490c8f110206461a87c6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:12.313852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7892230632b490c8f110206461a87c6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:12.333912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7892230632b490c8f110206461a87c6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:12.350250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7892230632b490c8f110206461a87c6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:12.351558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7892230632b490c8f110206461a87c6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:12.394429 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1555/3083] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:04:12 2022] DELETE /identity/v3/users/b7892230632b490c8f110206461a87c6 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:04:12.399260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149d2c-55ea-4d6d-b6f5-886d30ecced7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:12.400101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149d2c-55ea-4d6d-b6f5-886d30ecced7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:12.400644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149d2c-55ea-4d6d-b6f5-886d30ecced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:12.426750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7e149d2c-55ea-4d6d-b6f5-886d30ecced7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:12.429010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7e149d2c-55ea-4d6d-b6f5-886d30ecced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:12.524010 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1529/3084] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:12.572365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:12.575030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:12.672611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:12.674145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:12.674664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:12.703637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92af0ac-0196-492a-8038-f84b5a167d84 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:12.717247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:12.719265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:12.719575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:12.720018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:12.727079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:12.728952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:12.741865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:12.776117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:12.778075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:12.791133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:12.812291 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1556/3085] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:12.875419 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1530/3086] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:04:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:04:12.910217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:12.913535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:13.047794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7329d-68fd-4550-b6ce-bf65f8cc1afd service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:13.056124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:13.056978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:13.057438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:13.057813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:13.070800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:13.073514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:13.089024 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1557/3087] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:13.387839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:13.393518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:13.434699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf8334-ebe9-4bac-8910-7bf7bdabbdcf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:13.481628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:13.482305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:13.482908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:13.483376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] PATH_INFO: `/v3/projects/8b87f09e1bf645a4b71d8b31ee512ec0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:13.486166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b87f09e1bf645a4b71d8b31ee512ec0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:13.488555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:13.596780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b87f09e1bf645a4b71d8b31ee512ec0', 'request_id': 'req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.628993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b87f09e1bf645a4b71d8b31ee512ec0', 'request_id': 'req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.662137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b87f09e1bf645a4b71d8b31ee512ec0', 'request_id': 'req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.687393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] The token cache is being invalidate because project 8b87f09e1bf645a4b71d8b31ee512ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:04:13.687972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.689281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.691442 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1531/3088] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:13 2022] DELETE /identity/v3/projects/8b87f09e1bf645a4b71d8b31ee512ec0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:13.695297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:13.713996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605baccb-22c1-4c32-8519-a3ae6d74b1f9 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:13.831357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:13.833007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:13.833469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:13.833648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] PATH_INFO: `/v3/users/5fdeecb899e949f998041edf57ae4548` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:13.835065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fdeecb899e949f998041edf57ae4548)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:13.837652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:13.912907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fdeecb899e949f998041edf57ae4548'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.914046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fdeecb899e949f998041edf57ae4548'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.932158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fdeecb899e949f998041edf57ae4548'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.946883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fdeecb899e949f998041edf57ae4548'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.949051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3b25cd9f-28ea-413e-8f43-5ab0d5670474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fdeecb899e949f998041edf57ae4548'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:13.992240 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1558/3089] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:04:13 2022] DELETE /identity/v3/users/5fdeecb899e949f998041edf57ae4548 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:14.475201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:14.477101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:14.495731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c82d1-846d-4a4d-9be8-8a2c7fd91dd9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:14.581308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:14.582073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:14.582595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:14.583062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] PATH_INFO: `/v3/projects/6e1a0a1845644719b8c9a7c7fcbd6f2b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:14.584504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1a0a1845644719b8c9a7c7fcbd6f2b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:14.587547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:14.654887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:14.666544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1a0a1845644719b8c9a7c7fcbd6f2b', 'request_id': 'req-f2f3e42c-1442-405c-a9b2-e30f911b23be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:14.683120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1a0a1845644719b8c9a7c7fcbd6f2b', 'request_id': 'req-f2f3e42c-1442-405c-a9b2-e30f911b23be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:14.706238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1a0a1845644719b8c9a7c7fcbd6f2b', 'request_id': 'req-f2f3e42c-1442-405c-a9b2-e30f911b23be'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:14.725244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] The token cache is being invalidate because project 6e1a0a1845644719b8c9a7c7fcbd6f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:04:14.725977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:14.727394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f2f3e42c-1442-405c-a9b2-e30f911b23be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:14.735658 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1532/3090] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:14 2022] DELETE /identity/v3/projects/6e1a0a1845644719b8c9a7c7fcbd6f2b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:14.738178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7f0cd-0525-49d4-8f2e-f7b720419934 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:14.738992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7f0cd-0525-49d4-8f2e-f7b720419934 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:14.739423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7f0cd-0525-49d4-8f2e-f7b720419934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:14.762878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:14.778814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ffa7f0cd-0525-49d4-8f2e-f7b720419934 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:14.782049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ffa7f0cd-0525-49d4-8f2e-f7b720419934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:14.896067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1559/3091] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:14.978735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189834e-0198-40f1-aa3b-acf115b7a10a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:14.979018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189834e-0198-40f1-aa3b-acf115b7a10a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:14.979333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189834e-0198-40f1-aa3b-acf115b7a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.048219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5189834e-0198-40f1-aa3b-acf115b7a10a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:15.059131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5189834e-0198-40f1-aa3b-acf115b7a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:15.074110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1533/3092] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:15.079632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7f0cd-0525-49d4-8f2e-f7b720419934 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.105904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7f0cd-0525-49d4-8f2e-f7b720419934 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:15.112091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:15.112777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.113413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.113560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.113697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.113869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.113982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.115015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:04:15.115393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:15.117841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:15.138689 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1560/3093] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:04:15.143157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:15.145712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:15.152647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.171961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e449b-9411-4018-a9f2-d1d49c92b3a2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:15.188922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:15.190092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.190453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.191045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.191805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:15.194479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:15.247967 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1561/3094] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:04:15 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:15.249244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.259937 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1534/3095] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:04:15.278160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.280073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:15.372388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addab00-f29b-42e7-be65-8e1809d46523 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:15.380129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:15.381189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.381571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.381945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.382944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:15.385353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:15.418788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c124b-206b-40fa-ad62-51ff07b2624e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:15.420579 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1562/3096] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:04:15 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:15.422543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.439934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1dc51-9a97-48c9-8043-ffb090db62a7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:15.447680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:15.448693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.448998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.449823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.450782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:15.452774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:15.472104 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1563/3097] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:15.473622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.491005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5586-da37-4b1a-8539-f721da292920 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:15.502612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:15.504014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.504431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.505071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.505897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:15.507814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:15.547111 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1564/3098] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:15.558276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:15.559428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.559877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.560716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.561914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.587408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed325b-47b5-46b8-8017-88066fbc0965 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:15.592958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c44b7-f009-47fd-bd14-2503c687f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:15.595286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c44b7-f009-47fd-bd14-2503c687f8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.595286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c44b7-f009-47fd-bd14-2503c687f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.595286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c44b7-f009-47fd-bd14-2503c687f8f8 admin admin] PATH_INFO: `/v3/projects/63978eadb2a84ba7bada10574661830d/users/bff991c97d5a44969f435b4af0d71045/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.620902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:15.631089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c44b7-f009-47fd-bd14-2503c687f8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63978eadb2a84ba7bada10574661830d, user_id=bff991c97d5a44969f435b4af0d71045, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:15.635448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c44b7-f009-47fd-bd14-2503c687f8f8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:15.669522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1565/3099] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:04:15 2022] PUT /identity/v3/projects/63978eadb2a84ba7bada10574661830d/users/bff991c97d5a44969f435b4af0d71045/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:15.676221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:15.677162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:15.680479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:15.702735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:15.704488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:15.716620 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1535/3100] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:15.737896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] MFA Rules not processed for user `bff991c97d5a44969f435b4af0d71045`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:15.742264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:15.750074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:15.901842 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1566/3101] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:15.911955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:15.915951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:16.018785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71784abf-37c6-4a27-a4bc-cdf79277658d service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:16.136036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a51094-97c3-46e1-88c1-f5acfea68c56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:16.136865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a51094-97c3-46e1-88c1-f5acfea68c56 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:16.140344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a51094-97c3-46e1-88c1-f5acfea68c56 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:16.140344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a51094-97c3-46e1-88c1-f5acfea68c56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:16.148310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a51094-97c3-46e1-88c1-f5acfea68c56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:16.150849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a51094-97c3-46e1-88c1-f5acfea68c56 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:16.163054 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1536/3102] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:16.282592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:16.284630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:16.359385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c9261-485c-4954-b944-696a645fcbec None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:16.359706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c9261-485c-4954-b944-696a645fcbec None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:16.360002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c9261-485c-4954-b944-696a645fcbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:16.396500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9bbc4-32ca-4baa-b98a-012638075ae6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:16.397835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-728c9261-485c-4954-b944-696a645fcbec None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:16.399778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-728c9261-485c-4954-b944-696a645fcbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:16.402088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:16.402864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:16.403385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:16.403672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:16.405824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:16.420834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:16.422984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:16.435752 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1567/3103] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:16.518522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1537/3104] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:04:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:16.532050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:16.705843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed04f44-c636-4411-9ada-a438849a7e9d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:16.712548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:16.713671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:16.713769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:16.714041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:16.726194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:16.728833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:16.741189 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1568/3105] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:17.281338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c9261-485c-4954-b944-696a645fcbec None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:17.284353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-728c9261-485c-4954-b944-696a645fcbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:17.342917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:17.462785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:17.464652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:17.472646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c9261-485c-4954-b944-696a645fcbec None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:17.479418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad21f4-51d2-427e-8ffe-afbf83c518c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:17.480248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad21f4-51d2-427e-8ffe-afbf83c518c4 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:17.480557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad21f4-51d2-427e-8ffe-afbf83c518c4 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:17.481059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad21f4-51d2-427e-8ffe-afbf83c518c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:17.483887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1dad21f4-51d2-427e-8ffe-afbf83c518c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:17.500688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad21f4-51d2-427e-8ffe-afbf83c518c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:17.503093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad21f4-51d2-427e-8ffe-afbf83c518c4 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:17.517040 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1538/3106] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:17.532277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:17.609793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909727bb-ed3a-4a84-b83c-022e20dafc5c service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:17.623045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:17.625833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:17.626032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:17.626549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] PATH_INFO: `/v3/users/ac98fc0d0296493999906e8e3607c7f3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:17.630044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac98fc0d0296493999906e8e3607c7f3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:17.633110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:17.761105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac98fc0d0296493999906e8e3607c7f3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:17.763047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac98fc0d0296493999906e8e3607c7f3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:17.779650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac98fc0d0296493999906e8e3607c7f3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:17.792875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac98fc0d0296493999906e8e3607c7f3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:17.794921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac98fc0d0296493999906e8e3607c7f3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:17.827076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1569/3107] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:04:17 2022] DELETE /identity/v3/users/ac98fc0d0296493999906e8e3607c7f3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:04:18.043434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:18.043801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:18.044288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:18.103718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:18.105139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:18.231342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1539/3108] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:04:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:18.262049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:18.263503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:18.321722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:18.323673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:18.478505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea397c0-41bb-474a-b4f6-8ea113068326 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:18.487239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51322544-7cda-46ab-905c-f408a09b7609 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:18.488214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322544-7cda-46ab-905c-f408a09b7609 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:18.488350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322544-7cda-46ab-905c-f408a09b7609 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:18.489121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322544-7cda-46ab-905c-f408a09b7609 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:18.500378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51322544-7cda-46ab-905c-f408a09b7609 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:18.502543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51322544-7cda-46ab-905c-f408a09b7609 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:18.527930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28eda05-11f2-419d-8450-bb7dfcc826b0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:18.534219 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1570/3109] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:18.539464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:18.540348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:18.541412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:18.541692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] PATH_INFO: `/v3/projects/f48a592eb8fa4221a56eac1a50152c91` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:18.543021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48a592eb8fa4221a56eac1a50152c91)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:18.552636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:18.661058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48a592eb8fa4221a56eac1a50152c91', 'request_id': 'req-1e723d59-2ec7-4566-b608-f6a6f58e96d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:18.670888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48a592eb8fa4221a56eac1a50152c91', 'request_id': 'req-1e723d59-2ec7-4566-b608-f6a6f58e96d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:18.672970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:18.691994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48a592eb8fa4221a56eac1a50152c91', 'request_id': 'req-1e723d59-2ec7-4566-b608-f6a6f58e96d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:18.725067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] The token cache is being invalidate because project f48a592eb8fa4221a56eac1a50152c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:04:18.725371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:18.726713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1e723d59-2ec7-4566-b608-f6a6f58e96d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:18.759725 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1540/3110] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:18 2022] DELETE /identity/v3/projects/f48a592eb8fa4221a56eac1a50152c91 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:18.761688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72efe8-508a-447b-a6f4-7ec7e94a5917 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:18.762284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72efe8-508a-447b-a6f4-7ec7e94a5917 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:18.763092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72efe8-508a-447b-a6f4-7ec7e94a5917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:18.799961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ed72efe8-508a-447b-a6f4-7ec7e94a5917 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:18.801680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed72efe8-508a-447b-a6f4-7ec7e94a5917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:18.906111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:18.970201 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1571/3111] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:18.971469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb18c4-d41f-4640-940a-10b030f46063 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:18.971770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb18c4-d41f-4640-940a-10b030f46063 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:18.972319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb18c4-d41f-4640-940a-10b030f46063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.017641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-65eb18c4-d41f-4640-940a-10b030f46063 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:19.021230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-65eb18c4-d41f-4640-940a-10b030f46063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:19.060053 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1541/3112] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:04:19.066045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72efe8-508a-447b-a6f4-7ec7e94a5917 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:19.082302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72efe8-508a-447b-a6f4-7ec7e94a5917 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:19.102031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:19.103620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.103947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.104224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.104942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:04:19.105360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:19.107084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:19.140352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1572/3113] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:04:19.142121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756133b1-e31c-422b-ac5b-7698a9e5716d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.142542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756133b1-e31c-422b-ac5b-7698a9e5716d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.143059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756133b1-e31c-422b-ac5b-7698a9e5716d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.146939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:19.162946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23cd2-c53b-4c40-a7a3-3b7390e6e621 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:19.170964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:19.171732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.172166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.172535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.173418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:19.175954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:19.214722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-756133b1-e31c-422b-ac5b-7698a9e5716d None None] MFA Rules not processed for user `032de99964be4424ba019adf2eedf680`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:19.216768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-756133b1-e31c-422b-ac5b-7698a9e5716d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:19.249150 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1573/3114] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:04:19 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:19.253121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:19.356792 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1542/3115] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:04:19.375747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34902669-7564-420f-bdc2-da39a77280bd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:19.383382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:19.384189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.384631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.385228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.385918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:19.388407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:19.469611 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1574/3116] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:04:19 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:04:19.470358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756133b1-e31c-422b-ac5b-7698a9e5716d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:19.490579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756133b1-e31c-422b-ac5b-7698a9e5716d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:19.505276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:19.506002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.506345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.506623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.507293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:19.509212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:19.534162 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1543/3117] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:04:19.539066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:19.562565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98332cd1-765a-45dd-9e1a-f7b7a9dad713 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:19.578140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0ffe5-1160-4ba8-9498-72959009f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:19.582810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0ffe5-1160-4ba8-9498-72959009f7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.583355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0ffe5-1160-4ba8-9498-72959009f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.584078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0ffe5-1160-4ba8-9498-72959009f7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.586372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0ffe5-1160-4ba8-9498-72959009f7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:19.591933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0ffe5-1160-4ba8-9498-72959009f7e9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:19.626314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1575/3118] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:19.631667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:19.648918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a7605-6a8b-45e9-b626-f1c3e382773b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:19.683948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:19.685000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.685897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.686371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] PATH_INFO: `/v3/projects/3f33c209ffd1480e94175241ccc90414/users/297a11c331fb473291750ced9c843a6e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.722433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f33c209ffd1480e94175241ccc90414, user_id=297a11c331fb473291750ced9c843a6e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:19.730513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:19.760950 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1544/3119] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:04:19 2022] PUT /identity/v3/projects/3f33c209ffd1480e94175241ccc90414/users/297a11c331fb473291750ced9c843a6e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:04:19.762546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51af795-c9eb-41bf-bc83-a99a7597f499 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:19.764789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51af795-c9eb-41bf-bc83-a99a7597f499 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:19.765216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51af795-c9eb-41bf-bc83-a99a7597f499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:19.820013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c51af795-c9eb-41bf-bc83-a99a7597f499 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:19.824252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c51af795-c9eb-41bf-bc83-a99a7597f499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:19.982633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1576/3120] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:19.994152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:19.996025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:20.081747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba19-9356-4927-a634-6cb8c69ae5bb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:20.082529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba19-9356-4927-a634-6cb8c69ae5bb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:20.083066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba19-9356-4927-a634-6cb8c69ae5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:20.143439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-94c8ba19-9356-4927-a634-6cb8c69ae5bb None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:20.145908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80c8f8-6522-4586-8cb4-b316501ed4d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:20.147700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-94c8ba19-9356-4927-a634-6cb8c69ae5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:20.159759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:20.164644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:20.165405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:20.166230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:20.193460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:20.204604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:20.208308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:20.267001 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1545/3121] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:20.288113 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1577/3122] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:04:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:20.309436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:20.311452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:20.473073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:20.523493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9423ff-29a5-4e12-873f-09abe22d1cc1 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:20.541089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e4a47-fce6-4131-8009-d7f19c0e650d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:20.542254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4a47-fce6-4131-8009-d7f19c0e650d service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:20.542536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4a47-fce6-4131-8009-d7f19c0e650d service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:20.545702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4a47-fce6-4131-8009-d7f19c0e650d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:20.578921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4a47-fce6-4131-8009-d7f19c0e650d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:20.580552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4a47-fce6-4131-8009-d7f19c0e650d service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:20.615962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:20.616609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:20.616609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:20.617598 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1546/3123] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:20.692801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] MFA Rules not processed for user `032de99964be4424ba019adf2eedf680`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:20.698047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:20.816253 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1578/3124] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:22.222474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d19c3b-ff41-4b7b-b5c0-96ad0600fcbb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:22.223299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d19c3b-ff41-4b7b-b5c0-96ad0600fcbb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:22.226793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d19c3b-ff41-4b7b-b5c0-96ad0600fcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:22.349745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-90d19c3b-ff41-4b7b-b5c0-96ad0600fcbb None None] MFA Rules not processed for user `29d2f7b0e8274c769521d13930be9825`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:22.351012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90d19c3b-ff41-4b7b-b5c0-96ad0600fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:22.571677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1547/3125] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:28.020247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:28.022346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:28.269423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad7a35-fef1-4adb-be62-df1713713a13 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:28.288351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:28.289435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:28.290122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:28.291371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:28.298020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:28.336710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:28.339120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:28.361586 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1579/3126] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:04:29.046820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfa630-2d3a-4d51-b6fb-25b44c818eff None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:29.047573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfa630-2d3a-4d51-b6fb-25b44c818eff None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:29.047573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfa630-2d3a-4d51-b6fb-25b44c818eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:29.086491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-acbfa630-2d3a-4d51-b6fb-25b44c818eff None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:29.090449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-acbfa630-2d3a-4d51-b6fb-25b44c818eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:29.110864 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1548/3127] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:04:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:29.128511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:29.146564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68344108-fe74-4bab-8c63-6e444a3889d3 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:29.162552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b39b7-0514-4e1d-a453-2adeaf3044e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:29.163508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b39b7-0514-4e1d-a453-2adeaf3044e5 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:29.163943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b39b7-0514-4e1d-a453-2adeaf3044e5 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:29.164486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b39b7-0514-4e1d-a453-2adeaf3044e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:29.182685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b39b7-0514-4e1d-a453-2adeaf3044e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:29.184979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b39b7-0514-4e1d-a453-2adeaf3044e5 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:29.210960 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1580/3128] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:33.082145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:33.083831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:33.084362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:33.152509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] MFA Rules not processed for user `297a11c331fb473291750ced9c843a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:33.154243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:33.394968 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1549/3129] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:33.396215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f430515-107e-48fe-9392-b9452d6a7c24 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:33.396754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f430515-107e-48fe-9392-b9452d6a7c24 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:33.396872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f430515-107e-48fe-9392-b9452d6a7c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:33.445402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7f430515-107e-48fe-9392-b9452d6a7c24 None None] MFA Rules not processed for user `297a11c331fb473291750ced9c843a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:33.451347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7f430515-107e-48fe-9392-b9452d6a7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:33.481248 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1581/3130] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:43.378064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=70 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:43.423383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=79 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:44.162256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:44.164614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:44.191082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80017fe-cc4e-4e1a-ae9c-b9b7c2d05c04 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:44.201557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:44.203990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:44.205009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:44.206473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:44.228080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:44.231772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:44.249093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1550/3131] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:45.147448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:45.148357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:45.148875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:45.189792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:45.191700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:45.224321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1582/3132] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:04:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:45.255916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:45.279248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe970f3-59a7-4bc2-bafd-149f38b4dacd service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:45.295997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686821e-ba53-4241-8daf-d6860380dd27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:45.296645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686821e-ba53-4241-8daf-d6860380dd27 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:45.296948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686821e-ba53-4241-8daf-d6860380dd27 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:45.297207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686821e-ba53-4241-8daf-d6860380dd27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:45.328368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686821e-ba53-4241-8daf-d6860380dd27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:45.333020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686821e-ba53-4241-8daf-d6860380dd27 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:45.362419 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1551/3133] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:47.093297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:47.095130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:47.348516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0411b-122c-4902-9349-8d849017ddc1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:47.401577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:47.404620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:47.404931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:47.406360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:47.408966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:47.427398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:47.430310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:47.449748 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1583/3134] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:47.763960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f02ab-d7cb-4756-9ca0-58ccc7f4f662 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:47.765119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f02ab-d7cb-4756-9ca0-58ccc7f4f662 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:47.765756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f02ab-d7cb-4756-9ca0-58ccc7f4f662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:47.820601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-514f02ab-d7cb-4756-9ca0-58ccc7f4f662 None None] MFA Rules not processed for user `6eb3ccfcd9c344e1a7ad55f8398ec87a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:47.823196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-514f02ab-d7cb-4756-9ca0-58ccc7f4f662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:48.014604 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1552/3135] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:48.101599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:48.140407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef21a0-8fe3-4cd4-85b7-7251f5c850bf service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:48.147822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:48.149285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:48.149904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:48.150376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:48.152859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:48.233222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:48.235349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:48.249330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1584/3136] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:48.879141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:48.879665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:48.879665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:48.912745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:48.914261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:48.955868 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1553/3137] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:04:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:48.971380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:48.997948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ac36-04f7-4a62-8a1f-171c771e4df4 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:49.003537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0b178-f9bd-4a60-837f-f8321d2e8151 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:49.004393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b178-f9bd-4a60-837f-f8321d2e8151 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:49.004890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b178-f9bd-4a60-837f-f8321d2e8151 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:49.005329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0b178-f9bd-4a60-837f-f8321d2e8151 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:49.017357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0b178-f9bd-4a60-837f-f8321d2e8151 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:49.019283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0b178-f9bd-4a60-837f-f8321d2e8151 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:49.028755 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1585/3138] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:49.243947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:49.245893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:49.270583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef406e-478e-40ab-9ea9-b6c3ec099505 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:49.286512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:49.287410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:49.287858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:49.288309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] PATH_INFO: `/v3/users/bff991c97d5a44969f435b4af0d71045` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:49.290135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff991c97d5a44969f435b4af0d71045)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:49.294175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:49.384887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff991c97d5a44969f435b4af0d71045'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:49.385722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff991c97d5a44969f435b4af0d71045'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:49.395098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff991c97d5a44969f435b4af0d71045'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:49.417159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff991c97d5a44969f435b4af0d71045'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:49.494355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff991c97d5a44969f435b4af0d71045'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:49.507305 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1554/3139] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:04:49 2022] DELETE /identity/v3/users/bff991c97d5a44969f435b4af0d71045 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:49.508877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:49.509503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:49.509503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:49.548515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:49.549814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:49.695189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:49.748354 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1586/3140] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:04:49.802444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:49.804412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:49.914842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:49.944417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33adc6a-d2be-4d19-bc68-272ccee94c86 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:49.955446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf2fa7-0921-4e82-92bb-7889457fb551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:49.956722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf2fa7-0921-4e82-92bb-7889457fb551 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:49.957197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf2fa7-0921-4e82-92bb-7889457fb551 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:49.957644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf2fa7-0921-4e82-92bb-7889457fb551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:49.983517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf2fa7-0921-4e82-92bb-7889457fb551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:49.986523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf2fa7-0921-4e82-92bb-7889457fb551 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:50.001601 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1555/3141] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:04:51.875785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:51.877698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:51.897863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f1b71-3bfb-46ef-b045-b5655a804d69 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:51.915663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:51.916333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:51.916607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:51.917078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] PATH_INFO: `/v3/projects/63978eadb2a84ba7bada10574661830d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:51.918257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=63978eadb2a84ba7bada10574661830d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:51.920220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:52.128699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63978eadb2a84ba7bada10574661830d', 'request_id': 'req-877d193a-86b3-440a-ac8e-7c760d440b4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:52.156198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63978eadb2a84ba7bada10574661830d', 'request_id': 'req-877d193a-86b3-440a-ac8e-7c760d440b4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:52.169575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63978eadb2a84ba7bada10574661830d', 'request_id': 'req-877d193a-86b3-440a-ac8e-7c760d440b4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:52.192130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] The token cache is being invalidate because project 63978eadb2a84ba7bada10574661830d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:04:52.192876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:52.203719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-877d193a-86b3-440a-ac8e-7c760d440b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:04:52.206404 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1587/3142] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:51 2022] DELETE /identity/v3/projects/63978eadb2a84ba7bada10574661830d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:52.218387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679317-dffd-49ec-9407-95ffda6a3268 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:52.219563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679317-dffd-49ec-9407-95ffda6a3268 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:52.220214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679317-dffd-49ec-9407-95ffda6a3268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:52.273254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fa679317-dffd-49ec-9407-95ffda6a3268 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:52.274961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fa679317-dffd-49ec-9407-95ffda6a3268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:52.437991 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1556/3143] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:52.446642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cef9f-94e8-4028-b74d-669622cea515 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:52.447196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cef9f-94e8-4028-b74d-669622cea515 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:52.447669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cef9f-94e8-4028-b74d-669622cea515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:52.513661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-387cef9f-94e8-4028-b74d-669622cea515 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:52.515001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-387cef9f-94e8-4028-b74d-669622cea515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:52.563750 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1588/3144] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:52.568635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa679317-dffd-49ec-9407-95ffda6a3268 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:52.591815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa679317-dffd-49ec-9407-95ffda6a3268 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:52.621432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:52.622076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:52.622360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:52.622701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:52.623373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:04:52.623679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:52.625351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:52.664012 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1557/3145] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:04:52.664360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cef9f-94e8-4028-b74d-669622cea515 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:52.683329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cef9f-94e8-4028-b74d-669622cea515 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:52.703517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbb7cc-c38f-4989-9be5-128c24d63ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:52.704079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbb7cc-c38f-4989-9be5-128c24d63ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:52.704325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbb7cc-c38f-4989-9be5-128c24d63ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:52.704621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbb7cc-c38f-4989-9be5-128c24d63ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:52.705287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbb7cc-c38f-4989-9be5-128c24d63ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:52.707263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbb7cc-c38f-4989-9be5-128c24d63ec7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:52.756706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:52.757207 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1589/3146] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:04:52 2022] POST /identity/v3/projects => generated 379 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:04:52.826093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56cf3f-edd4-4a3f-b824-41090a073712 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:53.035442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:53.036011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:53.036550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:53.108975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:53.109812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:53.109961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:53.110155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:53.113728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:53.115548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:53.116858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:53.139277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:53.278074 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1558/3147] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:04:52 2022] POST /identity/v3/users => generated 290 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:04:53.302751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:53.342246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8f86e-2e1d-481e-b011-92b66a7d7681 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:53.353545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:53.354562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:53.355336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:53.356097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:53.357110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:53.359884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:53.377723 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1559/3148] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:04:53.381321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1590/3149] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:04:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:53.382266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:53.400005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa1e7-303a-4119-85ef-3f586d8907e2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:53.416208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:53.417554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:53.418027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:53.418900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:53.420717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:53.421856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:53.427379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:53.432042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:53.448103 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1560/3150] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:04:53.452500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:53.480642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696b031-68ff-4768-896c-657a426715d9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:53.498802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86457d1-3e21-46b2-b071-4cecdac45d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:53.499601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86457d1-3e21-46b2-b071-4cecdac45d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:53.500283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86457d1-3e21-46b2-b071-4cecdac45d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:53.500562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86457d1-3e21-46b2-b071-4cecdac45d2f admin admin] PATH_INFO: `/v3/projects/513655e40ccd4152a2e343e5415ce347/users/866c1a2a2ca8452eb0a66cbe9692ab75/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:53.554088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86457d1-3e21-46b2-b071-4cecdac45d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=513655e40ccd4152a2e343e5415ce347, user_id=866c1a2a2ca8452eb0a66cbe9692ab75, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:53.559072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86457d1-3e21-46b2-b071-4cecdac45d2f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:53.601247 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1561/3151] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:04:53 2022] PUT /identity/v3/projects/513655e40ccd4152a2e343e5415ce347/users/866c1a2a2ca8452eb0a66cbe9692ab75/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:04:53.603648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:53.603941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:53.604215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:53.643432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:53.673096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:04:53.675708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:53.703891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0adf81-9f72-492a-b1a5-7cded9b1ef9d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:53.729582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:53.868884 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1562/3152] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:04:53.881350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863f2e1-1a38-4fc9-acc4-f5e64f118b4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:53.882449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863f2e1-1a38-4fc9-acc4-f5e64f118b4d service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:53.882788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863f2e1-1a38-4fc9-acc4-f5e64f118b4d service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:53.883202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863f2e1-1a38-4fc9-acc4-f5e64f118b4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:54.031730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863f2e1-1a38-4fc9-acc4-f5e64f118b4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:54.034561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863f2e1-1a38-4fc9-acc4-f5e64f118b4d service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:54.040316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:04:54.041930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:04:54.050546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1591/3153] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:04:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:54.198131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a0ac4-24b1-46e2-9cda-b8c7a6e02c0a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:04:54.209115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:04:54.209614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:04:54.210021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:04:54.210221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:04:54.224176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:04:54.229593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:04:54.303790 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1563/3154] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:05:01.443270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:01.443840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:01.443840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:01.473507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:01.475216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:01.494456 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1592/3155] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:05:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:01.501589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:01.516808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90def5-7199-4dcd-ad8a-40f008c522ae service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:01.524289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:01.525401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:01.525998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:01.526507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:01.537298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:01.539428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:01.551239 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1564/3156] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:05:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:06.664164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:06.665947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:06.694131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1b3a3-9590-4fb1-a6b2-a94b44afb897 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:06.705433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:06.706592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:06.707080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:06.748409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] PATH_INFO: `/v3/users/06dc3376ada04ffe922c9023f4182259` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:06.748409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06dc3376ada04ffe922c9023f4182259)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:06.748409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:06.831206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06dc3376ada04ffe922c9023f4182259'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:06.832069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06dc3376ada04ffe922c9023f4182259'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:06.851045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06dc3376ada04ffe922c9023f4182259'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:06.865149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06dc3376ada04ffe922c9023f4182259'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:06.866222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06dc3376ada04ffe922c9023f4182259'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:06.891838 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1593/3157] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:06 2022] DELETE /identity/v3/users/06dc3376ada04ffe922c9023f4182259 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:07.285407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:07.287767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:07.556506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd887f-0174-4dd8-81e0-50920ce6fac5 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:07.662998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:07.682193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e279d9f-e673-4018-bfb8-b2a47f3aff77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:07.682926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e279d9f-e673-4018-bfb8-b2a47f3aff77 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:07.683210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e279d9f-e673-4018-bfb8-b2a47f3aff77 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:07.683640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e279d9f-e673-4018-bfb8-b2a47f3aff77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:07.689211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7e279d9f-e673-4018-bfb8-b2a47f3aff77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:07.869050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e279d9f-e673-4018-bfb8-b2a47f3aff77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:07.871498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e279d9f-e673-4018-bfb8-b2a47f3aff77 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:07.892627 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1565/3158] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:08.512171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:08.514711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:08.537498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:08.646998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f341894-fe1c-454d-905b-3f06d8aff405 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:08.661429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:08.662694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:08.663042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:08.663399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] PATH_INFO: `/v3/projects/183cb066566c4510945d6eb0d41a2853` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:08.673688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=183cb066566c4510945d6eb0d41a2853)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:08.675917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:08.801359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183cb066566c4510945d6eb0d41a2853', 'request_id': 'req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:08.817407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183cb066566c4510945d6eb0d41a2853', 'request_id': 'req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:08.827227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183cb066566c4510945d6eb0d41a2853', 'request_id': 'req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:08.871032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] The token cache is being invalidate because project 183cb066566c4510945d6eb0d41a2853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:05:08.871032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:08.872700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5d3d367-6a4b-42ea-9c3c-82ad7768b96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:08.880048 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1594/3159] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:08 2022] DELETE /identity/v3/projects/183cb066566c4510945d6eb0d41a2853 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:05:08.881965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a142c05-0cb6-4809-9f0d-21780d31269a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:08.882404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a142c05-0cb6-4809-9f0d-21780d31269a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:08.882502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a142c05-0cb6-4809-9f0d-21780d31269a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:08.929244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0a142c05-0cb6-4809-9f0d-21780d31269a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:08.930927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a142c05-0cb6-4809-9f0d-21780d31269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:09.109667 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1566/3160] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:09.123884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912770c0-160c-4568-b180-dd4b5bbbffdc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:09.124173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912770c0-160c-4568-b180-dd4b5bbbffdc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:09.124445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912770c0-160c-4568-b180-dd4b5bbbffdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:09.185032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-912770c0-160c-4568-b180-dd4b5bbbffdc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:09.201127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-912770c0-160c-4568-b180-dd4b5bbbffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:09.244433 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1595/3161] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:09.246221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a142c05-0cb6-4809-9f0d-21780d31269a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:09.257685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a142c05-0cb6-4809-9f0d-21780d31269a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:09.277448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:09.278342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:09.278898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:09.279412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:09.280233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:05:09.280683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:09.282416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:09.353363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912770c0-160c-4568-b180-dd4b5bbbffdc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:09.360573 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1567/3162] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:05:09.370418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912770c0-160c-4568-b180-dd4b5bbbffdc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:09.423458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:09.424943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:09.425446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:09.425907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:09.426876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:09.429906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:09.506060 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1596/3163] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:05:09 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:09.513165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:09.523809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adbc13-a56d-4c5c-8c08-5f345432b556 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:09.652713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:09.653361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:09.653649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:09.657150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:09.658033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:09.660018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:09.724083 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1568/3164] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:05:09 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:05:09.728507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:09.744124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c635e0c-ac2b-4677-98fa-ff915d0e7591 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:09.760903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:09.761811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:09.762087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:09.762514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:09.763527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:09.765318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:09.808965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:09.810427 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1597/3165] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:09.835304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a67d07-c25e-40b2-8fa1-ecab9016e5c2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:09.863696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c199da-af1c-41b4-bec1-1c616879d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:09.864245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c199da-af1c-41b4-bec1-1c616879d261 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:09.864527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c199da-af1c-41b4-bec1-1c616879d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:09.864829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c199da-af1c-41b4-bec1-1c616879d261 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:09.873035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c199da-af1c-41b4-bec1-1c616879d261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:09.874588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c199da-af1c-41b4-bec1-1c616879d261 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:09.890055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:09.920921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:09.923867 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1569/3166] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:05:09.935172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc6bfd-f60a-40f1-93cd-b72e27de883b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:09.958693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:09.959438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:09.959721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:09.959998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] PATH_INFO: `/v3/projects/6c681ff8fe354f97abb5fe0d5181e6f1/users/7f7c5adbac4f48b6bc141e0f888f2496/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:09.979338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c681ff8fe354f97abb5fe0d5181e6f1, user_id=7f7c5adbac4f48b6bc141e0f888f2496, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:09.984239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:10.008132 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1598/3167] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:05:09 2022] PUT /identity/v3/projects/6c681ff8fe354f97abb5fe0d5181e6f1/users/7f7c5adbac4f48b6bc141e0f888f2496/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:05:10.013150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd7d3-bb85-44b0-bf9c-f498645132c8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:10.013477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd7d3-bb85-44b0-bf9c-f498645132c8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:10.013758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd7d3-bb85-44b0-bf9c-f498645132c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:10.045166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8a3bd7d3-bb85-44b0-bf9c-f498645132c8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:10.047157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a3bd7d3-bb85-44b0-bf9c-f498645132c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:10.202812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1570/3168] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:10.231949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:10.233969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:10.325810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:10.388727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37979afe-9518-46ec-9c93-a6a011a0cf3d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:10.395530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:10.396474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:10.397026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:10.397425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:10.418602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:10.426671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:10.484849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1599/3169] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:11.088474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d0ec8-7440-48fa-956c-4d9531c5de09 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:11.089601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d0ec8-7440-48fa-956c-4d9531c5de09 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:11.090057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d0ec8-7440-48fa-956c-4d9531c5de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:11.196182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5b4d0ec8-7440-48fa-956c-4d9531c5de09 None None] MFA Rules not processed for user `866c1a2a2ca8452eb0a66cbe9692ab75`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:11.197683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5b4d0ec8-7440-48fa-956c-4d9531c5de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:11.432470 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1571/3170] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:11.640340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:11.643848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:11.876085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89936e93-c7dc-47fc-ae67-02e8efff071c service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:11.892833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:11.893780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:11.894153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:11.894520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:11.928602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:11.930475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:11.967710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1600/3171] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:12.225646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605b337-738e-47bc-a088-a55c072944db None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:12.225646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605b337-738e-47bc-a088-a55c072944db None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:12.226072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605b337-738e-47bc-a088-a55c072944db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:12.267758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c605b337-738e-47bc-a088-a55c072944db None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:12.269579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c605b337-738e-47bc-a088-a55c072944db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:12.450914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1572/3172] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:05:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:12.473608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:12.475617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:12.598703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d34b8-a731-4ae5-864a-f165228b3b3e service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:12.623676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585d3d6-a897-46d3-be1b-41e9af4a541e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:12.624639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d3d6-a897-46d3-be1b-41e9af4a541e service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:12.625134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d3d6-a897-46d3-be1b-41e9af4a541e service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:12.625589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d3d6-a897-46d3-be1b-41e9af4a541e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:12.643636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585d3d6-a897-46d3-be1b-41e9af4a541e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:12.646197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585d3d6-a897-46d3-be1b-41e9af4a541e service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:12.670080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1601/3173] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:05:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:22.432475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:22.665251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:22.895609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd9f65-44eb-4532-9a07-396ac8d7278c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:22.896066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd9f65-44eb-4532-9a07-396ac8d7278c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:22.896843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd9f65-44eb-4532-9a07-396ac8d7278c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:22.983643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-28cd9f65-44eb-4532-9a07-396ac8d7278c None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:22.985244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-28cd9f65-44eb-4532-9a07-396ac8d7278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:23.189254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:23.189527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:23.189797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:23.193894 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1573/3174] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:23.234680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:23.236363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:23.260874 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1602/3175] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:26.704824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4b43-c4f1-453c-a2e4-c392ef7cb7b5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:26.706296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4b43-c4f1-453c-a2e4-c392ef7cb7b5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:26.706712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4b43-c4f1-453c-a2e4-c392ef7cb7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:26.741718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0b1f4b43-c4f1-453c-a2e4-c392ef7cb7b5 None None] MFA Rules not processed for user `866c1a2a2ca8452eb0a66cbe9692ab75`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:26.743964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b1f4b43-c4f1-453c-a2e4-c392ef7cb7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:26.761150 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1574/3176] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:27.133691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:27.136159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:27.179541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf2439-a81d-4361-a5f6-5a863b6da6a5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:27.208598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:27.211122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:27.214820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:27.214820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:27.257388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:27.259442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:27.273650 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1603/3177] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:30.472298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccab23e-7d3f-4754-8367-e80d592d545b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:30.473450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccab23e-7d3f-4754-8367-e80d592d545b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:30.482415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccab23e-7d3f-4754-8367-e80d592d545b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:30.550578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0ccab23e-7d3f-4754-8367-e80d592d545b None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:30.551842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0ccab23e-7d3f-4754-8367-e80d592d545b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:30.570343 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1575/3178] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:05:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:30.613601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:30.631707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a31a-fb61-496a-a556-7599dd3dcae1 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:30.649165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:30.650436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:30.650716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:30.650979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:30.686033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:30.688302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:30.707613 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1604/3179] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:05:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:37.023801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccab23e-7d3f-4754-8367-e80d592d545b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:37.045095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccab23e-7d3f-4754-8367-e80d592d545b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:37.055486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fd894-f8b6-4ed9-b6c6-ed91cfb05ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:37.056234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fd894-f8b6-4ed9-b6c6-ed91cfb05ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:37.056736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fd894-f8b6-4ed9-b6c6-ed91cfb05ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:37.057172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fd894-f8b6-4ed9-b6c6-ed91cfb05ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:37.059358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b3fd894-f8b6-4ed9-b6c6-ed91cfb05ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:37.078734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fd894-f8b6-4ed9-b6c6-ed91cfb05ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:37.081927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fd894-f8b6-4ed9-b6c6-ed91cfb05ef4 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:37.105237 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1576/3180] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:37.753579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:37.755567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:37.774933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8b684-77a3-4762-b53b-edb52a30cb59 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:37.782973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:37.783641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:37.783920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:37.784196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] PATH_INFO: `/v3/users/a10600a9277a436396fc70971dd27893` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:37.785560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10600a9277a436396fc70971dd27893)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:37.787663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:37.856112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10600a9277a436396fc70971dd27893'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:37.857976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10600a9277a436396fc70971dd27893'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:37.876475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10600a9277a436396fc70971dd27893'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:37.890743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10600a9277a436396fc70971dd27893'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:37.891920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10600a9277a436396fc70971dd27893'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:37.931385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1605/3181] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:37 2022] DELETE /identity/v3/users/a10600a9277a436396fc70971dd27893 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:37.936649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:37.937442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:37.937855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:37.982545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:37.989169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:38.114304 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1577/3182] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:38.132229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:38.134740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:38.273126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47296d40-59ea-4b2c-96e6-48640d796580 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:38.280900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c9fdd-0aed-422d-8b84-15e244b306c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:38.283063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9fdd-0aed-422d-8b84-15e244b306c8 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:38.283518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9fdd-0aed-422d-8b84-15e244b306c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:38.283973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9fdd-0aed-422d-8b84-15e244b306c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:38.298973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c9fdd-0aed-422d-8b84-15e244b306c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:38.301566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c9fdd-0aed-422d-8b84-15e244b306c8 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:38.315294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1606/3183] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:38.825636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:38.827825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:38.848448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718c03-082f-4473-b90e-01e9de14dc2d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:38.857997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:38.858874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:38.859306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:38.859738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] PATH_INFO: `/v3/projects/58c33b046dc7494c970f26eb5900b3f3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:38.861145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c33b046dc7494c970f26eb5900b3f3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:38.863491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:38.940520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c33b046dc7494c970f26eb5900b3f3', 'request_id': 'req-5bc264be-2d97-41c2-8393-6501c10b8321'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:38.952614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c33b046dc7494c970f26eb5900b3f3', 'request_id': 'req-5bc264be-2d97-41c2-8393-6501c10b8321'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:38.960363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:38.970220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c33b046dc7494c970f26eb5900b3f3', 'request_id': 'req-5bc264be-2d97-41c2-8393-6501c10b8321'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:38.990030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] The token cache is being invalidate because project 58c33b046dc7494c970f26eb5900b3f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:05:38.991735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:38.993584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5bc264be-2d97-41c2-8393-6501c10b8321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:38.999503 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1578/3184] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:38 2022] DELETE /identity/v3/projects/58c33b046dc7494c970f26eb5900b3f3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:39.012333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25c32c-a04d-4203-b950-49068782c4fa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.013322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25c32c-a04d-4203-b950-49068782c4fa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.013804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25c32c-a04d-4203-b950-49068782c4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.050119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-be25c32c-a04d-4203-b950-49068782c4fa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:39.053488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-be25c32c-a04d-4203-b950-49068782c4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:39.135778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:39.187894 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1607/3185] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:39.195936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173cc6e-3614-4fca-9f79-2b9bb0fb9e15 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.196515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173cc6e-3614-4fca-9f79-2b9bb0fb9e15 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.196515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173cc6e-3614-4fca-9f79-2b9bb0fb9e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.234526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7173cc6e-3614-4fca-9f79-2b9bb0fb9e15 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:39.235919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7173cc6e-3614-4fca-9f79-2b9bb0fb9e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:39.251548 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1579/3186] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:05:39.252995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25c32c-a04d-4203-b950-49068782c4fa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:39.287210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25c32c-a04d-4203-b950-49068782c4fa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:39.293748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:39.294587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.294878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.295499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.296333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:05:39.296855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:39.298637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:39.317237 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1608/3187] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:05:39.319634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173cc6e-3614-4fca-9f79-2b9bb0fb9e15 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:39.332088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173cc6e-3614-4fca-9f79-2b9bb0fb9e15 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:39.339871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:39.340577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.341072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.341552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.342207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:39.344468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:39.399851 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1580/3188] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:05:39 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:39.400749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:39.416091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55fdb-151f-436c-8f5e-27a699db9211 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:39.520154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:39.520808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.521091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.521357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.522246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:39.525193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:39.572920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:39.582846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1609/3189] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:05:39 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:39.594755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c447e-ea55-455b-9192-b22a7a5601ac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:39.604907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:39.606307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.606792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.607254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.609600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:39.612911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:39.636847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1581/3190] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:39.638550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:39.655345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5265c-6ba1-48f0-8fb8-0e1c35c9db96 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:39.665431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6083d27-0337-484c-b6e3-cd80ecfb36b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:39.666064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6083d27-0337-484c-b6e3-cd80ecfb36b3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.666411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6083d27-0337-484c-b6e3-cd80ecfb36b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.667827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6083d27-0337-484c-b6e3-cd80ecfb36b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.668497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6083d27-0337-484c-b6e3-cd80ecfb36b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:39.670149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6083d27-0337-484c-b6e3-cd80ecfb36b3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:39.685793 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1610/3191] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:39.689809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:39.702787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893c6f3-3f55-4be5-a184-f0a20ba9e3f3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:39.715636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:39.716841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.717311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.718552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] PATH_INFO: `/v3/projects/daeb3231e3b54f95b8fe082ef62c330e/users/a27051677d2e492eb0ebe1287c69a9d5/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.741815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daeb3231e3b54f95b8fe082ef62c330e, user_id=a27051677d2e492eb0ebe1287c69a9d5, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:39.746302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:39.776795 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1582/3192] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:05:39 2022] PUT /identity/v3/projects/daeb3231e3b54f95b8fe082ef62c330e/users/a27051677d2e492eb0ebe1287c69a9d5/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:39.790179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259bc1f-cc4c-463b-b5d4-bebf946ed6c6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:39.790487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259bc1f-cc4c-463b-b5d4-bebf946ed6c6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:39.790724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259bc1f-cc4c-463b-b5d4-bebf946ed6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:39.838754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9259bc1f-cc4c-463b-b5d4-bebf946ed6c6 None None] MFA Rules not processed for user `a27051677d2e492eb0ebe1287c69a9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:39.840333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9259bc1f-cc4c-463b-b5d4-bebf946ed6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:40.013305 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1611/3193] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:40.047198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:40.049349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:40.166842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d4d8c-9d77-4d92-ba1b-6c254e0b4c90 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:40.174780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad2479-331d-4c83-bb9b-c891e717bac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:40.175441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad2479-331d-4c83-bb9b-c891e717bac0 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:40.175719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad2479-331d-4c83-bb9b-c891e717bac0 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:40.175984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad2479-331d-4c83-bb9b-c891e717bac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:40.185492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad2479-331d-4c83-bb9b-c891e717bac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:40.187532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad2479-331d-4c83-bb9b-c891e717bac0 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:40.199411 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1583/3194] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:40.891202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259bc1f-cc4c-463b-b5d4-bebf946ed6c6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:41.009306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:41.059701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259bc1f-cc4c-463b-b5d4-bebf946ed6c6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:41.066413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:41.067216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:41.067591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:41.067961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] PATH_INFO: `/v3/users/a27051677d2e492eb0ebe1287c69a9d5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:41.069703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a27051677d2e492eb0ebe1287c69a9d5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:41.071823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:41.150473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a27051677d2e492eb0ebe1287c69a9d5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:41.151423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a27051677d2e492eb0ebe1287c69a9d5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:41.163521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a27051677d2e492eb0ebe1287c69a9d5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:41.173535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a27051677d2e492eb0ebe1287c69a9d5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:41.174383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a27051677d2e492eb0ebe1287c69a9d5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:41.204558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1612/3195] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:40 2022] DELETE /identity/v3/users/a27051677d2e492eb0ebe1287c69a9d5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:41.206508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:41.207048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:41.207048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:41.230341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:41.231571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:41.316931 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1584/3196] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:41.333882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:41.336468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:41.463682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec4ae3-75d2-475e-b418-66d83093433d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:41.469170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76ff15-697f-49de-abca-7f6f8a9bd15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:41.469784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ff15-697f-49de-abca-7f6f8a9bd15a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:41.470061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ff15-697f-49de-abca-7f6f8a9bd15a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:41.470331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ff15-697f-49de-abca-7f6f8a9bd15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:41.482184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76ff15-697f-49de-abca-7f6f8a9bd15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:41.483645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76ff15-697f-49de-abca-7f6f8a9bd15a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:41.511212 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1613/3197] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:42.626914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:42.629593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:42.670828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af127fd1-fb75-4041-9989-7257d864ef79 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:42.692700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:42.696175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:42.697380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:42.699323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] PATH_INFO: `/v3/projects/daeb3231e3b54f95b8fe082ef62c330e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:42.702769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daeb3231e3b54f95b8fe082ef62c330e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:42.707017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:42.718712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:42.870613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daeb3231e3b54f95b8fe082ef62c330e', 'request_id': 'req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:42.885160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daeb3231e3b54f95b8fe082ef62c330e', 'request_id': 'req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:42.898853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daeb3231e3b54f95b8fe082ef62c330e', 'request_id': 'req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:42.916348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] The token cache is being invalidate because project daeb3231e3b54f95b8fe082ef62c330e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:05:42.917593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:42.918562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb5b5ad1-27ea-499e-bbbc-3b8d0ea6cbf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:42.921490 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1585/3198] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:42 2022] DELETE /identity/v3/projects/daeb3231e3b54f95b8fe082ef62c330e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:42.926895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42546-9e25-4ebb-82b8-aba154b2ed39 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:42.928227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42546-9e25-4ebb-82b8-aba154b2ed39 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:42.928887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42546-9e25-4ebb-82b8-aba154b2ed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:42.956236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f8d42546-9e25-4ebb-82b8-aba154b2ed39 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:42.957936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8d42546-9e25-4ebb-82b8-aba154b2ed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:43.041289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:43.093612 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1614/3199] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:05:43.102537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe867303-7ee0-4b78-82e2-2d2f09dda48c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.102978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe867303-7ee0-4b78-82e2-2d2f09dda48c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.103346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe867303-7ee0-4b78-82e2-2d2f09dda48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.152476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fe867303-7ee0-4b78-82e2-2d2f09dda48c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:43.154240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe867303-7ee0-4b78-82e2-2d2f09dda48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:43.172794 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1586/3200] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:43.174212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42546-9e25-4ebb-82b8-aba154b2ed39 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:43.186686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42546-9e25-4ebb-82b8-aba154b2ed39 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:43.202858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:43.203793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.204176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.205289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.206114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:05:43.206513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:43.213290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:43.240452 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1615/3201] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:05:43.245456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe867303-7ee0-4b78-82e2-2d2f09dda48c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:43.258450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe867303-7ee0-4b78-82e2-2d2f09dda48c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:43.275560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:43.276219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.279813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.280803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.281959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:43.284309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:43.346717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:43.347103 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1587/3202] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:05:43 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:43.363139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca00ba-b7b5-4fe3-b57d-fd3cf8dd71f6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:43.452777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:43.453692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.454273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.454760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.455688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:43.458129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:43.509274 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1616/3203] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:05:43 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:43.513551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:43.527497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0077d82-8a89-4649-aa48-bf773f037908 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:43.544132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:43.545055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.545334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.545608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.546276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:43.548621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:43.577735 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1588/3204] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:43.581581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:43.594798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224f97e-ced1-4c43-ae9b-d189f9e87340 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:43.605765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4d7e9-2329-4ff3-8bb5-80da6c6305db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:43.607187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4d7e9-2329-4ff3-8bb5-80da6c6305db admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.607574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4d7e9-2329-4ff3-8bb5-80da6c6305db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.607968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4d7e9-2329-4ff3-8bb5-80da6c6305db admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.609761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4d7e9-2329-4ff3-8bb5-80da6c6305db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:43.612018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4d7e9-2329-4ff3-8bb5-80da6c6305db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:43.631874 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1617/3205] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:43.637037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:43.697116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b0a3-263c-40d4-8ba0-77ad6dde8b96 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:43.709091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:43.711984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.714584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.715155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] PATH_INFO: `/v3/projects/092557b4387f406e851d6b09a9076193/users/0f4dad886747497094caf63cb33f56d6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.777799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] RBAC: Authorizing `identity:create_grant(project_id=092557b4387f406e851d6b09a9076193, user_id=0f4dad886747497094caf63cb33f56d6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:43.780609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:43.801981 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1589/3206] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:05:43 2022] PUT /identity/v3/projects/092557b4387f406e851d6b09a9076193/users/0f4dad886747497094caf63cb33f56d6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:43.813616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:43.814243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:43.814720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:43.872624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] MFA Rules not processed for user `0f4dad886747497094caf63cb33f56d6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:43.874102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:44.033501 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1618/3207] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:44.039733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:44.041959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:44.198567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e940-9612-4f03-9754-7e626379106c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:44.205681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:44.206548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:44.207133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:44.207598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:44.238375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:44.240012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:44.285761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1590/3208] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:05:45.586993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:45.588588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:45.721306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268d38-71cd-49d5-a26e-f4c0a85aa467 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:45.733136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa307dbb-da51-411c-8013-03867105a0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:45.738299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa307dbb-da51-411c-8013-03867105a0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:45.738913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa307dbb-da51-411c-8013-03867105a0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:45.739388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa307dbb-da51-411c-8013-03867105a0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:45.748414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fa307dbb-da51-411c-8013-03867105a0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:45.768447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:45.931036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa307dbb-da51-411c-8013-03867105a0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:45.932739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa307dbb-da51-411c-8013-03867105a0e5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:45.960979 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1619/3209] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:46.550865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:46.565131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d15c-2cca-4efc-9eb2-218eb43409a0 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:46.579713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:46.580509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:46.580873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:46.581216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] PATH_INFO: `/v3/users/0f4dad886747497094caf63cb33f56d6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:46.582700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4dad886747497094caf63cb33f56d6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:46.587894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:46.596979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:46.706339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4dad886747497094caf63cb33f56d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:46.708875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4dad886747497094caf63cb33f56d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:46.734944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4dad886747497094caf63cb33f56d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:46.761448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4dad886747497094caf63cb33f56d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:46.789990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4dad886747497094caf63cb33f56d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:46.808857 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1591/3210] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:46 2022] DELETE /identity/v3/users/0f4dad886747497094caf63cb33f56d6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:46.810298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:46.810769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:46.810864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:46.846823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:46.848390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:46.943229 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1620/3211] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:05:47.015387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:47.018513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:47.209331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557577d-ebc8-4fb9-9e4f-ce752878fa2c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:47.220022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:47.220993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:47.221435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:47.222988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:47.243916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:47.245478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:47.281244 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1592/3212] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:49.263283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:49.266131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:49.292538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40b286-f2b1-4321-b77c-33b85994a6ef None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:49.299911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:49.300755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:49.301177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:49.301902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] PATH_INFO: `/v3/projects/092557b4387f406e851d6b09a9076193` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:49.304827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=092557b4387f406e851d6b09a9076193)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:49.307354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:49.411442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '092557b4387f406e851d6b09a9076193', 'request_id': 'req-b6279964-257b-4bfa-a38e-4c549334ecd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.413261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:49.417778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:49.433814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '092557b4387f406e851d6b09a9076193', 'request_id': 'req-b6279964-257b-4bfa-a38e-4c549334ecd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.442693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '092557b4387f406e851d6b09a9076193', 'request_id': 'req-b6279964-257b-4bfa-a38e-4c549334ecd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.460154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] The token cache is being invalidate because project 092557b4387f406e851d6b09a9076193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:05:49.460857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.462574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6279964-257b-4bfa-a38e-4c549334ecd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.464395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1621/3213] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:49 2022] DELETE /identity/v3/projects/092557b4387f406e851d6b09a9076193 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:49.469797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60415aa8-7b1e-4177-b912-84fdecb7129d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:49.470262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce2b3-7701-4e78-bf71-71f809014439 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:49.470434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60415aa8-7b1e-4177-b912-84fdecb7129d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:49.470872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60415aa8-7b1e-4177-b912-84fdecb7129d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:49.529815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-60415aa8-7b1e-4177-b912-84fdecb7129d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:49.536704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-60415aa8-7b1e-4177-b912-84fdecb7129d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:49.622560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:49.660163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:49.662915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:49.663261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:49.663467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] PATH_INFO: `/v3/users/297a11c331fb473291750ced9c843a6e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:49.666351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=297a11c331fb473291750ced9c843a6e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:49.669793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:49.746948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1622/3214] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:49.756079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:49.760095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:49.760095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:49.787154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:49.840326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '297a11c331fb473291750ced9c843a6e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.850168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:49.851741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:49.859268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '297a11c331fb473291750ced9c843a6e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.888786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '297a11c331fb473291750ced9c843a6e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.896157 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1623/3215] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:49.897097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:49.902453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:49.916927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '297a11c331fb473291750ced9c843a6e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.925054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2b091-56db-4897-b2c9-b494ac65ff4a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:49.931897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:49.932798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:49.933780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:49.933780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] PATH_INFO: `/v3/users/6eb3ccfcd9c344e1a7ad55f8398ec87a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:49.934793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb3ccfcd9c344e1a7ad55f8398ec87a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:49.937629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:49.941658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '297a11c331fb473291750ced9c843a6e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:49.952233 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1593/3216] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:49 2022] DELETE /identity/v3/users/297a11c331fb473291750ced9c843a6e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:05:49.953432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.007915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb3ccfcd9c344e1a7ad55f8398ec87a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.008993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb3ccfcd9c344e1a7ad55f8398ec87a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.020094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb3ccfcd9c344e1a7ad55f8398ec87a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.033176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dc244-f31c-4864-be9d-11c422938059 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.039408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb3ccfcd9c344e1a7ad55f8398ec87a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.040434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb3ccfcd9c344e1a7ad55f8398ec87a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.040956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.041840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.041840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.042128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.042994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:05:50.043303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.044941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.061075 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1594/3217] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:05:50.063243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.065194 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1624/3218] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:49 2022] DELETE /identity/v3/users/6eb3ccfcd9c344e1a7ad55f8398ec87a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:50.066096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:50.080012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc309f4-8273-494b-af14-d89f05c3779c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.153881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.154721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.155074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.155413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.156120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.159811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.207262 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1595/3219] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:50.208284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.220048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea6c85-c6c0-48e3-8eeb-15eb8923710d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.288181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.290633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.291137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.291549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.292798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.296048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.336218 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1625/3220] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:05:50 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:50.340425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.361035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.366418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:50.368350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dad54-8cef-438c-acd1-fb4df109bb24 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.381186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.381825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.382592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.384115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.386468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.391468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.397184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4fe97-5a6e-4531-a896-fbc86960acae admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.417776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.417907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1596/3221] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:50.418336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.418336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.418336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] PATH_INFO: `/v3/projects/3f33c209ffd1480e94175241ccc90414` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.418336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f33c209ffd1480e94175241ccc90414)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.418336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.419284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.441249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab2ba-ddb5-4bf3-ba22-d89a162e7d4f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.441249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.441249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.441249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.441934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.442247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.444056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.446936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:50.467397 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1597/3222] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:50.469912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.487704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:50.493363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f33c209ffd1480e94175241ccc90414', 'request_id': 'req-c58799e9-f383-4e44-a73b-a15a41f40a35'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.493888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711f6fc-d574-4809-843c-7c6825f02652 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.532284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f33c209ffd1480e94175241ccc90414', 'request_id': 'req-c58799e9-f383-4e44-a73b-a15a41f40a35'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.546078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f33c209ffd1480e94175241ccc90414', 'request_id': 'req-c58799e9-f383-4e44-a73b-a15a41f40a35'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.574561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] The token cache is being invalidate because project 3f33c209ffd1480e94175241ccc90414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:05:50.574859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.575765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.581160 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1626/3223] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:50 2022] DELETE /identity/v3/projects/3f33c209ffd1480e94175241ccc90414 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:50.582176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.590487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:50.603145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54c936-2402-42cf-be06-0d787ba7327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.603145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54c936-2402-42cf-be06-0d787ba7327c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.603145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54c936-2402-42cf-be06-0d787ba7327c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.603526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54c936-2402-42cf-be06-0d787ba7327c admin admin] PATH_INFO: `/v3/projects/fedcdffc12b94b489ce433d5caa5e4be/users/430de2723f234fac8f09068195d8ce2c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.634106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54c936-2402-42cf-be06-0d787ba7327c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedcdffc12b94b489ce433d5caa5e4be, user_id=430de2723f234fac8f09068195d8ce2c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.641906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54c936-2402-42cf-be06-0d787ba7327c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.668619 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1598/3224] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:05:50 2022] PUT /identity/v3/projects/fedcdffc12b94b489ce433d5caa5e4be/users/430de2723f234fac8f09068195d8ce2c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:50.671404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d380-e0f7-4928-9297-2096520c817e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.672249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d380-e0f7-4928-9297-2096520c817e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.672913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d380-e0f7-4928-9297-2096520c817e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.705412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d089d380-e0f7-4928-9297-2096520c817e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:50.707276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d089d380-e0f7-4928-9297-2096520c817e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:50.760240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58799e9-f383-4e44-a73b-a15a41f40a35 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:50.854723 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1599/3225] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:50.856778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a3458-9d03-4679-bc9d-49e3739b5eb9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.857023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a3458-9d03-4679-bc9d-49e3739b5eb9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.857301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a3458-9d03-4679-bc9d-49e3739b5eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.883964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-635a3458-9d03-4679-bc9d-49e3739b5eb9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:50.885324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-635a3458-9d03-4679-bc9d-49e3739b5eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:50.887339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:50.888425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.888931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.889381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] PATH_INFO: `/v3/projects/e5a565f65b3849a99b74454791397ca5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.890857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5a565f65b3849a99b74454791397ca5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:50.892893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:50.897738 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1600/3226] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:50.900152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:50.900385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:50.900814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:50.930199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:50.932976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:50.946932 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1601/3227] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:50.947963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:50.951725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:50.980941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5a565f65b3849a99b74454791397ca5', 'request_id': 'req-71970c68-c99d-4fca-a118-ef80b3f07119'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.994183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5a565f65b3849a99b74454791397ca5', 'request_id': 'req-71970c68-c99d-4fca-a118-ef80b3f07119'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:50.997386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:51.006886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5a565f65b3849a99b74454791397ca5', 'request_id': 'req-71970c68-c99d-4fca-a118-ef80b3f07119'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:51.024571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] The token cache is being invalidate because project e5a565f65b3849a99b74454791397ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:05:51.025246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:51.026206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:51.029090 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1627/3228] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:50 2022] DELETE /identity/v3/projects/e5a565f65b3849a99b74454791397ca5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:51.036646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.039393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:51.084691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca51ef6-221c-4ebb-a8dc-bf753c5e3eec None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.138023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71970c68-c99d-4fca-a118-ef80b3f07119 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.152270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:51.153099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2f22e-9c69-4362-83d9-5063c2a2ff2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.153852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2f22e-9c69-4362-83d9-5063c2a2ff2b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.154244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2f22e-9c69-4362-83d9-5063c2a2ff2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.154760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2f22e-9c69-4362-83d9-5063c2a2ff2b admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.155853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2f22e-9c69-4362-83d9-5063c2a2ff2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:05:51.156261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2f22e-9c69-4362-83d9-5063c2a2ff2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.158519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2f22e-9c69-4362-83d9-5063c2a2ff2b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.176862 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1628/3229] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:05:51.178312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d6ce6-a792-4072-a332-dd5305d7b41f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.178603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d6ce6-a792-4072-a332-dd5305d7b41f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.178880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d6ce6-a792-4072-a332-dd5305d7b41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.205409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e742ac9-ccd5-45c9-b8c1-c5ab0828faa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.206947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e742ac9-ccd5-45c9-b8c1-c5ab0828faa0 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.207432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e742ac9-ccd5-45c9-b8c1-c5ab0828faa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.207852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e742ac9-ccd5-45c9-b8c1-c5ab0828faa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.210963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e742ac9-ccd5-45c9-b8c1-c5ab0828faa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:51.211924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-378d6ce6-a792-4072-a332-dd5305d7b41f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:51.213404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-378d6ce6-a792-4072-a332-dd5305d7b41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:51.227993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e742ac9-ccd5-45c9-b8c1-c5ab0828faa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.229848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e742ac9-ccd5-45c9-b8c1-c5ab0828faa0 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.242902 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1629/3230] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:51.242902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d6ce6-a792-4072-a332-dd5305d7b41f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.255701 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1602/3231] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:51.257904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbd11c-8781-4404-a717-f2b624c40c56 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.258317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbd11c-8781-4404-a717-f2b624c40c56 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.258634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbd11c-8781-4404-a717-f2b624c40c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.270501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d6ce6-a792-4072-a332-dd5305d7b41f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.284115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.284896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.285275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.285669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.286853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.292123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.292814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2acbd11c-8781-4404-a717-f2b624c40c56 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:51.293921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2acbd11c-8781-4404-a717-f2b624c40c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:51.316337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1603/3232] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:51.323471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbd11c-8781-4404-a717-f2b624c40c56 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.352987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbd11c-8781-4404-a717-f2b624c40c56 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.365631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.366316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.368064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.368345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.369310 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1630/3233] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:05:51 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:51.369889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:05:51.370199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.372561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.374110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.411304 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1604/3234] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:05:51.415843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.475118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8279377-52ab-4544-b5c0-4054e69a324e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.652753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b672fc-c297-4ea6-8fce-a8da3556d721 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.666502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.667402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.668223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.669706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.670766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.676149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.721172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.721805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.722139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.722354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.723008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.725413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.733595 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1631/3235] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:05:51 2022] POST /identity/v3/users => generated 292 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:05:51.734715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.748984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95a54-f35f-4b86-ba95-2eceef8b2310 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.754032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.754805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.755676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.756047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.757025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.759433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.771597 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1605/3236] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:05:51 2022] POST /identity/v3/projects => generated 399 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:05:51.776615 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1632/3237] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:51.776798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.777761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.795589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f51-f685-4b0e-a1e5-3124d24c774e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.833793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:51.906623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.910368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.910782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.911166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.913639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.915702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.929660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b8a1e-bebc-417e-afa4-fc10da5111b6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.934129 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1633/3238] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:51.936256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:51.938254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.938884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.939157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.939428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.940062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:51.942884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:51.957144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858109df-bf94-4ec9-b845-a89b60d84ed6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:51.968726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909858fe-f7ae-4e8c-91b4-d26b0076516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:51.970762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909858fe-f7ae-4e8c-91b4-d26b0076516e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:51.971720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909858fe-f7ae-4e8c-91b4-d26b0076516e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:51.972253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909858fe-f7ae-4e8c-91b4-d26b0076516e admin admin] PATH_INFO: `/v3/projects/027612746ecc44f4ab8e7af121ec27a1/users/92b0186cd92744dabd001b86560b5c90/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:51.977771 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1606/3239] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:05:51 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:05:51.983575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:52.002030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32624f-cbbd-466d-9e8d-eedfb47dac2a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:52.012016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909858fe-f7ae-4e8c-91b4-d26b0076516e admin admin] RBAC: Authorizing `identity:create_grant(project_id=027612746ecc44f4ab8e7af121ec27a1, user_id=92b0186cd92744dabd001b86560b5c90, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:52.015613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909858fe-f7ae-4e8c-91b4-d26b0076516e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:52.016333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:52.017005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:52.017275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:52.017544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:52.018206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:52.021717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:52.048940 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1634/3240] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:05:51 2022] PUT /identity/v3/projects/027612746ecc44f4ab8e7af121ec27a1/users/92b0186cd92744dabd001b86560b5c90/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:52.051066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:52.051454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:52.052408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:52.058331 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1607/3241] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:05:52.059974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:52.086454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849af5-ee56-4164-828d-ba18ca466783 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:52.102711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:52.106396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:52.139339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:52.258445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:52.259341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:52.259796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:52.260145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:52.260960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:52.262993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:52.281114 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1608/3242] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:05:52.283072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:52.293671 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1635/3243] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:52.297572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941621f-47e2-4093-b721-8199717e5654 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:52.302969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90f06f-136c-47dd-85a4-cfc148c7a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:52.303745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90f06f-136c-47dd-85a4-cfc148c7a968 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:52.304123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90f06f-136c-47dd-85a4-cfc148c7a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:52.304466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90f06f-136c-47dd-85a4-cfc148c7a968 admin admin] PATH_INFO: `/v3/projects/f8130333e2c04a3f960dc6465e07cc46/users/68d9297ca0884c2fa7f5283f722451ba/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:52.321837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90f06f-136c-47dd-85a4-cfc148c7a968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8130333e2c04a3f960dc6465e07cc46, user_id=68d9297ca0884c2fa7f5283f722451ba, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:52.325264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90f06f-136c-47dd-85a4-cfc148c7a968 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:52.335239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:52.338869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:52.352798 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1609/3244] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:05:52 2022] PUT /identity/v3/projects/f8130333e2c04a3f960dc6465e07cc46/users/68d9297ca0884c2fa7f5283f722451ba/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:52.355274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18cf4a-9960-48f8-9232-eea54436da19 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:52.355641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18cf4a-9960-48f8-9232-eea54436da19 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:52.356081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18cf4a-9960-48f8-9232-eea54436da19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:52.379582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7e18cf4a-9960-48f8-9232-eea54436da19 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:52.381192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7e18cf4a-9960-48f8-9232-eea54436da19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:52.545304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995e505-114c-4176-9181-1811a3133ec0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:52.589231 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1610/3245] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:52.672246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a2e32-fece-430e-8cb8-71bf37a2cac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:52.672932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a2e32-fece-430e-8cb8-71bf37a2cac1 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:52.673214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a2e32-fece-430e-8cb8-71bf37a2cac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:52.673487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a2e32-fece-430e-8cb8-71bf37a2cac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:52.677944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18cf4a-9960-48f8-9232-eea54436da19 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:52.689300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a2e32-fece-430e-8cb8-71bf37a2cac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:52.690519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18cf4a-9960-48f8-9232-eea54436da19 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:52.691024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a2e32-fece-430e-8cb8-71bf37a2cac1 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:52.696747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:52.697429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:52.698827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:52.699344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:52.699799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:52.704203 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1636/3246] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:52.729307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:52.735031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:52.784772 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1611/3247] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:54.009732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d33f99-1925-41bc-9764-cf366af440d6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:54.010236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d33f99-1925-41bc-9764-cf366af440d6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:54.010338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d33f99-1925-41bc-9764-cf366af440d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:54.072951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-47d33f99-1925-41bc-9764-cf366af440d6 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:05:54.079588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-47d33f99-1925-41bc-9764-cf366af440d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:54.226733 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1637/3248] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:05:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:05:54.241485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:54.244556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:54.352385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96a4b0-2c07-499b-b352-ea18972df451 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:54.357661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:54.358333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:54.358594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:54.359029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:54.385023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:54.386762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d33f99-1925-41bc-9764-cf366af440d6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:54.398981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d33f99-1925-41bc-9764-cf366af440d6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:54.400391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:54.413292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:54.415809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:54.416569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:54.417462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:54.423542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:54.426152 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1612/3249] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:05:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:54.456427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:54.458982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:54.474973 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1638/3250] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:56.323176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:56.325596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:56.358660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba33040-5dbb-416a-a8f6-f7ae18f5e659 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:56.373628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:56.374546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:56.375026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:56.375526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] PATH_INFO: `/v3/users/dce01d3212934161a38be853b4cd93ae` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:56.381606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce01d3212934161a38be853b4cd93ae)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:56.385326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:56.389842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:56.521299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce01d3212934161a38be853b4cd93ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:56.522169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce01d3212934161a38be853b4cd93ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:56.535095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce01d3212934161a38be853b4cd93ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:56.554294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce01d3212934161a38be853b4cd93ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:56.573385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce01d3212934161a38be853b4cd93ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:56.580941 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1613/3251] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:56 2022] DELETE /identity/v3/users/dce01d3212934161a38be853b4cd93ae => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:58.288698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:58.322240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc17280-18bd-4682-b338-76c3d463ebfa service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:58.489510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:58.490614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:58.491746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:58.492717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] PATH_INFO: `/v3/projects/ec0980b5824b425fbad9b8554cf5c42c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:58.494548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0980b5824b425fbad9b8554cf5c42c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:58.496776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:58.584394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0980b5824b425fbad9b8554cf5c42c', 'request_id': 'req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.597947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0980b5824b425fbad9b8554cf5c42c', 'request_id': 'req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.627824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0980b5824b425fbad9b8554cf5c42c', 'request_id': 'req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.643921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] The token cache is being invalidate because project ec0980b5824b425fbad9b8554cf5c42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:05:58.644459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.645504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.647861 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1639/3252] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:58 2022] DELETE /identity/v3/projects/ec0980b5824b425fbad9b8554cf5c42c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:58.651945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:58.656370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:58.705655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4cd34-6f8e-44a8-b66e-ab94c3dbf488 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:58.845543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:05:58.846241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:58.846524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:58.846939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] PATH_INFO: `/v3/users/0f8ac55ed39d4ef7bc080ec339c0aca9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:58.848242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f8ac55ed39d4ef7bc080ec339c0aca9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:05:58.850882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:05:58.955145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f8ac55ed39d4ef7bc080ec339c0aca9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.955885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f8ac55ed39d4ef7bc080ec339c0aca9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.972327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f8ac55ed39d4ef7bc080ec339c0aca9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:58.994781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f8ac55ed39d4ef7bc080ec339c0aca9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:59.023066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa87fcd-b572-4b63-9bba-4fea69e29303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f8ac55ed39d4ef7bc080ec339c0aca9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:05:59.024860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:59.041510 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1614/3253] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:05:58 2022] DELETE /identity/v3/users/0f8ac55ed39d4ef7bc080ec339c0aca9 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:05:59.766747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:05:59.770109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:05:59.835613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0d211-0e4e-42a2-9a2c-c2180bab8451 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:05:59.906049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:59.920993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04366a4f-1ccd-409a-9eab-9ea3da2dbff5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:05:59.922008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04366a4f-1ccd-409a-9eab-9ea3da2dbff5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:05:59.922481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04366a4f-1ccd-409a-9eab-9ea3da2dbff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:05:59.995933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-04366a4f-1ccd-409a-9eab-9ea3da2dbff5 None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:00.005381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04366a4f-1ccd-409a-9eab-9ea3da2dbff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:00.105723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:00.108066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:00.110414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:00.113189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] PATH_INFO: `/v3/projects/3edc67dee2b1455db651fa37a021e07e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:00.121912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3edc67dee2b1455db651fa37a021e07e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:00.135318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:00.273065 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1615/3254] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:05:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:00.356066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3edc67dee2b1455db651fa37a021e07e', 'request_id': 'req-da7e6e9e-c1b7-4384-b870-b720db0a1819'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:00.376573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3edc67dee2b1455db651fa37a021e07e', 'request_id': 'req-da7e6e9e-c1b7-4384-b870-b720db0a1819'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:00.393113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3edc67dee2b1455db651fa37a021e07e', 'request_id': 'req-da7e6e9e-c1b7-4384-b870-b720db0a1819'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:00.433147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] The token cache is being invalidate because project 3edc67dee2b1455db651fa37a021e07e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:06:00.433988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:00.436288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da7e6e9e-c1b7-4384-b870-b720db0a1819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:00.442403 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1640/3255] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:59 2022] DELETE /identity/v3/projects/3edc67dee2b1455db651fa37a021e07e => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:00.448350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eda54-0ac0-4227-852a-a3491e19d7ee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:00.448901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eda54-0ac0-4227-852a-a3491e19d7ee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:00.449271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eda54-0ac0-4227-852a-a3491e19d7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:00.491326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2d0eda54-0ac0-4227-852a-a3491e19d7ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:00.493327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2d0eda54-0ac0-4227-852a-a3491e19d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:00.662291 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1616/3256] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:00.663804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daddcd3-d3f9-4976-91bd-a136c6ef6eb2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:00.664753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daddcd3-d3f9-4976-91bd-a136c6ef6eb2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:00.664753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daddcd3-d3f9-4976-91bd-a136c6ef6eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:00.705134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4daddcd3-d3f9-4976-91bd-a136c6ef6eb2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:00.706915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4daddcd3-d3f9-4976-91bd-a136c6ef6eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:00.716712 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1641/3257] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:00.720679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0eda54-0ac0-4227-852a-a3491e19d7ee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:00.738288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0eda54-0ac0-4227-852a-a3491e19d7ee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:00.744374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:00.744374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:00.744798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:00.745760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:00.746596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:00.747006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:00.748801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:00.768949 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1617/3258] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:00.770143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daddcd3-d3f9-4976-91bd-a136c6ef6eb2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:00.782061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daddcd3-d3f9-4976-91bd-a136c6ef6eb2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:00.788816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:00.790706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:00.791043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:00.791667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:00.792929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:00.796885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:00.845676 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1642/3259] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:00 2022] POST /identity/v3/projects => generated 393 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:06:00.847398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:00.864143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02290d9-bb02-42cb-b62b-f4a10fe9331e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:01.011022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:01.013456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:01.013808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:01.014191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:01.015145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:01.020240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:01.078455 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1618/3260] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:06:00 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:01.079166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:01.095390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418faa9c-0150-447d-83c1-db8fc70d628d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:01.105777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:01.107296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:01.111166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:01.111646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:01.112761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:01.116126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:01.149142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1643/3261] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:01.153522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:01.169143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbbcce-740a-405d-bc45-f145c27f9273 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:01.177184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108aadc3-aa35-4566-8e5c-c03440d8cdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:01.177917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aadc3-aa35-4566-8e5c-c03440d8cdee admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:01.178272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aadc3-aa35-4566-8e5c-c03440d8cdee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:01.178647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aadc3-aa35-4566-8e5c-c03440d8cdee admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:01.179373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108aadc3-aa35-4566-8e5c-c03440d8cdee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:01.183696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108aadc3-aa35-4566-8e5c-c03440d8cdee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:01.212703 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1619/3262] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:06:01.214200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:01.228236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f14d8-d181-479b-b0e1-e32e82937541 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:01.250301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:01.251114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:01.251462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:01.251804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] PATH_INFO: `/v3/projects/7fb866af50c14a0393c03c1463330238/users/f7bf3532492e43719ce03e29f9740b5f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:01.293033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb866af50c14a0393c03c1463330238, user_id=f7bf3532492e43719ce03e29f9740b5f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:01.296624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:01.341311 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1644/3263] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:01 2022] PUT /identity/v3/projects/7fb866af50c14a0393c03c1463330238/users/f7bf3532492e43719ce03e29f9740b5f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:06:01.343420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01142-10dc-4d3b-87be-973e85ba2609 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:01.343706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01142-10dc-4d3b-87be-973e85ba2609 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:01.344009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01142-10dc-4d3b-87be-973e85ba2609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:01.383461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:01.462071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-22f01142-10dc-4d3b-87be-973e85ba2609 None None] MFA Rules not processed for user `f7bf3532492e43719ce03e29f9740b5f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:01.472553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-22f01142-10dc-4d3b-87be-973e85ba2609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:01.710890 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1620/3264] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:01.711254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:01.893328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:01.954169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fb554-14e1-4913-ad13-a87076470cd9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:01.978735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:01.982013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:01.985244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:01.986156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:01.989819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:01.996086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:02.105776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01142-10dc-4d3b-87be-973e85ba2609 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:02.107493 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1645/3265] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:01 2022] POST /identity/v3/projects => generated 393 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:02.118812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01142-10dc-4d3b-87be-973e85ba2609 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:02.297237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:02.297968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:02.298243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:02.298518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:02.305998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:02.308871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:02.396606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:02.424709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1621/3266] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:06:02 2022] POST /identity/v3/users => generated 303 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:02.431845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316474e-3064-4dd0-a553-eb28fb8d8584 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:02.446745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:02.447455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:02.447747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:02.448007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:02.448710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:02.450385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:02.515406 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1646/3267] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:02.516502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:02.529737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d578cee-b8a0-4749-87c0-ef6f24d8439c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:02.536482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:02.537425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:02.537711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:02.542767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] PATH_INFO: `/v3/projects/881e25ee5abe4c2c91f0edd74c934a9b/users/e6764e58f49944c48f8310960aa58d32/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:02.572839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] RBAC: Authorizing `identity:create_grant(project_id=881e25ee5abe4c2c91f0edd74c934a9b, user_id=e6764e58f49944c48f8310960aa58d32, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:02.575597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:02.641283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:02.641714 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1622/3268] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:02 2022] PUT /identity/v3/projects/881e25ee5abe4c2c91f0edd74c934a9b/users/e6764e58f49944c48f8310960aa58d32/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:02.657286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ef20b-5b05-4c15-ad76-82e5c5b3cc85 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:02.849497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cb280-4f48-4c7e-b06c-6920de17df0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:02.850141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cb280-4f48-4c7e-b06c-6920de17df0c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:02.852594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cb280-4f48-4c7e-b06c-6920de17df0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:02.853028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cb280-4f48-4c7e-b06c-6920de17df0c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:02.853721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cb280-4f48-4c7e-b06c-6920de17df0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:02.858183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cb280-4f48-4c7e-b06c-6920de17df0c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:02.893323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:02.905415 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1647/3269] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:06:02.907496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e61b8-54d4-4d32-a050-e22f57c662da admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:02.923600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65376e0-791d-4390-bda4-1f1148f9329b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:02.924627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65376e0-791d-4390-bda4-1f1148f9329b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:02.924900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65376e0-791d-4390-bda4-1f1148f9329b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:02.925180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65376e0-791d-4390-bda4-1f1148f9329b admin admin] PATH_INFO: `/v3/projects/881e25ee5abe4c2c91f0edd74c934a9b/users/e6764e58f49944c48f8310960aa58d32/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:02.932841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65376e0-791d-4390-bda4-1f1148f9329b admin admin] RBAC: Authorizing `identity:create_grant(project_id=881e25ee5abe4c2c91f0edd74c934a9b, user_id=e6764e58f49944c48f8310960aa58d32, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:02.935723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65376e0-791d-4390-bda4-1f1148f9329b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:02.968171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1623/3270] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:02 2022] PUT /identity/v3/projects/881e25ee5abe4c2c91f0edd74c934a9b/users/e6764e58f49944c48f8310960aa58d32/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:02.979446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:02.979897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:02.980298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:03.044826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] MFA Rules not processed for user `e6764e58f49944c48f8310960aa58d32`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:03.048804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:03.275127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da64ab2-6bc6-4aaf-ad58-b776bc7cf5d3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:03.276201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da64ab2-6bc6-4aaf-ad58-b776bc7cf5d3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:03.276201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da64ab2-6bc6-4aaf-ad58-b776bc7cf5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:03.280113 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1648/3271] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:03.309464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1da64ab2-6bc6-4aaf-ad58-b776bc7cf5d3 None None] MFA Rules not processed for user `f7bf3532492e43719ce03e29f9740b5f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:03.310726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1da64ab2-6bc6-4aaf-ad58-b776bc7cf5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:03.429142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:03.470589 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1624/3272] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:03.858956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:03.861368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:04.027895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:04.131004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840cb1f-442e-46c8-8568-122a6a1ea1a5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:04.144279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1ddb-481c-4699-967e-c658507b72c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:04.144977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1ddb-481c-4699-967e-c658507b72c2 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:04.145256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1ddb-481c-4699-967e-c658507b72c2 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:04.145525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1ddb-481c-4699-967e-c658507b72c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:04.167144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da1ddb-481c-4699-967e-c658507b72c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:04.168913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da1ddb-481c-4699-967e-c658507b72c2 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:04.202061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1649/3273] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:06:09.998614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da64ab2-6bc6-4aaf-ad58-b776bc7cf5d3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:10.014511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da64ab2-6bc6-4aaf-ad58-b776bc7cf5d3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:10.020460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e620277-7196-421b-b508-aec5c6f7d032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:10.021583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620277-7196-421b-b508-aec5c6f7d032 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:10.022057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620277-7196-421b-b508-aec5c6f7d032 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:10.022511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620277-7196-421b-b508-aec5c6f7d032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:10.033739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e620277-7196-421b-b508-aec5c6f7d032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:10.035688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e620277-7196-421b-b508-aec5c6f7d032 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:10.068683 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1625/3274] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:11.519309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c85cb7-476f-4d6f-9828-bff91bb87557 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:11.520396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c85cb7-476f-4d6f-9828-bff91bb87557 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:11.520941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c85cb7-476f-4d6f-9828-bff91bb87557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:11.618062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-81c85cb7-476f-4d6f-9828-bff91bb87557 None None] MFA Rules not processed for user `68d9297ca0884c2fa7f5283f722451ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:11.619308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-81c85cb7-476f-4d6f-9828-bff91bb87557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:11.867683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1650/3275] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:11.877946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:11.878514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:11.878514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:11.935215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] MFA Rules not processed for user `68d9297ca0884c2fa7f5283f722451ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:11.936598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:11.973675 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1626/3276] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:12.843998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:12.845225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:12.847050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:12.981701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] MFA Rules not processed for user `430de2723f234fac8f09068195d8ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:12.986307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:13.156526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1651/3277] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:13.200557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:13.201814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:13.361490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1ca78-728c-4913-8664-bdeaf7158a39 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:13.378062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec89b87-96e0-4295-9fea-c7c23108f86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:13.379150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec89b87-96e0-4295-9fea-c7c23108f86d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:13.385264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec89b87-96e0-4295-9fea-c7c23108f86d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:13.385842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec89b87-96e0-4295-9fea-c7c23108f86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:13.423244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec89b87-96e0-4295-9fea-c7c23108f86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:13.423244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec89b87-96e0-4295-9fea-c7c23108f86d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:13.454695 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1627/3278] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:13.557632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:13.559097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:13.592281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a36c65-44cc-4245-85eb-821fb8a1c88b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:13.613406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f65e3-0432-4419-9040-c7b27b58c25e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:13.614452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f65e3-0432-4419-9040-c7b27b58c25e service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:13.614961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f65e3-0432-4419-9040-c7b27b58c25e service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:13.615446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f65e3-0432-4419-9040-c7b27b58c25e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:13.629743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f65e3-0432-4419-9040-c7b27b58c25e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:13.631744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f65e3-0432-4419-9040-c7b27b58c25e service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:13.659683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1652/3279] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:13 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:20.281790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392065df-48b7-47dd-bf56-4a658cd60b57 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:20.281790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392065df-48b7-47dd-bf56-4a658cd60b57 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:20.281790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392065df-48b7-47dd-bf56-4a658cd60b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:20.300420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e7a29-fb9c-402d-b83b-9711069eb960 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:20.301023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e7a29-fb9c-402d-b83b-9711069eb960 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:20.305735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e7a29-fb9c-402d-b83b-9711069eb960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:20.346289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f41e7a29-fb9c-402d-b83b-9711069eb960 None None] MFA Rules not processed for user `430de2723f234fac8f09068195d8ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:20.347962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-392065df-48b7-47dd-bf56-4a658cd60b57 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:20.349727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-392065df-48b7-47dd-bf56-4a658cd60b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:20.351479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f41e7a29-fb9c-402d-b83b-9711069eb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:20.370463 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1653/3280] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:20.546089 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1628/3281] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:06:20.556948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b2e6d-8f6b-4cd9-a01e-65ea63366cd0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:20.557391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b2e6d-8f6b-4cd9-a01e-65ea63366cd0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:20.557770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b2e6d-8f6b-4cd9-a01e-65ea63366cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:20.628265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d43b2e6d-8f6b-4cd9-a01e-65ea63366cd0 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:20.655657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d43b2e6d-8f6b-4cd9-a01e-65ea63366cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:20.683830 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1654/3282] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:23.030972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392065df-48b7-47dd-bf56-4a658cd60b57 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:23.068219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392065df-48b7-47dd-bf56-4a658cd60b57 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:23.097931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3f228-c1c2-4b71-86e7-06fc16810e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:23.098675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f228-c1c2-4b71-86e7-06fc16810e2d service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:23.099055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f228-c1c2-4b71-86e7-06fc16810e2d service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:23.099557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f228-c1c2-4b71-86e7-06fc16810e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:23.112364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3f228-c1c2-4b71-86e7-06fc16810e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:23.129263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3f228-c1c2-4b71-86e7-06fc16810e2d service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:23.148963 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1629/3283] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:30.582214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:31.207801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b2e6d-8f6b-4cd9-a01e-65ea63366cd0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:31.238530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b2e6d-8f6b-4cd9-a01e-65ea63366cd0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:31.248114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:31.249330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:31.250465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:31.250926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:31.253571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:31.275075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:31.292893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:31.326826 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1655/3284] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:32.247640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564c82b-6bfc-42d4-9bb8-e1bc0d62d242 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:32.248149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564c82b-6bfc-42d4-9bb8-e1bc0d62d242 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:32.248273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564c82b-6bfc-42d4-9bb8-e1bc0d62d242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:32.270447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b564c82b-6bfc-42d4-9bb8-e1bc0d62d242 None None] MFA Rules not processed for user `866c1a2a2ca8452eb0a66cbe9692ab75`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:32.272159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b564c82b-6bfc-42d4-9bb8-e1bc0d62d242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:32.289560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:32.291020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:32.384228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9643d0f-9481-435a-8863-f5e8aba9b5a7 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:32.389855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:32.391295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:32.391531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:32.391957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:32.392376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] PATH_INFO: `/v3/users/29d2f7b0e8274c769521d13930be9825` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:32.394803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d2f7b0e8274c769521d13930be9825)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:32.397071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:32.409251 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1630/3285] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:32.479839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d2f7b0e8274c769521d13930be9825'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:32.481166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d2f7b0e8274c769521d13930be9825'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:32.495802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d2f7b0e8274c769521d13930be9825'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:32.512571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d2f7b0e8274c769521d13930be9825'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:32.513759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d2f7b0e8274c769521d13930be9825'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:32.545822 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1656/3286] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:06:32 2022] DELETE /identity/v3/users/29d2f7b0e8274c769521d13930be9825 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:33.136482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564c82b-6bfc-42d4-9bb8-e1bc0d62d242 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:33.165747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564c82b-6bfc-42d4-9bb8-e1bc0d62d242 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:33.183324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:33.185071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:33.242822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:33.243556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:33.243833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:33.244105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] PATH_INFO: `/v3/projects/2369f5ee467643948f618bbdee837521` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:33.245454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2369f5ee467643948f618bbdee837521)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:33.247429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:33.263472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70b0c5-6126-4f1a-af17-e7e9ff6f8dd7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:33.269630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:33.271329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:33.271822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:33.272289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:33.275053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:33.310986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:33.318674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2369f5ee467643948f618bbdee837521', 'request_id': 'req-36a11ea7-bbc0-4939-b27e-aacbbc31116e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.327459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2369f5ee467643948f618bbdee837521', 'request_id': 'req-36a11ea7-bbc0-4939-b27e-aacbbc31116e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.340810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2369f5ee467643948f618bbdee837521', 'request_id': 'req-36a11ea7-bbc0-4939-b27e-aacbbc31116e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.354295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] The token cache is being invalidate because project 2369f5ee467643948f618bbdee837521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:06:33.354579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.355405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.361690 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1631/3287] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:33 2022] DELETE /identity/v3/projects/2369f5ee467643948f618bbdee837521 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:33.362927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:33.365032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:33.426761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:33.428456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:33.441627 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1657/3288] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:33.463539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a11ea7-bbc0-4939-b27e-aacbbc31116e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:33.468968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:33.469623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:33.470012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:33.470231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] PATH_INFO: `/v3/users/f495627b28c44a979882d447f73d97fd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:33.471530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f495627b28c44a979882d447f73d97fd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:33.473565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:33.542520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:33.564031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f495627b28c44a979882d447f73d97fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.564795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f495627b28c44a979882d447f73d97fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.573251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f495627b28c44a979882d447f73d97fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.584244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f495627b28c44a979882d447f73d97fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.609716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24405f9c-dd63-4cad-bdf4-a7b46006280e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f495627b28c44a979882d447f73d97fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:33.617006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1632/3289] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:06:33 2022] DELETE /identity/v3/users/f495627b28c44a979882d447f73d97fd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:33.830212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:33.832284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:33.858780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c14e15-392c-4dcc-8988-f5d05c341adf service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:33.978878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:33.980081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:33.980458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:33.980831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] PATH_INFO: `/v3/projects/0ad373626b794b5482182e11b5d83183` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:33.982043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad373626b794b5482182e11b5d83183)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:33.984083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:34.078776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad373626b794b5482182e11b5d83183', 'request_id': 'req-c09325a5-b3ac-4245-931a-0612d5f954f1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:34.096067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad373626b794b5482182e11b5d83183', 'request_id': 'req-c09325a5-b3ac-4245-931a-0612d5f954f1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:34.104311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad373626b794b5482182e11b5d83183', 'request_id': 'req-c09325a5-b3ac-4245-931a-0612d5f954f1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:34.119879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] The token cache is being invalidate because project 0ad373626b794b5482182e11b5d83183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:06:34.120211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:34.120896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c09325a5-b3ac-4245-931a-0612d5f954f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:34.126101 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1658/3290] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:33 2022] DELETE /identity/v3/projects/0ad373626b794b5482182e11b5d83183 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:34.127626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9fd4-5f62-4136-aabd-8a8acd92053b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.127909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9fd4-5f62-4136-aabd-8a8acd92053b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.128181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9fd4-5f62-4136-aabd-8a8acd92053b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:34.169692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6bbb9fd4-5f62-4136-aabd-8a8acd92053b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:34.170914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6bbb9fd4-5f62-4136-aabd-8a8acd92053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:34.278131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb9dd-89af-4f07-882c-025a0ea0239e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.279075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb9dd-89af-4f07-882c-025a0ea0239e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.279358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb9dd-89af-4f07-882c-025a0ea0239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:34.288108 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1633/3291] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:34.307054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1ddcb9dd-89af-4f07-882c-025a0ea0239e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:34.308370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ddcb9dd-89af-4f07-882c-025a0ea0239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:34.332279 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1659/3292] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:34.336796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb9fd4-5f62-4136-aabd-8a8acd92053b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:34.354442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb9fd4-5f62-4136-aabd-8a8acd92053b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:34.368990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:34.369760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.370044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.370314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:34.370984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:34.371709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:34.373371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:34.428841 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1634/3293] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:34.433996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcb9dd-89af-4f07-882c-025a0ea0239e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:34.450112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcb9dd-89af-4f07-882c-025a0ea0239e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:34.472061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:34.473973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.476993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.477482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:34.478230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:34.484414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:34.548698 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1660/3294] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:34 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:34.550986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:34.564249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d585210-d4f0-4de3-8b4b-f397203cadfd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:34.689195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:34.689831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.690495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.690495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:34.691003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:34.692971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:34.756365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:34.757579 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1635/3295] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:06:34 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:34.782234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17e182-9f30-42c4-9676-73f03ca8e1cd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:34.798338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:34.799550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.799876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.803997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:34.805393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:34.811626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:34.817488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:34.852596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:34.859345 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1661/3296] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:34.871258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525b81b-c478-478a-be1d-dde9dffa7c0e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:34.885161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1209c3-fd64-4060-b982-80fde7bfe731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:34.887618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1209c3-fd64-4060-b982-80fde7bfe731 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.887618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1209c3-fd64-4060-b982-80fde7bfe731 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.887618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1209c3-fd64-4060-b982-80fde7bfe731 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:34.889142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1209c3-fd64-4060-b982-80fde7bfe731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:34.894588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1209c3-fd64-4060-b982-80fde7bfe731 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:34.947409 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1636/3297] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:34.948684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:34.969219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e15eb-bd42-44a5-ae21-0150016ab6d4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:34.977943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:34.978987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:34.979529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:34.980030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] PATH_INFO: `/v3/projects/81a16a9273f644b8b07e4ae5e4b03c60/users/3c4f75f592484d549a134998ca606cab/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:35.002147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a16a9273f644b8b07e4ae5e4b03c60, user_id=3c4f75f592484d549a134998ca606cab, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:35.005560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:35.043152 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1662/3298] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:34 2022] PUT /identity/v3/projects/81a16a9273f644b8b07e4ae5e4b03c60/users/3c4f75f592484d549a134998ca606cab/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:06:35.065269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a6d3e-3fb5-4e9d-96ff-49d16049c8cf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:35.065573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a6d3e-3fb5-4e9d-96ff-49d16049c8cf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:35.065848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a6d3e-3fb5-4e9d-96ff-49d16049c8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:35.159115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d36a6d3e-3fb5-4e9d-96ff-49d16049c8cf None None] MFA Rules not processed for user `3c4f75f592484d549a134998ca606cab`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:35.161249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d36a6d3e-3fb5-4e9d-96ff-49d16049c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:35.166908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:35.393501 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1637/3299] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:35.399053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:35.577861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca997b-b067-4569-b246-7b2d14aa3b2d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:35.589652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:35.590754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:35.591347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:35.591892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:35.593071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:35.596191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:35.672471 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1663/3300] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:35 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:35.675800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a6d3e-3fb5-4e9d-96ff-49d16049c8cf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:35.699231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a6d3e-3fb5-4e9d-96ff-49d16049c8cf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:35.853015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:35.853887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:35.854170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:35.854679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:35.855788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:35.860756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:35.898713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1638/3301] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:06:35 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:35.907145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:35.925850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb914f-8f33-44cc-ae64-acce0af31b5b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:35.948720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:35.949383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:35.950371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:35.950371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:35.950731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:35.952385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:35.975385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1664/3302] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:35.978891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:35.995183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc307-61c4-4225-881d-e5875454f201 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:36.013344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:36.014033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:36.014312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:36.014586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] PATH_INFO: `/v3/projects/b83c95d025dc4c0a8b645282e65e4c55/users/74d2bb9b1e8245a58966495a07247f06/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:36.049091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83c95d025dc4c0a8b645282e65e4c55, user_id=74d2bb9b1e8245a58966495a07247f06, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:36.052346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:36.081547 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1639/3303] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:35 2022] PUT /identity/v3/projects/b83c95d025dc4c0a8b645282e65e4c55/users/74d2bb9b1e8245a58966495a07247f06/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:06:36.085918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:36.105587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b96097-389a-493a-98ca-4fec087e804b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:36.168522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea885f9c-686f-4b27-a19a-b79788ba6e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:36.169075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea885f9c-686f-4b27-a19a-b79788ba6e32 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:36.169374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea885f9c-686f-4b27-a19a-b79788ba6e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:36.169652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea885f9c-686f-4b27-a19a-b79788ba6e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:36.170304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea885f9c-686f-4b27-a19a-b79788ba6e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:36.171893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea885f9c-686f-4b27-a19a-b79788ba6e32 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:36.183085 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1665/3304] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:36.187502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:36.199571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681b96-5919-42eb-836e-b01959d1af10 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:36.204809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:36.205503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:36.205779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:36.206075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] PATH_INFO: `/v3/projects/b83c95d025dc4c0a8b645282e65e4c55/users/74d2bb9b1e8245a58966495a07247f06/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:36.208008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83c95d025dc4c0a8b645282e65e4c55, user_id=74d2bb9b1e8245a58966495a07247f06, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:36.210685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:36.222498 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1640/3305] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:36 2022] PUT /identity/v3/projects/b83c95d025dc4c0a8b645282e65e4c55/users/74d2bb9b1e8245a58966495a07247f06/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:36.233507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d598a5-0aa9-43c0-a237-bd49992d8a45 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:36.233795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d598a5-0aa9-43c0-a237-bd49992d8a45 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:36.234066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d598a5-0aa9-43c0-a237-bd49992d8a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:36.264674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e8d598a5-0aa9-43c0-a237-bd49992d8a45 None None] MFA Rules not processed for user `74d2bb9b1e8245a58966495a07247f06`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:36.265942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e8d598a5-0aa9-43c0-a237-bd49992d8a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:36.302590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:36.360520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1666/3306] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:06:36.373593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:36.375713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:36.454814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:36.483669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae605-e3fe-472a-aa0b-195e784abd79 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:36.489205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc339954-4d57-4629-a722-53751aa245be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:36.489854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339954-4d57-4629-a722-53751aa245be service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:36.490164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339954-4d57-4629-a722-53751aa245be service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:36.490406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339954-4d57-4629-a722-53751aa245be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:36.502684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc339954-4d57-4629-a722-53751aa245be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:36.504627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc339954-4d57-4629-a722-53751aa245be service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:36.518010 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1641/3307] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:37.157846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d598a5-0aa9-43c0-a237-bd49992d8a45 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:37.256765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d598a5-0aa9-43c0-a237-bd49992d8a45 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:37.278129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:37.279050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:37.279506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:37.279914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] PATH_INFO: `/v3/users/3c4f75f592484d549a134998ca606cab` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:37.281364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4f75f592484d549a134998ca606cab)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:37.291407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:37.383985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4f75f592484d549a134998ca606cab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:37.385759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4f75f592484d549a134998ca606cab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:37.402681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4f75f592484d549a134998ca606cab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:37.416109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4f75f592484d549a134998ca606cab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:37.418434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4f75f592484d549a134998ca606cab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:37.458066 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1667/3308] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:06:37 2022] DELETE /identity/v3/users/3c4f75f592484d549a134998ca606cab => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:37.462543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:37.463033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:37.463316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:37.498282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:37.500092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:37.660069 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1642/3309] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:37.669037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:37.670572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:37.814317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:37.824809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cbf06-b266-470f-8492-f67f552fab12 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:37.829205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:37.832691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:37.833172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:37.833573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:37.844928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:37.846453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:37.859223 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1668/3310] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:38.455394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:38.457480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:38.479423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da165e4-f4f8-4baa-bb4b-a21495506eed None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:38.485749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:38.490235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:38.490834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:38.491256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] PATH_INFO: `/v3/projects/81a16a9273f644b8b07e4ae5e4b03c60` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:38.494732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a16a9273f644b8b07e4ae5e4b03c60)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:38.496895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:38.628928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a16a9273f644b8b07e4ae5e4b03c60', 'request_id': 'req-5a668b29-59b6-4e93-84af-e56858482b44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.646622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a16a9273f644b8b07e4ae5e4b03c60', 'request_id': 'req-5a668b29-59b6-4e93-84af-e56858482b44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.673418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a16a9273f644b8b07e4ae5e4b03c60', 'request_id': 'req-5a668b29-59b6-4e93-84af-e56858482b44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.694422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] The token cache is being invalidate because project 81a16a9273f644b8b07e4ae5e4b03c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:06:38.694943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.697414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.699606 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1643/3311] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:38 2022] DELETE /identity/v3/projects/81a16a9273f644b8b07e4ae5e4b03c60 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:38.713349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:38.728788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b099e-49ad-4835-a70d-15a9bede77fd service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:38.833515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:38.834096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:38.834378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:38.834639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] PATH_INFO: `/v3/users/74d2bb9b1e8245a58966495a07247f06` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:38.836165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d2bb9b1e8245a58966495a07247f06)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:38.838642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:38.906252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d2bb9b1e8245a58966495a07247f06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.907549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d2bb9b1e8245a58966495a07247f06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.924764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d2bb9b1e8245a58966495a07247f06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.954361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d2bb9b1e8245a58966495a07247f06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.955295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb96748-384f-41f3-a6df-b0461615c038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d2bb9b1e8245a58966495a07247f06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:38.978830 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1669/3312] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:06:38 2022] DELETE /identity/v3/users/74d2bb9b1e8245a58966495a07247f06 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:40.129777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:40.131574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:40.145976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668b29-59b6-4e93-84af-e56858482b44 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:40.230242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:40.230901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.231175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.231446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] PATH_INFO: `/v3/projects/b83c95d025dc4c0a8b645282e65e4c55` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.232618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83c95d025dc4c0a8b645282e65e4c55)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:40.236163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:40.245641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:40.318777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83c95d025dc4c0a8b645282e65e4c55', 'request_id': 'req-7d7a40f8-e782-41c5-9779-765343f46449'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:40.326485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83c95d025dc4c0a8b645282e65e4c55', 'request_id': 'req-7d7a40f8-e782-41c5-9779-765343f46449'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:40.349269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83c95d025dc4c0a8b645282e65e4c55', 'request_id': 'req-7d7a40f8-e782-41c5-9779-765343f46449'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:40.364003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] The token cache is being invalidate because project b83c95d025dc4c0a8b645282e65e4c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:06:40.364296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:40.365311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d7a40f8-e782-41c5-9779-765343f46449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:40.367608 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1644/3313] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:40 2022] DELETE /identity/v3/projects/b83c95d025dc4c0a8b645282e65e4c55 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:40.376189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2478-6736-4966-89d7-f06ef0bea6d7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.376797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2478-6736-4966-89d7-f06ef0bea6d7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.376909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2478-6736-4966-89d7-f06ef0bea6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.397972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-976a2478-6736-4966-89d7-f06ef0bea6d7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:40.399614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-976a2478-6736-4966-89d7-f06ef0bea6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:40.492866 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1670/3314] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:40.499073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146187-dc4b-4024-a92a-d2d53de6005f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.499493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146187-dc4b-4024-a92a-d2d53de6005f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.499864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146187-dc4b-4024-a92a-d2d53de6005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.534432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d3146187-dc4b-4024-a92a-d2d53de6005f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:40.535877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3146187-dc4b-4024-a92a-d2d53de6005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:40.549838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1645/3315] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:40.550016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2478-6736-4966-89d7-f06ef0bea6d7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:40.560989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2478-6736-4966-89d7-f06ef0bea6d7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:40.566466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:40.567128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.567409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.567683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.568349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:40.568690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:40.570557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:40.584757 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1671/3316] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:40.585625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:40.590820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3146187-dc4b-4024-a92a-d2d53de6005f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:40.602093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3146187-dc4b-4024-a92a-d2d53de6005f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:40.607746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:40.608790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.609210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.609917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.610714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:40.613791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:40.690086 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1646/3317] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:40 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:40.694208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:40.708271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bfe05-cd3c-4a0f-82a2-20f5350c2033 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:40.793364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:40.793985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.794541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.795030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.796068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:40.798373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:40.834835 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1672/3318] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:06:40 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:06:40.839385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:40.864821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c3a1-0d65-4bf3-bbdb-ebe3df1c273b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:40.886398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:40.887923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.888274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.888616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.889367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:40.893840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:40.912650 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1647/3319] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:40.914056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:40.928316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094041-b50f-42bc-b452-44c0e2d6c878 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:40.947405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6f4a9-2928-4501-a078-3e43534cc58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:40.948493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6f4a9-2928-4501-a078-3e43534cc58c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:40.948797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6f4a9-2928-4501-a078-3e43534cc58c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:40.949097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6f4a9-2928-4501-a078-3e43534cc58c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:40.949946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6f4a9-2928-4501-a078-3e43534cc58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:40.951628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6f4a9-2928-4501-a078-3e43534cc58c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:40.981772 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1673/3320] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:40.983961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:41.002856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d1e4f-2e74-4cbc-b113-01fcef8a6ef6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:41.017376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:41.018048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:41.018328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:41.019240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] PATH_INFO: `/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e/users/3d943b3d2c704d5b8b0780f25c72a4db/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:41.055571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59d3d57f7ee4b08a2fb841333c7212e, user_id=3d943b3d2c704d5b8b0780f25c72a4db, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:41.058999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:41.089748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1648/3321] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:40 2022] PUT /identity/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e/users/3d943b3d2c704d5b8b0780f25c72a4db/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:41.091912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9646-3fce-4510-97eb-bfbcddf2e3b0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:41.092302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9646-3fce-4510-97eb-bfbcddf2e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:41.092646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9646-3fce-4510-97eb-bfbcddf2e3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:41.143724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-544b9646-3fce-4510-97eb-bfbcddf2e3b0 None None] MFA Rules not processed for user `3d943b3d2c704d5b8b0780f25c72a4db`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:41.146058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-544b9646-3fce-4510-97eb-bfbcddf2e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:41.347476 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1674/3322] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:41.347772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:41.498465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e762-5e19-4d88-ac68-7e531f973737 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:41.505604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648fa7d-205c-4687-b138-e531362a2418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:41.506779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648fa7d-205c-4687-b138-e531362a2418 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:41.506779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648fa7d-205c-4687-b138-e531362a2418 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:41.506996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648fa7d-205c-4687-b138-e531362a2418 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:41.508195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648fa7d-205c-4687-b138-e531362a2418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:41.511791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648fa7d-205c-4687-b138-e531362a2418 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:41.574262 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1649/3323] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:41 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:06:41.578561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9646-3fce-4510-97eb-bfbcddf2e3b0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:41.592313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:41.593892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:41.594167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:41.597114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9646-3fce-4510-97eb-bfbcddf2e3b0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:41.673927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:41.734766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:41.735395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:41.736309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:41.736591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:41.737645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:41.740119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:41.747907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:41.755976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:41.773754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:41.779546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1675/3324] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:06:41 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:41.780496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:41.808407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49084f6-01e9-4b59-a7ad-17d485e0578b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:41.826044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:41.827929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:41.828972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:41.830068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:41.834049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:41.837142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:41.885051 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1676/3325] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:41.886631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:41.901419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8af35e-2a80-4d77-932e-0e58a2b5d5cc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:41.910652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:41.910864 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1650/3326] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:06:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:41.910964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:41.910964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:41.910964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] PATH_INFO: `/v3/projects/f7c5cef34c274936b644a11a27937823/users/a94ad86c3a5240dcb146d862d7064219/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:41.925237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:41.926894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:41.945420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c5cef34c274936b644a11a27937823, user_id=a94ad86c3a5240dcb146d862d7064219, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:41.951046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:41.977962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1677/3327] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:41 2022] PUT /identity/v3/projects/f7c5cef34c274936b644a11a27937823/users/a94ad86c3a5240dcb146d862d7064219/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:41.979195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:42.112939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be37ce4-a265-4c46-ab8d-6404ccf47e7c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:42.117778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9ee72-e063-4a66-9d04-0cb1ed7030ac None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:42.128781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:42.132077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:42.133057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:42.134859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:42.136857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:42.138683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:42.173949 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1678/3328] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:06:42.175398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:42.203673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bea1d-7bb2-4a8a-a319-9e68667dbb16 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:42.218797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1f620-8f1e-4e23-bc89-48ac0e590115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:42.220691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f620-8f1e-4e23-bc89-48ac0e590115 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:42.220973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f620-8f1e-4e23-bc89-48ac0e590115 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:42.221249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f620-8f1e-4e23-bc89-48ac0e590115 admin admin] PATH_INFO: `/v3/projects/f7c5cef34c274936b644a11a27937823/users/a94ad86c3a5240dcb146d862d7064219/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:42.227637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1f620-8f1e-4e23-bc89-48ac0e590115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c5cef34c274936b644a11a27937823, user_id=a94ad86c3a5240dcb146d862d7064219, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:42.240600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1f620-8f1e-4e23-bc89-48ac0e590115 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:42.285015 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1679/3329] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:06:42 2022] PUT /identity/v3/projects/f7c5cef34c274936b644a11a27937823/users/a94ad86c3a5240dcb146d862d7064219/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:42.286790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b671235-0763-4bab-93c1-5ef7ea72297c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:42.287081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b671235-0763-4bab-93c1-5ef7ea72297c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:42.287358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b671235-0763-4bab-93c1-5ef7ea72297c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:42.300114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:42.300803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:42.301090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:42.301358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:42.351921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:42.356343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1b671235-0763-4bab-93c1-5ef7ea72297c None None] MFA Rules not processed for user `a94ad86c3a5240dcb146d862d7064219`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:42.357764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1b671235-0763-4bab-93c1-5ef7ea72297c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:42.429306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:42.431619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:42.445453 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1651/3330] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:06:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:06:42.563531 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1680/3331] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:42.564986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:42.772700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4495a-d086-4e19-84c2-f5822aa59e2a service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:42.842101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f1b21-810c-4699-931c-32a5f6de351d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:42.843815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f1b21-810c-4699-931c-32a5f6de351d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:42.844103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f1b21-810c-4699-931c-32a5f6de351d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:42.844376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f1b21-810c-4699-931c-32a5f6de351d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:42.845147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f1b21-810c-4699-931c-32a5f6de351d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:42.845460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f1b21-810c-4699-931c-32a5f6de351d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:42.860368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f1b21-810c-4699-931c-32a5f6de351d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:42.890614 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1652/3332] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:42.891363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b671235-0763-4bab-93c1-5ef7ea72297c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:42.903664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b671235-0763-4bab-93c1-5ef7ea72297c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:42.907392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05f7a-58c3-4984-9de3-295b9b97c7d8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:42.907392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05f7a-58c3-4984-9de3-295b9b97c7d8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:42.907392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05f7a-58c3-4984-9de3-295b9b97c7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:42.932483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:42.934008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:42.934299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:42.934575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:42.937657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:42.942401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:42.962019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-24f05f7a-58c3-4984-9de3-295b9b97c7d8 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:42.963286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-24f05f7a-58c3-4984-9de3-295b9b97c7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:43.025131 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1681/3333] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:42 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:06:43.029907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:43.067317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:43.110915 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1653/3334] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:06:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:43.170979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f05f7a-58c3-4984-9de3-295b9b97c7d8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:43.187188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7c134-a750-4944-a0b9-1c6e37e4976a tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:43.199813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:43.200513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:43.200827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:43.201102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:43.201768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:43.202075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:43.203701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:43.254615 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1682/3335] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:43.314045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:43.364468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f05f7a-58c3-4984-9de3-295b9b97c7d8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:43.491853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3fd59-8cfa-440a-b8c3-6477472b12e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:43.492645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3fd59-8cfa-440a-b8c3-6477472b12e7 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:43.493017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3fd59-8cfa-440a-b8c3-6477472b12e7 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:43.493450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3fd59-8cfa-440a-b8c3-6477472b12e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:43.621852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:43.623561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:43.667417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3fd59-8cfa-440a-b8c3-6477472b12e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:43.669244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3fd59-8cfa-440a-b8c3-6477472b12e7 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:43.689017 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1654/3336] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:06:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:06:43.796113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417b012-45ca-4cbd-932a-86dc486c13d2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:43.803519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:43.804392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:43.804917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:43.806005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:43.817328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:43.823300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:43.838754 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1683/3337] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:06:43.857434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab5442-af9e-45a8-a5f1-1c781ed40090 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:43.857790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab5442-af9e-45a8-a5f1-1c781ed40090 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:43.857991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab5442-af9e-45a8-a5f1-1c781ed40090 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:43.859453 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1655/3338] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:06:43.868784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:43.884193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:43.895766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e72505-0827-4df9-b94f-1ca61f78dd71 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:43.902204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:43.905462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:43.906086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:43.906585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/86cb911098764324864c44a5d4ea9fcb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:43.908185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=86cb911098764324864c44a5d4ea9fcb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:43.912136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:43.927832 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1684/3339] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:43 2022] GET /identity/v3/projects/86cb911098764324864c44a5d4ea9fcb => generated 369 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:44.922804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8f3a8-3631-4dd9-b7f2-262376da7b67 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:44.922804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8f3a8-3631-4dd9-b7f2-262376da7b67 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:44.922804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8f3a8-3631-4dd9-b7f2-262376da7b67 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:44.923484 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1656/3340] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:06:44.931372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:44.934240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:45.021464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:45.082905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb372952-fdab-4725-bb1e-ca6940529ffb tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:45.114708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87628f7-958a-4691-8f40-2db8ef82f2f7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:45.116004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87628f7-958a-4691-8f40-2db8ef82f2f7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:45.116350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87628f7-958a-4691-8f40-2db8ef82f2f7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:45.116622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87628f7-958a-4691-8f40-2db8ef82f2f7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/86cb911098764324864c44a5d4ea9fcb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:45.126272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87628f7-958a-4691-8f40-2db8ef82f2f7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=86cb911098764324864c44a5d4ea9fcb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:45.134667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87628f7-958a-4691-8f40-2db8ef82f2f7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:45.140490 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1685/3341] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:44 2022] GET /identity/v3/projects/86cb911098764324864c44a5d4ea9fcb => generated 369 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:47.223203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:47.223846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:47.223846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:47.281656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:47.283160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:47.389655 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1657/3342] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:47.457022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa72aec-608c-4e12-b83d-d798ff5da14a None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:47.458081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa72aec-608c-4e12-b83d-d798ff5da14a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:47.458575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa72aec-608c-4e12-b83d-d798ff5da14a None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:47.460548 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1686/3343] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:06:47.466829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:47.485929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:47.498107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c0e6e-d7fe-4e11-92f5-7f814feb3005 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:47.503633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24918eb-2b3b-46ca-b81b-5dc4382aa288 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:47.504700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24918eb-2b3b-46ca-b81b-5dc4382aa288 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:47.505213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24918eb-2b3b-46ca-b81b-5dc4382aa288 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:47.505691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24918eb-2b3b-46ca-b81b-5dc4382aa288 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/86cb911098764324864c44a5d4ea9fcb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:47.507274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24918eb-2b3b-46ca-b81b-5dc4382aa288 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=86cb911098764324864c44a5d4ea9fcb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:47.509889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24918eb-2b3b-46ca-b81b-5dc4382aa288 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:47.526633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1658/3344] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:47 2022] GET /identity/v3/projects/86cb911098764324864c44a5d4ea9fcb => generated 369 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:47.542366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa72aec-608c-4e12-b83d-d798ff5da14a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:47.566678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa72aec-608c-4e12-b83d-d798ff5da14a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:47.580809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:47.581889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:47.582436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:47.582920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/86cb911098764324864c44a5d4ea9fcb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:47.585358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:delete_project(project_id=86cb911098764324864c44a5d4ea9fcb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:47.593052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:47.749977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86cb911098764324864c44a5d4ea9fcb', 'request_id': 'req-e4760f6b-2f68-491a-bda8-056e981f506e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:47.769094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86cb911098764324864c44a5d4ea9fcb', 'request_id': 'req-e4760f6b-2f68-491a-bda8-056e981f506e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:47.817733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:47.821539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86cb911098764324864c44a5d4ea9fcb', 'request_id': 'req-e4760f6b-2f68-491a-bda8-056e981f506e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:47.866081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] The token cache is being invalidate because project 86cb911098764324864c44a5d4ea9fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:06:47.867624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:47.868828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4760f6b-2f68-491a-bda8-056e981f506e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:06:47.870935 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1687/3345] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:47 2022] DELETE /identity/v3/projects/86cb911098764324864c44a5d4ea9fcb => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:06:48.529889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:48.530930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:48.531447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:48.597422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:06:48.598775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:48.672614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:48.822486 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1659/3346] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:06:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:06:55.016123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c5c8f-01b1-4b7b-b108-4b70e1ff0236 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:55.016775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c5c8f-01b1-4b7b-b108-4b70e1ff0236 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:55.016775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c5c8f-01b1-4b7b-b108-4b70e1ff0236 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:55.018453 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1688/3347] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:06:55.025461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:55.027152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:55.175169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:55.176567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:55.299655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a903d-3361-41ee-a0fb-2cbc543aaf52 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:55.311275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:55.312296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:55.312857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:55.313253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:55.315346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=d59d3d57f7ee4b08a2fb841333c7212e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:55.318825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:55.321973 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1660/3348] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:55 2022] GET /identity/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e => generated 375 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:06:55.345352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c5c8f-01b1-4b7b-b108-4b70e1ff0236 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:55.376412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c5c8f-01b1-4b7b-b108-4b70e1ff0236 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:55.384315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:55.386354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:55.387188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:55.387680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:55.388566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:55.389107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:55.391081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:55.408737 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1689/3349] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:55.415242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:55.429262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d84a2-dcb8-47d6-8ef9-919f17ce83a4 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:55.442777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:55.443180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:55.443458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:55.443730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:55.444365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:55.452561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:55.499471 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1661/3350] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:06:55 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:06:55.505823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:55.519712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d75db2-2ada-4243-9953-16559701c1d7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:55.599314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f61fd8-f4a8-44a2-ac90-22a7bee88ac5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:55.600861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f61fd8-f4a8-44a2-ac90-22a7bee88ac5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:55.601351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f61fd8-f4a8-44a2-ac90-22a7bee88ac5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:55.601865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f61fd8-f4a8-44a2-ac90-22a7bee88ac5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:55.602712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f61fd8-f4a8-44a2-ac90-22a7bee88ac5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:55.603179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f61fd8-f4a8-44a2-ac90-22a7bee88ac5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:55.605061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f61fd8-f4a8-44a2-ac90-22a7bee88ac5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:55.629085 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1690/3351] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:06:55.698092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:55.699483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:55.802904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:55.834557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36166a31-bcfb-4bdd-a36e-cad47804fbe6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:55.843845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:55.844593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:55.845065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:55.845402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:55.848843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:55.871271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:55.873277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:55.906527 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1662/3352] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:56.643411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e1d7-3311-4d76-8c0d-547ae79e5618 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:56.644002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e1d7-3311-4d76-8c0d-547ae79e5618 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:56.644002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e1d7-3311-4d76-8c0d-547ae79e5618 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:56.645361 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1691/3353] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:06:56.652322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:56.662352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:56.796092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:56.807251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71c11-7a9b-46c2-b0dd-ac2a38953de9 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:56.812803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:56.813914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:56.814284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:56.814649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/56ad5c69df974e3f9230cccf025c7f66` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:56.815879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=56ad5c69df974e3f9230cccf025c7f66)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:56.818060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:56.819855 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1663/3354] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:56 2022] GET /identity/v3/projects/56ad5c69df974e3f9230cccf025c7f66 => generated 369 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:57.985119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749917cf-09fc-4f2a-838e-7af5cba9455f None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:57.987552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749917cf-09fc-4f2a-838e-7af5cba9455f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:57.988682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749917cf-09fc-4f2a-838e-7af5cba9455f None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:57.990851 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1692/3355] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:06:57.998597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:58.017257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:58.060306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed8a3a-1544-4be6-b8c8-a1c6cdbfddb1 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:58.078777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:58.079631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:58.079911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:58.080179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/56ad5c69df974e3f9230cccf025c7f66` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:58.082068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=56ad5c69df974e3f9230cccf025c7f66)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:58.086371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:58.088828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1664/3356] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:57 2022] GET /identity/v3/projects/56ad5c69df974e3f9230cccf025c7f66 => generated 369 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:58.183614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749917cf-09fc-4f2a-838e-7af5cba9455f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:58.208922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749917cf-09fc-4f2a-838e-7af5cba9455f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:58.215273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:58.216164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:58.216597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:58.217051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:58.218513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:58.218964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:58.220957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:58.237841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1693/3357] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:58.242421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:58.264593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c8e6f-9d08-4d3b-9e35-4e2ff5bf66c6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:58.281451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:58.282083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:58.282363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:58.282635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:58.283288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:58.290662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:58.347452 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1665/3358] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:06:58 2022] POST /identity/v3/users => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:06:58.352748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:58.384300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd91b9d-1fc6-4973-bc3c-23b077f5a05c tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:58.403075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78257a0-5761-4c7a-8805-b5dbdbc76871 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:58.403865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78257a0-5761-4c7a-8805-b5dbdbc76871 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:58.404714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78257a0-5761-4c7a-8805-b5dbdbc76871 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:58.405218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78257a0-5761-4c7a-8805-b5dbdbc76871 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:58.406105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78257a0-5761-4c7a-8805-b5dbdbc76871 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:06:58.406620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78257a0-5761-4c7a-8805-b5dbdbc76871 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:58.408732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78257a0-5761-4c7a-8805-b5dbdbc76871 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:58.441935 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1694/3359] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:58.737888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:58.761289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8550-1a4d-413f-926b-78b167af9d48 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:58.793645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:58.794293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:58.794660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:58.794937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:58.800498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:58.824149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:58.824942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:58.849231 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1666/3360] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:06:59.059618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f789-6b57-4bf9-91ac-2102f60d947b None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:59.060219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f789-6b57-4bf9-91ac-2102f60d947b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:59.060719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f789-6b57-4bf9-91ac-2102f60d947b None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:59.064831 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1695/3361] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:06:59.073032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:59.086534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:59.101082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0af6-8c4c-4ab9-8a24-41e0d98349b8 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:59.108816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4b45f-a3ed-46a7-93f8-000d7cd19f2f tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:59.109490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4b45f-a3ed-46a7-93f8-000d7cd19f2f tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:59.109772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4b45f-a3ed-46a7-93f8-000d7cd19f2f tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:59.110245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4b45f-a3ed-46a7-93f8-000d7cd19f2f tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/56ad5c69df974e3f9230cccf025c7f66` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:59.112218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4b45f-a3ed-46a7-93f8-000d7cd19f2f tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=56ad5c69df974e3f9230cccf025c7f66)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:59.114388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4b45f-a3ed-46a7-93f8-000d7cd19f2f tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:59.123073 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1667/3362] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:59 2022] GET /identity/v3/projects/56ad5c69df974e3f9230cccf025c7f66 => generated 369 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:59.624939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f789-6b57-4bf9-91ac-2102f60d947b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:59.629151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1323f789-6b57-4bf9-91ac-2102f60d947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:59.711058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f789-6b57-4bf9-91ac-2102f60d947b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:59.716074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:59.716838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:59.717114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:59.717379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:59.719589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:59.731999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:59.733530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:59.734812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1a5e-c063-4f02-b211-8bd339583247 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:59.735289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1a5e-c063-4f02-b211-8bd339583247 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:59.735592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1a5e-c063-4f02-b211-8bd339583247 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:59.737266 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1668/3363] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 10:06:59.743215 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1696/3364] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:06:59.747852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c1a5e-c063-4f02-b211-8bd339583247 None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:06:59.763536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c1a5e-c063-4f02-b211-8bd339583247 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:59.773831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c1a5e-c063-4f02-b211-8bd339583247 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:59.778873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:59.779512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:59.779873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:59.780069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/56ad5c69df974e3f9230cccf025c7f66` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:59.781244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=56ad5c69df974e3f9230cccf025c7f66)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:59.783747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:59.791834 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1669/3365] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:06:59 2022] GET /identity/v3/projects/56ad5c69df974e3f9230cccf025c7f66 => generated 369 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:06:59.825857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:59.839829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1842e-fa03-40b9-9a63-a64bc26a36e0 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:06:59.847096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:06:59.847940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:06:59.848315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:06:59.848736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/users/8d2ea4e9f97e4bf3bc76ecc51cba3182` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:06:59.893714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8d2ea4e9f97e4bf3bc76ecc51cba3182)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:06:59.900007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:06:59.903770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:06:59.906179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:06:59.958127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:00.011631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2ea4e9f97e4bf3bc76ecc51cba3182'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.012537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2ea4e9f97e4bf3bc76ecc51cba3182'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.036946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2ea4e9f97e4bf3bc76ecc51cba3182'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.051230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2ea4e9f97e4bf3bc76ecc51cba3182'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.052195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2ea4e9f97e4bf3bc76ecc51cba3182'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.066578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374bb63-18c8-4633-9968-39f8029f45b7 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:00.072066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a2cae-ba41-4a53-a9a9-1249fd7b10f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:00.073269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a2cae-ba41-4a53-a9a9-1249fd7b10f9 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:00.075125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a2cae-ba41-4a53-a9a9-1249fd7b10f9 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:00.076543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a2cae-ba41-4a53-a9a9-1249fd7b10f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:00.081043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e45a2cae-ba41-4a53-a9a9-1249fd7b10f9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:00.094142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1697/3366] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:06:59 2022] DELETE /identity/v3/users/8d2ea4e9f97e4bf3bc76ecc51cba3182 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:00.095895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:00.097804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:00.115371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a2cae-ba41-4a53-a9a9-1249fd7b10f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:00.118041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a2cae-ba41-4a53-a9a9-1249fd7b10f9 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:00.126466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:00.135375 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1670/3367] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:06:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:00.221351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b50bce-3293-4994-99bb-61d8588d33e9 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:00.234133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:00.235093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:00.235531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:00.235947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/56ad5c69df974e3f9230cccf025c7f66` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:00.237330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:delete_project(project_id=56ad5c69df974e3f9230cccf025c7f66)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:00.240641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:00.336047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ad5c69df974e3f9230cccf025c7f66', 'request_id': 'req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.363567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ad5c69df974e3f9230cccf025c7f66', 'request_id': 'req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.380116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ad5c69df974e3f9230cccf025c7f66', 'request_id': 'req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.403956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] The token cache is being invalidate because project 56ad5c69df974e3f9230cccf025c7f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:00.404254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.404966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:00.406520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1698/3368] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:00 2022] DELETE /identity/v3/projects/56ad5c69df974e3f9230cccf025c7f66 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:01.019346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb774ac-e840-486b-bfeb-b47afd92dc87 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:01.019781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb774ac-e840-486b-bfeb-b47afd92dc87 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:01.020157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb774ac-e840-486b-bfeb-b47afd92dc87 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:01.021765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1671/3369] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:07:01.029325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:07:01.030771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:01.114626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:01.162971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:01.167650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:01.335270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ade1c1-9b49-43c8-bf85-fd4047ee5a97 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:01.340562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:01.341726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:01.342291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:01.342898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:01.346263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=d59d3d57f7ee4b08a2fb841333c7212e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:01.349618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:01.363711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1699/3370] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:07:01 2022] GET /identity/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e => generated 375 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:01.395361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce307d8c-bb23-44b2-ad11-124a8539ea13 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:01.396286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce307d8c-bb23-44b2-ad11-124a8539ea13 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:01.396863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce307d8c-bb23-44b2-ad11-124a8539ea13 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:01.399185 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1672/3371] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:07:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:07:01.406194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:07:01.408001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:01.433267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:01.444454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccfa34-c25d-4b32-a3ca-6c7fb5d9bcab tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:01.451086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:01.453218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:01.453685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:01.453959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:01.455159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=d59d3d57f7ee4b08a2fb841333c7212e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:01.458304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:01.460165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1700/3372] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:07:01 2022] GET /identity/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e => generated 375 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:02.329594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ead6b7-2526-4cf5-a525-579a37c646cb None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:02.330133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ead6b7-2526-4cf5-a525-579a37c646cb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:02.330585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ead6b7-2526-4cf5-a525-579a37c646cb None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:02.332271 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1673/3373] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:07:02.338287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:07:02.352432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:02.363330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0722d2-18e5-4ad6-a339-e4092ae598e0 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:02.368983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a94ad86c3a5240dcb146d862d7064219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7c5cef34c274936b644a11a27937823', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:02.369954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:02.370220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:02.370636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] PATH_INFO: `/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:02.372093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorizing `identity:get_project(project_id=d59d3d57f7ee4b08a2fb841333c7212e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:02.374320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:02.376823 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1701/3374] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:07:02 2022] GET /identity/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e => generated 375 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:02.418726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:02.419289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:02.419585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:02.445521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:02.449914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:02.537200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1674/3375] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:07:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:02.552361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:02.554140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:02.668272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00282d44-aa20-4255-b4b9-7cc1a556cdb3 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:02.674071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e3749-5094-4df7-8f0e-368adb70fc79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:02.674972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3749-5094-4df7-8f0e-368adb70fc79 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:02.675427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3749-5094-4df7-8f0e-368adb70fc79 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:02.675855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3749-5094-4df7-8f0e-368adb70fc79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:02.689620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e3749-5094-4df7-8f0e-368adb70fc79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:02.693393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e3749-5094-4df7-8f0e-368adb70fc79 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:02.708428 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1702/3376] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:03.013311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:03.015163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:03.210552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90d2a1-6526-4cef-8793-10c4a3c2feaf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:03.225061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:03.225776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:03.226371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:03.226912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] PATH_INFO: `/v3/users/3d943b3d2c704d5b8b0780f25c72a4db` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:03.229173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d943b3d2c704d5b8b0780f25c72a4db)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:03.231427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:03.368184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d943b3d2c704d5b8b0780f25c72a4db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:03.370833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d943b3d2c704d5b8b0780f25c72a4db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:03.401781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d943b3d2c704d5b8b0780f25c72a4db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:03.426653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d943b3d2c704d5b8b0780f25c72a4db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:03.477311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d943b3d2c704d5b8b0780f25c72a4db'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:03.497963 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1675/3377] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:07:03 2022] DELETE /identity/v3/users/3d943b3d2c704d5b8b0780f25c72a4db => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:03.502618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:03.503154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:03.503154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:03.542666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:03.546576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:03.673502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:03.702199 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1703/3378] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:03.741442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:03.743323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:03.812886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:03.954713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe17150-7740-49c0-97c1-2d96f17fe445 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:03.965649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:03.966974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:03.967498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:03.968671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:03.997392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:03.999095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:04.024899 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1676/3379] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:04.986644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:04.988905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:05.147758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43b1ef-1807-40e3-9d45-1edc89e0abfe None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:05.179269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba68be-3e8a-452c-b70e-573393b7c206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:05.180049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba68be-3e8a-452c-b70e-573393b7c206 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:05.181384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba68be-3e8a-452c-b70e-573393b7c206 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:05.181682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba68be-3e8a-452c-b70e-573393b7c206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:05.195845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b3ba68be-3e8a-452c-b70e-573393b7c206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:05.214034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba68be-3e8a-452c-b70e-573393b7c206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:05.216003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba68be-3e8a-452c-b70e-573393b7c206 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:05.250073 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1704/3380] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:05.853002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:05.855494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:05.883288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e5dbb-d4d0-499d-a260-7dc72f68cd51 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:05.898081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:05.901982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:05.905167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:05.905780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] PATH_INFO: `/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:05.909271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d59d3d57f7ee4b08a2fb841333c7212e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:05.916778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:06.049639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59d3d57f7ee4b08a2fb841333c7212e', 'request_id': 'req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.063116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59d3d57f7ee4b08a2fb841333c7212e', 'request_id': 'req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.085258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:06.087709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:06.088214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:06.093568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59d3d57f7ee4b08a2fb841333c7212e', 'request_id': 'req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.129188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:06.131317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:06.147635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] The token cache is being invalidate because project d59d3d57f7ee4b08a2fb841333c7212e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:06.147635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.148452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.151857 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1677/3381] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:05 2022] DELETE /identity/v3/projects/d59d3d57f7ee4b08a2fb841333c7212e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:06.160843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:06.166223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:06.275745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1705/3382] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:07:06.426427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeab6e1-d1bf-4b54-b9d4-490435c5ff60 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:06.440215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:06.443551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:06.444264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:06.444824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] PATH_INFO: `/v3/users/a94ad86c3a5240dcb146d862d7064219` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:06.448183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a94ad86c3a5240dcb146d862d7064219)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:06.454508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:06.624840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a94ad86c3a5240dcb146d862d7064219'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.625596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a94ad86c3a5240dcb146d862d7064219'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.629383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:06.642516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a94ad86c3a5240dcb146d862d7064219'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.665742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a94ad86c3a5240dcb146d862d7064219'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.717845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a94ad86c3a5240dcb146d862d7064219'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:06.733404 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1678/3383] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:07:06 2022] DELETE /identity/v3/users/a94ad86c3a5240dcb146d862d7064219 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:07:06.741233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:06.745066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:06.899507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d73fb-26c6-4ea5-9c7d-ec8af62a36f5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:06.969751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:07.025125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe30040-30c8-49df-8073-52e289ed848b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:07.025728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe30040-30c8-49df-8073-52e289ed848b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:07.026064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe30040-30c8-49df-8073-52e289ed848b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:07.026394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe30040-30c8-49df-8073-52e289ed848b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:07.028878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3fe30040-30c8-49df-8073-52e289ed848b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:07.046307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe30040-30c8-49df-8073-52e289ed848b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:07.052299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe30040-30c8-49df-8073-52e289ed848b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:07.080247 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1706/3384] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:07:07.745326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:07.747204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:07.889426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6a830-804d-4990-bdae-c4cf3b01fd09 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:07.894365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:07.895004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:07.895280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:07.895550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] PATH_INFO: `/v3/projects/f7c5cef34c274936b644a11a27937823` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:07.896732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c5cef34c274936b644a11a27937823)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:07.899020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:07.980278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c5cef34c274936b644a11a27937823', 'request_id': 'req-ab6e4847-6c47-4ec1-b723-85968ca75025'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:07.988872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c5cef34c274936b644a11a27937823', 'request_id': 'req-ab6e4847-6c47-4ec1-b723-85968ca75025'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:08.000750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c5cef34c274936b644a11a27937823', 'request_id': 'req-ab6e4847-6c47-4ec1-b723-85968ca75025'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:08.015115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] The token cache is being invalidate because project f7c5cef34c274936b644a11a27937823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:08.015486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:08.016335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ab6e4847-6c47-4ec1-b723-85968ca75025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:08.018213 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1679/3385] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:07 2022] DELETE /identity/v3/projects/f7c5cef34c274936b644a11a27937823 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:08.024260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b75848-7227-48b1-9178-2b700ba18261 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.024787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b75848-7227-48b1-9178-2b700ba18261 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.024866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b75848-7227-48b1-9178-2b700ba18261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.047944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-14b75848-7227-48b1-9178-2b700ba18261 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:08.049203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-14b75848-7227-48b1-9178-2b700ba18261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:08.160172 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1707/3386] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:08.160319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a7394-4ebe-401a-bef4-724f65d49124 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.160599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a7394-4ebe-401a-bef4-724f65d49124 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.160912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a7394-4ebe-401a-bef4-724f65d49124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.205042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0f9a7394-4ebe-401a-bef4-724f65d49124 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:08.206598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f9a7394-4ebe-401a-bef4-724f65d49124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:08.217151 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1680/3387] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:07:08.221382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b75848-7227-48b1-9178-2b700ba18261 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:08.237055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b75848-7227-48b1-9178-2b700ba18261 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:08.249397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:08.250060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.250323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.250645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.251255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:08.251558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:08.253445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:08.299628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1708/3388] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:08.304562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a7394-4ebe-401a-bef4-724f65d49124 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:08.315964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a7394-4ebe-401a-bef4-724f65d49124 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:08.323230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:08.324114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.324493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.324912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.326558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:08.328782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:08.369595 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1681/3389] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:07:08 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:08.370805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:08.383701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e85aa-060f-44e3-8cb4-65460451ab7e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:08.480384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464482-9881-40b6-922e-db489542e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:08.481039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464482-9881-40b6-922e-db489542e90d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.481309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464482-9881-40b6-922e-db489542e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.481582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464482-9881-40b6-922e-db489542e90d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.482260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464482-9881-40b6-922e-db489542e90d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:08.484201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464482-9881-40b6-922e-db489542e90d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:08.522072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:08.531843 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1709/3390] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:07:08 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:07:08.534418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da64346-38e9-4e7c-b0cb-4a5a31506a40 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:08.540881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:08.541534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.541865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.542143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.542807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:08.545167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:08.560457 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1682/3391] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:08.565337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464482-9881-40b6-922e-db489542e90d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:08.581401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464482-9881-40b6-922e-db489542e90d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:08.596570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf1416-e910-473d-9248-f1e2cea10320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:08.597243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf1416-e910-473d-9248-f1e2cea10320 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.597522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf1416-e910-473d-9248-f1e2cea10320 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.597800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf1416-e910-473d-9248-f1e2cea10320 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.598446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf1416-e910-473d-9248-f1e2cea10320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:08.614775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf1416-e910-473d-9248-f1e2cea10320 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:08.662363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:08.664404 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1710/3392] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:08.674921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94880a-8271-4ef7-ab87-33984922aa70 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:08.691463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:08.691463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.691463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.691901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] PATH_INFO: `/v3/projects/06d7e515b666445189b2ae77aa1f9251/users/ea20f07ef73b429498b8e703a6c0731f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.744162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d7e515b666445189b2ae77aa1f9251, user_id=ea20f07ef73b429498b8e703a6c0731f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:08.747426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:08.755839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:08.798706 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1683/3393] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:08 2022] PUT /identity/v3/projects/06d7e515b666445189b2ae77aa1f9251/users/ea20f07ef73b429498b8e703a6c0731f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:07:08.811608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300ea80-4857-42ba-be73-61785014df70 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:08.811948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300ea80-4857-42ba-be73-61785014df70 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:08.812226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300ea80-4857-42ba-be73-61785014df70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:08.883804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8300ea80-4857-42ba-be73-61785014df70 None None] MFA Rules not processed for user `ea20f07ef73b429498b8e703a6c0731f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:08.886716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8300ea80-4857-42ba-be73-61785014df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:09.049524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:09.166506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1711/3394] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:09.171561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:09.320824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eb361-e600-4714-9c0c-b67ac16d4ed8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:09.337018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:09.337662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:09.337947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:09.338223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:09.338876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:09.341055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:09.403892 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1684/3395] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:07:09 2022] POST /identity/v3/projects => generated 369 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:09.408981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300ea80-4857-42ba-be73-61785014df70 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:09.427844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300ea80-4857-42ba-be73-61785014df70 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:09.570948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:09.571742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:09.572091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:09.572460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:09.573868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:09.575394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:09.613802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:09.630003 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1712/3396] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:07:09 2022] POST /identity/v3/users => generated 291 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:09.649356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ccea-2a56-4237-99af-49a26caaa9d4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:09.663663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92b998-f568-4d0f-b9bb-15ee8fc0e047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:09.665553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b998-f568-4d0f-b9bb-15ee8fc0e047 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:09.668178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b998-f568-4d0f-b9bb-15ee8fc0e047 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:09.668542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b998-f568-4d0f-b9bb-15ee8fc0e047 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:09.669319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92b998-f568-4d0f-b9bb-15ee8fc0e047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:09.673286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92b998-f568-4d0f-b9bb-15ee8fc0e047 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:09.702743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1685/3397] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:07:09.708410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:09.723985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:09.724457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:09.724850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:09.728973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f599bfa-96d1-4caf-b9a8-105b9c69d7b5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:09.739195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:09.739879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:09.740153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:09.740529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] PATH_INFO: `/v3/projects/a2b3315c201d4ede80fa101567be149f/users/100455ad900f41b78732159492025860/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:09.755287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:09.757008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:09.772604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b3315c201d4ede80fa101567be149f, user_id=100455ad900f41b78732159492025860, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:09.777072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:09.811662 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1713/3398] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:09 2022] PUT /identity/v3/projects/a2b3315c201d4ede80fa101567be149f/users/100455ad900f41b78732159492025860/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:09.815495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:09.848236 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1686/3399] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:07:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:09.864201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:09.866190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:09.962065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2507c03-bca7-467d-a9e4-6f8d3de63b3d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:09.971277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:09.972545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:09.972856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:09.973129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:09.973792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:09.975414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:09.978913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f432ed6-842c-4dd3-a799-112ea6f102d0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:09.986729 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1714/3400] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:09.992012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:10.003510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:10.035651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8beedb5-39e8-4313-9017-3e749921c5f2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:10.047904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2b97e-17a1-495d-b44c-9da26847214f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:10.048718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2b97e-17a1-495d-b44c-9da26847214f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:10.049233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2b97e-17a1-495d-b44c-9da26847214f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:10.049578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2b97e-17a1-495d-b44c-9da26847214f admin admin] PATH_INFO: `/v3/projects/a2b3315c201d4ede80fa101567be149f/users/100455ad900f41b78732159492025860/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:10.052042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2b97e-17a1-495d-b44c-9da26847214f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b3315c201d4ede80fa101567be149f, user_id=100455ad900f41b78732159492025860, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:10.067791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2b97e-17a1-495d-b44c-9da26847214f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:10.093517 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1715/3401] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:09 2022] PUT /identity/v3/projects/a2b3315c201d4ede80fa101567be149f/users/100455ad900f41b78732159492025860/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:10.108481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:10.110150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a86f-075a-4a2b-aa12-585776712bb6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:10.110531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a86f-075a-4a2b-aa12-585776712bb6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:10.110821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a86f-075a-4a2b-aa12-585776712bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:10.111448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:10.111783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:10.112067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:10.170803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-63c9a86f-075a-4a2b-aa12-585776712bb6 None None] MFA Rules not processed for user `100455ad900f41b78732159492025860`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:10.172112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63c9a86f-075a-4a2b-aa12-585776712bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:10.198287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:10.214614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:10.217508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:10.236877 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1687/3402] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:07:10.381075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1716/3403] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:10.389158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:10.522008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ed8a-6bd9-43d3-88d0-a149edfcbaad service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:10.528552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:10.529511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:10.529890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:10.530265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:10.531125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:10.531709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:10.533963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:10.556164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1688/3404] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:10.557849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9a86f-075a-4a2b-aa12-585776712bb6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:10.578484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9a86f-075a-4a2b-aa12-585776712bb6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:10.603468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d756430-6738-4c88-8810-4502861c2eab tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:10.604124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d756430-6738-4c88-8810-4502861c2eab tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:10.604407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d756430-6738-4c88-8810-4502861c2eab tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:10.604699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d756430-6738-4c88-8810-4502861c2eab tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:10.609497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d756430-6738-4c88-8810-4502861c2eab tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:10.616146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d756430-6738-4c88-8810-4502861c2eab tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:10.748794 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1717/3405] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:07:10 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:10.750781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:10.770922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f03bd-a47e-4370-8c53-6a7b8e6acfd9 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:10.934009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:10.979664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:10.980319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:10.980605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:10.980905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:10.981577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:10.981884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:10.983521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:11.030430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1689/3406] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:11.247865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:11.249562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:11.249562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:11.301013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:11.302906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:11.447737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:11.473586 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1718/3407] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:07:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:11.474538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:11.477575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:11.585278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68290b0a-ae5e-47df-8770-5b82434e5c5a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:11.591734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b75fc-3932-4e4e-affc-b293c901af47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:11.593726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b75fc-3932-4e4e-affc-b293c901af47 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:11.594166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b75fc-3932-4e4e-affc-b293c901af47 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:11.594610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b75fc-3932-4e4e-affc-b293c901af47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:11.612524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab9e2-608b-4de1-b593-2ace1169c0d3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:11.613432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b75fc-3932-4e4e-affc-b293c901af47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:11.615514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b75fc-3932-4e4e-affc-b293c901af47 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:11.623321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:11.623961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:11.624236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:11.624501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:11.630731 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1690/3408] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:11.648179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:11.650551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:11.677729 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1719/3409] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:11.690591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cba4c6-3657-491b-9285-ee2fed95bfa5 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:11.691180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cba4c6-3657-491b-9285-ee2fed95bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:11.691637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cba4c6-3657-491b-9285-ee2fed95bfa5 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:11.696756 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1691/3410] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:07:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:07:11.702459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:07:11.715324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:11.736944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e229b-31ce-4e5a-b55a-c33c98a47820 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:11.753541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:11.754263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:11.754485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:11.754832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/projects/91e709a461b944f4a9118e979d016f7b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:11.757345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:get_project(project_id=91e709a461b944f4a9118e979d016f7b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:11.759710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:11.761875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1720/3411] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:07:11 2022] GET /identity/v3/projects/91e709a461b944f4a9118e979d016f7b => generated 371 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:07:11.970662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cba4c6-3657-491b-9285-ee2fed95bfa5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:11.973266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-34cba4c6-3657-491b-9285-ee2fed95bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:12.095921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cba4c6-3657-491b-9285-ee2fed95bfa5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:12.100727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396b82b-77c9-47f6-81f0-b8bc20d936f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:12.101609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b82b-77c9-47f6-81f0-b8bc20d936f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:12.101609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b82b-77c9-47f6-81f0-b8bc20d936f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:12.101944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b82b-77c9-47f6-81f0-b8bc20d936f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:12.104256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6396b82b-77c9-47f6-81f0-b8bc20d936f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:12.118632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396b82b-77c9-47f6-81f0-b8bc20d936f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:12.121091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396b82b-77c9-47f6-81f0-b8bc20d936f1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:12.132847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1692/3412] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:12.217574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:12.236228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a5231-6333-413a-8671-a250fe6494c2 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:12.243144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:12.243776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:12.244068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:12.244341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:12.248645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:12.266307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:12.270947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:12.273715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:12.323552 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1721/3413] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:12.561543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa0b0-ace8-4cea-966b-a72dcd1a89d6 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:12.561850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa0b0-ace8-4cea-966b-a72dcd1a89d6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:12.562062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa0b0-ace8-4cea-966b-a72dcd1a89d6 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:12.563554 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1693/3414] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:07:12.575530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:07:12.597519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:12.622823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d2334-07d1-4117-8469-05e8100acb5f service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:12.632810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:12.633944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:12.634382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:12.634867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/projects/91e709a461b944f4a9118e979d016f7b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:12.638357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:get_project(project_id=91e709a461b944f4a9118e979d016f7b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:12.640815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:12.642474 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1722/3415] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:07:12 2022] GET /identity/v3/projects/91e709a461b944f4a9118e979d016f7b => generated 371 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:12.713127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa0b0-ace8-4cea-966b-a72dcd1a89d6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:12.741361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa0b0-ace8-4cea-966b-a72dcd1a89d6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:12.761641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:12.762550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:12.768164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:12.768544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:12.769398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:12.769576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:12.771361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:12.812724 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1694/3416] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:07:12.817535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:12.832166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e70b5a-9316-44d9-953d-9f3b66d027d5 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:12.852928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415dc2b-48a6-4c48-8035-b2241248fa67 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:12.856148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415dc2b-48a6-4c48-8035-b2241248fa67 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:12.857571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415dc2b-48a6-4c48-8035-b2241248fa67 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:12.858750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415dc2b-48a6-4c48-8035-b2241248fa67 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:12.859790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415dc2b-48a6-4c48-8035-b2241248fa67 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:12.867245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415dc2b-48a6-4c48-8035-b2241248fa67 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:12.926699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1723/3417] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:07:12 2022] POST /identity/v3/users => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:12.929366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:12.945059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55f59-a61f-41b3-b288-8e2b4a8f298c tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:12.952719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2358bb2-4421-4637-bc6b-686f177ff679 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:12.954634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2358bb2-4421-4637-bc6b-686f177ff679 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:12.955654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2358bb2-4421-4637-bc6b-686f177ff679 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:12.956214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2358bb2-4421-4637-bc6b-686f177ff679 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:12.957519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2358bb2-4421-4637-bc6b-686f177ff679 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:12.958811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2358bb2-4421-4637-bc6b-686f177ff679 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:12.962590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2358bb2-4421-4637-bc6b-686f177ff679 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:13.001461 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1695/3418] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:07:13.535161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519d863-66ff-4301-a7c5-dd6a207cf9d9 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:13.535768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519d863-66ff-4301-a7c5-dd6a207cf9d9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:13.535768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519d863-66ff-4301-a7c5-dd6a207cf9d9 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:13.535916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0fcfd-2633-4d68-81f4-99f5451cf1c8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:13.536052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0fcfd-2633-4d68-81f4-99f5451cf1c8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:13.536628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0fcfd-2633-4d68-81f4-99f5451cf1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:13.537089 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1724/3419] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:07:13.557432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519d863-66ff-4301-a7c5-dd6a207cf9d9 None None] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:07:13.559208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5519d863-66ff-4301-a7c5-dd6a207cf9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:13.572122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-11f0fcfd-2633-4d68-81f4-99f5451cf1c8 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:13.573958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-11f0fcfd-2633-4d68-81f4-99f5451cf1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:13.585788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1696/3420] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:07:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 27 10:07:13.629812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519d863-66ff-4301-a7c5-dd6a207cf9d9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:13.643875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0fcfd-2633-4d68-81f4-99f5451cf1c8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:13.671596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0fcfd-2633-4d68-81f4-99f5451cf1c8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:13.686330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:13.688644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:13.689493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:13.689780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:13.690466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519d863-66ff-4301-a7c5-dd6a207cf9d9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:13.712886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e429f-97e3-4971-9648-f7c76567ac2b tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:13.713590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e429f-97e3-4971-9648-f7c76567ac2b tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:13.713896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e429f-97e3-4971-9648-f7c76567ac2b tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:13.714171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e429f-97e3-4971-9648-f7c76567ac2b tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/projects/91e709a461b944f4a9118e979d016f7b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:13.715411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e429f-97e3-4971-9648-f7c76567ac2b tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:get_project(project_id=91e709a461b944f4a9118e979d016f7b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:13.724338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:13.729905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:13.736220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e429f-97e3-4971-9648-f7c76567ac2b tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:13.738643 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1725/3421] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:07:13 2022] GET /identity/v3/projects/91e709a461b944f4a9118e979d016f7b => generated 371 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:13.759152 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1697/3422] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:15.454050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faac44-c7da-4d73-b393-09916d30e0fa None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:15.454981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faac44-c7da-4d73-b393-09916d30e0fa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:15.455465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faac44-c7da-4d73-b393-09916d30e0fa None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:15.457911 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1726/3423] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:07:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:07:15.475783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:07:15.517360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:15.552704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78808f7a-ba0c-45eb-a9c8-aa61197f8f2d service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:15.575451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:15.576011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:15.576307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:15.576827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/projects/91e709a461b944f4a9118e979d016f7b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:15.582321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:get_project(project_id=91e709a461b944f4a9118e979d016f7b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:15.598152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:15.615648 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1698/3424] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:07:15 2022] GET /identity/v3/projects/91e709a461b944f4a9118e979d016f7b => generated 371 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:07:15.697762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faac44-c7da-4d73-b393-09916d30e0fa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:15.749782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faac44-c7da-4d73-b393-09916d30e0fa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:15.764429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:15.765352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:15.768890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:15.769334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/users/0f1ba9f0940a46b59b18d87049d9eea9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:15.844260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0f1ba9f0940a46b59b18d87049d9eea9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:15.846739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:15.978488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1ba9f0940a46b59b18d87049d9eea9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:15.981483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1ba9f0940a46b59b18d87049d9eea9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.015033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1ba9f0940a46b59b18d87049d9eea9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.035583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1ba9f0940a46b59b18d87049d9eea9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.036858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1ba9f0940a46b59b18d87049d9eea9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.148829 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1727/3425] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:07:15 2022] DELETE /identity/v3/users/0f1ba9f0940a46b59b18d87049d9eea9 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:16.150085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:16.151863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:16.173798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2a0a-6a01-4537-8203-837befb81619 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:16.249184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:16.418803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '100455ad900f41b78732159492025860', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2b3315c201d4ede80fa101567be149f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:16.420431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:16.420752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:16.421181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] PATH_INFO: `/v3/projects/91e709a461b944f4a9118e979d016f7b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:16.425692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorizing `identity:delete_project(project_id=91e709a461b944f4a9118e979d016f7b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:16.431275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:16.628605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e709a461b944f4a9118e979d016f7b', 'request_id': 'req-a9b90e25-05b8-4d08-8e1f-890fbff35519'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.646947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e709a461b944f4a9118e979d016f7b', 'request_id': 'req-a9b90e25-05b8-4d08-8e1f-890fbff35519'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.666861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e709a461b944f4a9118e979d016f7b', 'request_id': 'req-a9b90e25-05b8-4d08-8e1f-890fbff35519'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.693330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] The token cache is being invalidate because project 91e709a461b944f4a9118e979d016f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:16.693941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.694911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9b90e25-05b8-4d08-8e1f-890fbff35519 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:16.702977 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1699/3426] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:16 2022] DELETE /identity/v3/projects/91e709a461b944f4a9118e979d016f7b => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:16.713427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:16.715501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:16.970888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06eb91-0830-4daa-b9f4-6fd8a70f5708 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:17.064577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:17.144401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:17.153382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:17.153382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:17.153382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] PATH_INFO: `/v3/users/ea20f07ef73b429498b8e703a6c0731f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:17.156598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea20f07ef73b429498b8e703a6c0731f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:17.161132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:17.277019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea20f07ef73b429498b8e703a6c0731f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:17.277825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea20f07ef73b429498b8e703a6c0731f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:17.293644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea20f07ef73b429498b8e703a6c0731f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:17.316924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea20f07ef73b429498b8e703a6c0731f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:17.317950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea20f07ef73b429498b8e703a6c0731f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:17.374481 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1728/3427] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:07:16 2022] DELETE /identity/v3/users/ea20f07ef73b429498b8e703a6c0731f => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:17.379417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:17.379878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:17.380251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:17.418595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:17.419975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:17.571388 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1700/3428] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:17.585065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:17.586875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:17.698427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce79477-fcec-4c3a-baee-e97fa92147ee admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:17.705256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:17.706027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:17.706638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:17.707168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:17.726362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:17.727988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:17.739584 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1729/3429] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:18.426575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:18.428460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:18.473278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75eee8-6ace-4cb1-9927-5d5082c56121 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:18.492490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:18.496152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:18.496679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:18.499407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] PATH_INFO: `/v3/projects/06d7e515b666445189b2ae77aa1f9251` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:18.501979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d7e515b666445189b2ae77aa1f9251)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:18.505806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:18.615234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:18.615234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d7e515b666445189b2ae77aa1f9251', 'request_id': 'req-ed06b338-dae9-4924-b068-8da39c26897f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.627848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d7e515b666445189b2ae77aa1f9251', 'request_id': 'req-ed06b338-dae9-4924-b068-8da39c26897f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.650597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d7e515b666445189b2ae77aa1f9251', 'request_id': 'req-ed06b338-dae9-4924-b068-8da39c26897f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.666885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] The token cache is being invalidate because project 06d7e515b666445189b2ae77aa1f9251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:18.667132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.667864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.674031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1701/3430] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:18 2022] DELETE /identity/v3/projects/06d7e515b666445189b2ae77aa1f9251 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:18.675303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:18.697278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656b66d-315a-425a-87da-6b6e6433bab5 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:18.814793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:18.835078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:18.835883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:18.836257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:18.836614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] PATH_INFO: `/v3/users/100455ad900f41b78732159492025860` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:18.838319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100455ad900f41b78732159492025860)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:18.840395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:18.960299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100455ad900f41b78732159492025860'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.961468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100455ad900f41b78732159492025860'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.973806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100455ad900f41b78732159492025860'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.990015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100455ad900f41b78732159492025860'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:18.992403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a10e1d6-68c8-4d6d-9ebd-4e1e7ac48166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100455ad900f41b78732159492025860'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:19.039959 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1730/3431] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:07:18 2022] DELETE /identity/v3/users/100455ad900f41b78732159492025860 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:19.996471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:19.998546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:20.013079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06b338-dae9-4924-b068-8da39c26897f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:20.127726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:20.128385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.128678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.129767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] PATH_INFO: `/v3/projects/a2b3315c201d4ede80fa101567be149f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.133235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b3315c201d4ede80fa101567be149f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:20.138224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:20.225686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b3315c201d4ede80fa101567be149f', 'request_id': 'req-19a83bae-8c73-40df-b4d4-42d99144b63e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:20.248023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b3315c201d4ede80fa101567be149f', 'request_id': 'req-19a83bae-8c73-40df-b4d4-42d99144b63e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:20.263602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b3315c201d4ede80fa101567be149f', 'request_id': 'req-19a83bae-8c73-40df-b4d4-42d99144b63e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:20.282958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] The token cache is being invalidate because project a2b3315c201d4ede80fa101567be149f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:20.283352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:20.284101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-19a83bae-8c73-40df-b4d4-42d99144b63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:20.286238 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1702/3432] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:19 2022] DELETE /identity/v3/projects/a2b3315c201d4ede80fa101567be149f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:20.296758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa94db-105f-43cf-840d-7d550c7eeadb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.297523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa94db-105f-43cf-840d-7d550c7eeadb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.297523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa94db-105f-43cf-840d-7d550c7eeadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.328814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-10fa94db-105f-43cf-840d-7d550c7eeadb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:20.330436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-10fa94db-105f-43cf-840d-7d550c7eeadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:20.400328 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1731/3433] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:20.409133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88222-558e-4463-9930-f56d70f4f2d8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.409305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88222-558e-4463-9930-f56d70f4f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.409583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88222-558e-4463-9930-f56d70f4f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.437845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-96c88222-558e-4463-9930-f56d70f4f2d8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:20.439425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-96c88222-558e-4463-9930-f56d70f4f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:20.449308 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1703/3434] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:20.453635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa94db-105f-43cf-840d-7d550c7eeadb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:20.469103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa94db-105f-43cf-840d-7d550c7eeadb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:20.476765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:20.477668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.478093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.478512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.479322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:20.479922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:20.481896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:20.495398 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1732/3435] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:20.499672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c88222-558e-4463-9930-f56d70f4f2d8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:20.510326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c88222-558e-4463-9930-f56d70f4f2d8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:20.516510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:20.517183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.517647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.518064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.519784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:20.522079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:20.569301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1704/3436] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:07:20 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:20.574870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:20.585501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5c70-6344-4e6f-984a-0e56aefa4832 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:20.648087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:20.658731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:20.659857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.660287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.661248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.661999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:20.665842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:20.700214 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1733/3437] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:07:20 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:07:20.701778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:20.714344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e072-94a6-4467-8838-64741491f9ef admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:20.719662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:20.720377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.720756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.721096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.721977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:20.723647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:20.736239 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1705/3438] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:20.740843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:20.752712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502519-a2ad-46a6-b3fa-d4768c3c68b8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:20.759347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fad0cd-1c47-4b1f-a5a4-c010220b9957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:20.760176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fad0cd-1c47-4b1f-a5a4-c010220b9957 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.760618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fad0cd-1c47-4b1f-a5a4-c010220b9957 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.761072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fad0cd-1c47-4b1f-a5a4-c010220b9957 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.761991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fad0cd-1c47-4b1f-a5a4-c010220b9957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:20.763713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fad0cd-1c47-4b1f-a5a4-c010220b9957 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:20.791129 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1734/3439] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:20.796119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:20.811145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6f8aa-477f-42a9-92df-448a049c6d43 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:20.817962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:20.818674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.818953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.819230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] PATH_INFO: `/v3/projects/52efcec2c8a14640a8c3df1dc576bbf1/users/b4e0f20e52074142a3b525bafbb30c97/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.838215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=52efcec2c8a14640a8c3df1dc576bbf1, user_id=b4e0f20e52074142a3b525bafbb30c97, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:20.841124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:20.870237 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1706/3440] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:20 2022] PUT /identity/v3/projects/52efcec2c8a14640a8c3df1dc576bbf1/users/b4e0f20e52074142a3b525bafbb30c97/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:20.871656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c342e-e9d7-416c-88c5-29b6086fab22 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:20.872013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c342e-e9d7-416c-88c5-29b6086fab22 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:20.872288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c342e-e9d7-416c-88c5-29b6086fab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:20.914280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2f4c342e-e9d7-416c-88c5-29b6086fab22 None None] MFA Rules not processed for user `b4e0f20e52074142a3b525bafbb30c97`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:20.915883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f4c342e-e9d7-416c-88c5-29b6086fab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:21.031676 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1735/3441] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:07:21.036314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:21.043676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:21.191898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:21.192299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:21.192789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:21.225100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122eb4e6-2431-4a20-9ef1-d5c24c6340ef admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:21.232115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:21.233181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:21.233760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:21.234369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:21.235163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:21.236643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:21.237998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:21.238122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:21.304970 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1707/3442] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:07:21 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:21.309162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:21.402890 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1736/3443] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:07:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:21.421293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:21.425225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:21.471765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08a8dd-6230-49e9-9979-d40ec24e2aa7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:21.493571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:21.494576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:21.496004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:21.500025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:21.501220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:21.510257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:21.538607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c278a0-0fb3-4eac-8ba4-18597f313ac5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:21.552392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:21.566224 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1708/3444] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:07:21 2022] POST /identity/v3/users => generated 295 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:21.572421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:21.600769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eafc6-7098-496f-9f7f-0a92f4cd8d7c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:21.609748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:21.610686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:21.611199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:21.611733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:21.612433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:21.614163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:21.675159 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1709/3445] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:21.677607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:21.701970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d4394-1a09-40bf-9986-ab91476d3226 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:21.708215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:21.709054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:21.709460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:21.709830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:21.714001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:21.715157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:21.715893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:21.717748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] PATH_INFO: `/v3/projects/89a2927af7e8412fa3fd454fbd375459/users/566ef52c26d044f8902be6f32e2b51c3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:21.758475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a2927af7e8412fa3fd454fbd375459, user_id=566ef52c26d044f8902be6f32e2b51c3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:21.762387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:21.794656 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1710/3446] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:21 2022] PUT /identity/v3/projects/89a2927af7e8412fa3fd454fbd375459/users/566ef52c26d044f8902be6f32e2b51c3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:21.799133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:21.810596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:21.812684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:21.839817 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1737/3447] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:21.913395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:21.953193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d101e9-98fd-4c36-9e68-56112fe5944f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:21.969758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d85d3a-fe58-493e-b4d2-72ffb40ef7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:21.970412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d85d3a-fe58-493e-b4d2-72ffb40ef7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:21.972996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d85d3a-fe58-493e-b4d2-72ffb40ef7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:21.973295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d85d3a-fe58-493e-b4d2-72ffb40ef7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:21.975402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d85d3a-fe58-493e-b4d2-72ffb40ef7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:21.979735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d85d3a-fe58-493e-b4d2-72ffb40ef7d9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:22.051164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1711/3448] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:07:22.056557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:22.082314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a6e60-2a95-4cdd-80bb-d086874b75b6 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:22.093136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737bbd3-b4ff-46bf-bcf4-ecf44e6d2e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:22.093904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737bbd3-b4ff-46bf-bcf4-ecf44e6d2e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:22.094116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737bbd3-b4ff-46bf-bcf4-ecf44e6d2e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:22.094437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737bbd3-b4ff-46bf-bcf4-ecf44e6d2e1f admin admin] PATH_INFO: `/v3/projects/89a2927af7e8412fa3fd454fbd375459/users/566ef52c26d044f8902be6f32e2b51c3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:22.097269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737bbd3-b4ff-46bf-bcf4-ecf44e6d2e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a2927af7e8412fa3fd454fbd375459, user_id=566ef52c26d044f8902be6f32e2b51c3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:22.104448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737bbd3-b4ff-46bf-bcf4-ecf44e6d2e1f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:22.137724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1738/3449] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:22 2022] PUT /identity/v3/projects/89a2927af7e8412fa3fd454fbd375459/users/566ef52c26d044f8902be6f32e2b51c3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:22.139758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:22.140050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:22.140331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:22.183889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] MFA Rules not processed for user `566ef52c26d044f8902be6f32e2b51c3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:22.185122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:22.404914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1712/3450] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:22.418395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1954a09-64f7-4fd2-bf78-d4419ea33739 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:22.418932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1954a09-64f7-4fd2-bf78-d4419ea33739 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:22.419677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1954a09-64f7-4fd2-bf78-d4419ea33739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:22.461618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e1954a09-64f7-4fd2-bf78-d4419ea33739 None None] MFA Rules not processed for user `b4e0f20e52074142a3b525bafbb30c97`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:22.465159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e1954a09-64f7-4fd2-bf78-d4419ea33739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:22.658075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1739/3451] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:22.752541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:22.754020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:22.981454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c129b-e7d6-4d8b-977a-c41b6159e0e4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:22.998622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482e69d-10bc-4803-8f9c-dfed1ed98430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:22.999517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e69d-10bc-4803-8f9c-dfed1ed98430 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:23.000718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e69d-10bc-4803-8f9c-dfed1ed98430 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:23.001759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e69d-10bc-4803-8f9c-dfed1ed98430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:23.034728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482e69d-10bc-4803-8f9c-dfed1ed98430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:23.036821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482e69d-10bc-4803-8f9c-dfed1ed98430 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:23.070471 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1713/3452] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:24.098816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dcac5-ee75-4033-a569-7071af769d87 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:24.101531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dcac5-ee75-4033-a569-7071af769d87 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:24.103831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dcac5-ee75-4033-a569-7071af769d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:24.175111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-373dcac5-ee75-4033-a569-7071af769d87 None None] MFA Rules not processed for user `b4e0f20e52074142a3b525bafbb30c97`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:24.177187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-373dcac5-ee75-4033-a569-7071af769d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:24.195565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1740/3453] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:24.281907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e3e8f-b86e-4ed0-987a-142bb7f2b560 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:24.282799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e3e8f-b86e-4ed0-987a-142bb7f2b560 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:24.283213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e3e8f-b86e-4ed0-987a-142bb7f2b560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:24.311141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-375e3e8f-b86e-4ed0-987a-142bb7f2b560 None None] MFA Rules not processed for user `68d9297ca0884c2fa7f5283f722451ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:24.312799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-375e3e8f-b86e-4ed0-987a-142bb7f2b560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:24.381309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124f05f-3005-4458-8680-1ac4685e638d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:24.381673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124f05f-3005-4458-8680-1ac4685e638d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:24.383877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124f05f-3005-4458-8680-1ac4685e638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:24.404807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:24.414503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9124f05f-3005-4458-8680-1ac4685e638d None None] MFA Rules not processed for user `430de2723f234fac8f09068195d8ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:24.416611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9124f05f-3005-4458-8680-1ac4685e638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:24.436292 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1714/3454] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:24.557830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:24.650909 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1741/3455] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:28.524772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f64d35-25e6-4263-89c1-064e059abc08 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:28.525806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f64d35-25e6-4263-89c1-064e059abc08 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:28.526248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f64d35-25e6-4263-89c1-064e059abc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:28.549859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-98f64d35-25e6-4263-89c1-064e059abc08 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:28.551339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-98f64d35-25e6-4263-89c1-064e059abc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:28.710683 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1715/3456] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:31.785935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98add2ef-56bd-4d2d-8fab-f4f0451e61fb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:31.786435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98add2ef-56bd-4d2d-8fab-f4f0451e61fb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:31.786550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98add2ef-56bd-4d2d-8fab-f4f0451e61fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:31.823571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-98add2ef-56bd-4d2d-8fab-f4f0451e61fb None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:31.824877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-98add2ef-56bd-4d2d-8fab-f4f0451e61fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:31.984488 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1742/3457] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:07:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:31.988398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f64d35-25e6-4263-89c1-064e059abc08 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:32.158386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f64d35-25e6-4263-89c1-064e059abc08 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:32.167758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:32.168397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:32.168706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:32.168997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:32.180297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:32.181938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:32.192803 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1716/3458] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:32.736303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3d9c5-1aef-412c-8e09-6c60fa7fd932 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:32.736975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3d9c5-1aef-412c-8e09-6c60fa7fd932 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:32.737397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3d9c5-1aef-412c-8e09-6c60fa7fd932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:32.765486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-08d3d9c5-1aef-412c-8e09-6c60fa7fd932 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:32.766704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08d3d9c5-1aef-412c-8e09-6c60fa7fd932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:32.775547 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1743/3459] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:07:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:32.790787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:32.805574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd350c1-f422-48e1-afa8-d989d67ee06a service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:32.822510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dca75-872f-4bec-8d47-5df03cfbe746 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:32.823458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dca75-872f-4bec-8d47-5df03cfbe746 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:32.823905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dca75-872f-4bec-8d47-5df03cfbe746 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:32.824322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dca75-872f-4bec-8d47-5df03cfbe746 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:32.833733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dca75-872f-4bec-8d47-5df03cfbe746 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:32.836005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dca75-872f-4bec-8d47-5df03cfbe746 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:32.852886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1717/3460] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:33.076757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23795446-51cc-46f6-bb51-67120247e50b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:33.076757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23795446-51cc-46f6-bb51-67120247e50b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:33.076757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23795446-51cc-46f6-bb51-67120247e50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:33.103944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-23795446-51cc-46f6-bb51-67120247e50b None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:33.105326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23795446-51cc-46f6-bb51-67120247e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:33.113350 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1744/3461] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:33.244538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:33.245548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:33.246033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:33.275376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:33.276642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:33.291942 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1718/3462] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:07:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:33.299986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23795446-51cc-46f6-bb51-67120247e50b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:33.314704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23795446-51cc-46f6-bb51-67120247e50b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:33.321353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f5765-27f3-4376-998f-2185accc0fea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:33.322070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f5765-27f3-4376-998f-2185accc0fea service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:33.322609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f5765-27f3-4376-998f-2185accc0fea service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:33.323077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f5765-27f3-4376-998f-2185accc0fea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:33.331976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f5765-27f3-4376-998f-2185accc0fea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:33.334084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f5765-27f3-4376-998f-2185accc0fea service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:33.352951 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1745/3463] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:33.488747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:33.490726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:33.576315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6dcb-f839-4f65-8e34-3132bca8d007 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:33.590236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:33.590939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:33.591269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:33.591544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] PATH_INFO: `/v3/users/032de99964be4424ba019adf2eedf680` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:33.592812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032de99964be4424ba019adf2eedf680)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:33.594831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:33.608193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:33.685339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032de99964be4424ba019adf2eedf680'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:33.685918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032de99964be4424ba019adf2eedf680'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:33.705162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032de99964be4424ba019adf2eedf680'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:33.729632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032de99964be4424ba019adf2eedf680'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:33.748243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032de99964be4424ba019adf2eedf680'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:33.758390 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1719/3464] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:07:33 2022] DELETE /identity/v3/users/032de99964be4424ba019adf2eedf680 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:33.762872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:33.763562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:33.764085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:33.789584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:33.791644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:33.881875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1746/3465] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:07:33.898897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:33.901883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:33.993028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73d286-0010-4612-b503-958a91f52da6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:33.997773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f59ec6-de09-486f-a858-29f42fa64316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:33.998730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f59ec6-de09-486f-a858-29f42fa64316 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:33.999078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f59ec6-de09-486f-a858-29f42fa64316 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:33.999423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f59ec6-de09-486f-a858-29f42fa64316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:34.010103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f59ec6-de09-486f-a858-29f42fa64316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:34.011673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f59ec6-de09-486f-a858-29f42fa64316 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:34.023293 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1720/3466] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:34.367331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:34.370139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:34.392602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229a3ec-2548-4ec5-b495-fe1f16fcf3d5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:34.398797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:34.399833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:34.400261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:34.400738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] PATH_INFO: `/v3/projects/b932820432be493b8a85520f58f49606` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:34.402498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b932820432be493b8a85520f58f49606)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:34.404629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:34.534049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b932820432be493b8a85520f58f49606', 'request_id': 'req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:34.548242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b932820432be493b8a85520f58f49606', 'request_id': 'req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:34.553123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:34.562872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b932820432be493b8a85520f58f49606', 'request_id': 'req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:34.594051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] The token cache is being invalidate because project b932820432be493b8a85520f58f49606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:34.594611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:34.595473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9bd06d19-85cf-4065-9985-5f8d9bde4c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:34.603312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9759c4-98bf-4ff6-909b-f9570870f986 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:34.608826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9759c4-98bf-4ff6-909b-f9570870f986 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:34.609713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9759c4-98bf-4ff6-909b-f9570870f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:34.615764 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1747/3467] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:34 2022] DELETE /identity/v3/projects/b932820432be493b8a85520f58f49606 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:34.650898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0b9759c4-98bf-4ff6-909b-f9570870f986 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:34.652550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b9759c4-98bf-4ff6-909b-f9570870f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:34.787681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df13a4-e11d-433e-8d0f-118229ab42a7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:34.788006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1721/3468] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:34.788150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df13a4-e11d-433e-8d0f-118229ab42a7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:34.788448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df13a4-e11d-433e-8d0f-118229ab42a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:34.818946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-76df13a4-e11d-433e-8d0f-118229ab42a7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:34.820583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-76df13a4-e11d-433e-8d0f-118229ab42a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:34.833419 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1748/3469] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:34.834624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9759c4-98bf-4ff6-909b-f9570870f986 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:34.855711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9759c4-98bf-4ff6-909b-f9570870f986 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:34.864340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:34.866098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:34.866656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:34.868017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:34.871863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:34.872282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:34.874010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:34.892482 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1722/3470] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:07:34.897506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df13a4-e11d-433e-8d0f-118229ab42a7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:34.907288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df13a4-e11d-433e-8d0f-118229ab42a7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:34.913704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:34.915065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:34.915349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:34.915624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:34.916283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:34.918301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:34.998909 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1749/3471] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:07:34 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:35.003500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:35.019341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa05cb-95d4-4903-b906-83b2336f92b4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:35.075265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:35.157663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:35.158699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:35.159075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:35.159477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:35.160340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:35.163290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:35.201641 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1723/3472] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:07:35 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:35.203073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:35.219499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f4c6-d1c9-41bd-be8a-8f06bd4b1810 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:35.245713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:35.245713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:35.245713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:35.248812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:35.252527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:35.259819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:35.297540 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1750/3473] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:35.299462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:35.314969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818a9b8-2ab6-458b-b644-e6d09b93e4c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:35.344690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d6743-60f2-48bc-901e-e2128777cc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:35.344690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d6743-60f2-48bc-901e-e2128777cc28 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:35.344690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d6743-60f2-48bc-901e-e2128777cc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:35.344690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d6743-60f2-48bc-901e-e2128777cc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:35.345116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d6743-60f2-48bc-901e-e2128777cc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:35.345116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d6743-60f2-48bc-901e-e2128777cc28 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:35.365337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1724/3474] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:35.370934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:35.387006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4440d3-369f-4feb-b585-7889333e6e86 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:35.398338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:35.399300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:35.399650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:35.400697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] PATH_INFO: `/v3/projects/462859c44a93480aba0df9344b789348/users/906b7b3d2366422cbc5da1f3080680f6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:35.445133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=462859c44a93480aba0df9344b789348, user_id=906b7b3d2366422cbc5da1f3080680f6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:35.465596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:35.526812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6607a09-e5af-444e-831a-d748502abd6d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:35.527296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6607a09-e5af-444e-831a-d748502abd6d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:35.527447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1751/3475] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:35 2022] PUT /identity/v3/projects/462859c44a93480aba0df9344b789348/users/906b7b3d2366422cbc5da1f3080680f6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:35.527546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6607a09-e5af-444e-831a-d748502abd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:35.560390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f6607a09-e5af-444e-831a-d748502abd6d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:35.562060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f6607a09-e5af-444e-831a-d748502abd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:35.654091 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1725/3476] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:35.681000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:35.683819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:35.824405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca95f0-50b7-48e1-a503-9bb2346004aa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:35.838280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42744f4e-29a2-4f0c-892c-3792614f8331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:35.839551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42744f4e-29a2-4f0c-892c-3792614f8331 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:35.840002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42744f4e-29a2-4f0c-892c-3792614f8331 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:35.840418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42744f4e-29a2-4f0c-892c-3792614f8331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:35.853546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42744f4e-29a2-4f0c-892c-3792614f8331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:35.855370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42744f4e-29a2-4f0c-892c-3792614f8331 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:35.871963 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1752/3477] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:36.472744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40211f46-846d-4022-bee1-e36a9347f336 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:36.473239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40211f46-846d-4022-bee1-e36a9347f336 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:36.473815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40211f46-846d-4022-bee1-e36a9347f336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:36.494923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-40211f46-846d-4022-bee1-e36a9347f336 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:36.496684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-40211f46-846d-4022-bee1-e36a9347f336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:36.574255 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1726/3478] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:37.981739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62556-3fc0-40ae-87d2-a7f7b6a6ac3e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:37.982632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62556-3fc0-40ae-87d2-a7f7b6a6ac3e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:37.983112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62556-3fc0-40ae-87d2-a7f7b6a6ac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:38.021717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-34b62556-3fc0-40ae-87d2-a7f7b6a6ac3e None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:38.023329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34b62556-3fc0-40ae-87d2-a7f7b6a6ac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:38.057704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1753/3479] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:07:40.139814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e1106-f8e5-416d-b150-236738d5ad70 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:40.140396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e1106-f8e5-416d-b150-236738d5ad70 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:40.140396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e1106-f8e5-416d-b150-236738d5ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:40.170682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-750e1106-f8e5-416d-b150-236738d5ad70 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:40.171922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-750e1106-f8e5-416d-b150-236738d5ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:40.197790 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1727/3480] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:43.261553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c3c6c-f56f-406a-a272-6d701b6cde37 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:43.262504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c3c6c-f56f-406a-a272-6d701b6cde37 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:43.262969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c3c6c-f56f-406a-a272-6d701b6cde37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:43.308211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cc6c3c6c-f56f-406a-a272-6d701b6cde37 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:43.309756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cc6c3c6c-f56f-406a-a272-6d701b6cde37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:43.339031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:43.339821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:43.340333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:43.353940 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1754/3481] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:43.380969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:43.382222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:43.443669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:43.523354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1728/3482] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:44.282424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b404f-ba9b-4a6e-ac41-ae3908a2544a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:44.282992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b404f-ba9b-4a6e-ac41-ae3908a2544a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:44.282992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b404f-ba9b-4a6e-ac41-ae3908a2544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:44.329671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5f1b404f-ba9b-4a6e-ac41-ae3908a2544a None None] MFA Rules not processed for user `906b7b3d2366422cbc5da1f3080680f6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:44.330913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5f1b404f-ba9b-4a6e-ac41-ae3908a2544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:44.406075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:44.415531 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1755/3483] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:44.714736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:44.716301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:44.850024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ae52c-b240-430a-b2c2-5109fdc37ed3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:44.857184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523191c8-84e6-425f-9e3a-b43c9c24b976 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:44.861006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523191c8-84e6-425f-9e3a-b43c9c24b976 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:44.861686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523191c8-84e6-425f-9e3a-b43c9c24b976 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:44.862688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523191c8-84e6-425f-9e3a-b43c9c24b976 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:44.878379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523191c8-84e6-425f-9e3a-b43c9c24b976 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:44.880402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523191c8-84e6-425f-9e3a-b43c9c24b976 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:44.891496 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1729/3484] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:45.505352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff246876-15ed-40aa-ab56-f569c85e7df9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:45.506202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff246876-15ed-40aa-ab56-f569c85e7df9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:45.506622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff246876-15ed-40aa-ab56-f569c85e7df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:45.532774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ff246876-15ed-40aa-ab56-f569c85e7df9 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:45.535026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ff246876-15ed-40aa-ab56-f569c85e7df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:45.558886 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1756/3485] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:46.339715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46aa9f-9057-40c8-af48-3422b85866a0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:46.340441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46aa9f-9057-40c8-af48-3422b85866a0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:46.340441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46aa9f-9057-40c8-af48-3422b85866a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:46.371535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9c46aa9f-9057-40c8-af48-3422b85866a0 None None] MFA Rules not processed for user `b4e0f20e52074142a3b525bafbb30c97`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:46.375494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c46aa9f-9057-40c8-af48-3422b85866a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:46.482394 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1730/3486] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:47.574724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c7cb-a1b0-41fe-a950-12b6cb05b655 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:47.575633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c7cb-a1b0-41fe-a950-12b6cb05b655 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:47.576067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c7cb-a1b0-41fe-a950-12b6cb05b655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:47.616372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-20c0c7cb-a1b0-41fe-a950-12b6cb05b655 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:47.621305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-20c0c7cb-a1b0-41fe-a950-12b6cb05b655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:47.646006 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1757/3487] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:50.548922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:50.549850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:50.550275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:50.572622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:50.573885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:50.586681 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1731/3488] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:51.967576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:51.968897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:51.969314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:52.000820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:52.002545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:52.118077 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1758/3489] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:07:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:52.127999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:52.132009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:52.272305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9d951-7b54-414e-83b9-65925b658c18 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:52.306963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b006f1c-764d-4838-8703-af594bd0a9e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:52.308082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b006f1c-764d-4838-8703-af594bd0a9e6 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:52.308645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b006f1c-764d-4838-8703-af594bd0a9e6 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:52.309123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b006f1c-764d-4838-8703-af594bd0a9e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:52.325494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:52.332635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b006f1c-764d-4838-8703-af594bd0a9e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:52.335116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b006f1c-764d-4838-8703-af594bd0a9e6 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:52.347226 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1732/3490] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:07:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:52.546039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:52.547555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:52.593367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8907a44-e0b9-45ad-a216-5d6e55606936 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:52.610695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:52.611900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:52.612360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:52.613003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] PATH_INFO: `/v3/users/866c1a2a2ca8452eb0a66cbe9692ab75` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:52.616156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] RBAC: Authorizing `identity:delete_user(user_id=866c1a2a2ca8452eb0a66cbe9692ab75)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:52.633941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:52.759613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866c1a2a2ca8452eb0a66cbe9692ab75'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:52.760532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866c1a2a2ca8452eb0a66cbe9692ab75'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:52.804673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866c1a2a2ca8452eb0a66cbe9692ab75'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:52.828352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866c1a2a2ca8452eb0a66cbe9692ab75'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:52.830292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866c1a2a2ca8452eb0a66cbe9692ab75'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:52.903232 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1759/3491] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:07:52 2022] DELETE /identity/v3/users/866c1a2a2ca8452eb0a66cbe9692ab75 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:53.728614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923f6d-0504-40df-a3d0-29479a237a5e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:53.729605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923f6d-0504-40df-a3d0-29479a237a5e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:53.730124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923f6d-0504-40df-a3d0-29479a237a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:53.791407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:53.802326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:53.805061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0f923f6d-0504-40df-a3d0-29479a237a5e None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:53.811706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f923f6d-0504-40df-a3d0-29479a237a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:54.000907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1733/3492] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:54.063027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb550dc7-864f-4f04-bce9-8b08b8ce907e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:54.082375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:54.084420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:54.089665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:54.090151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] PATH_INFO: `/v3/projects/513655e40ccd4152a2e343e5415ce347` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:54.092124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] RBAC: Authorizing `identity:delete_project(project_id=513655e40ccd4152a2e343e5415ce347)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:54.099724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:54.113614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:54.253011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513655e40ccd4152a2e343e5415ce347', 'request_id': 'req-4826624c-5200-4027-b4ed-68ea0d6bf16d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:54.274291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513655e40ccd4152a2e343e5415ce347', 'request_id': 'req-4826624c-5200-4027-b4ed-68ea0d6bf16d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:54.282623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513655e40ccd4152a2e343e5415ce347', 'request_id': 'req-4826624c-5200-4027-b4ed-68ea0d6bf16d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:54.318724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] The token cache is being invalidate because project 513655e40ccd4152a2e343e5415ce347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:07:54.319021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:54.324076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4826624c-5200-4027-b4ed-68ea0d6bf16d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:07:54.327682 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1760/3493] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:53 2022] DELETE /identity/v3/projects/513655e40ccd4152a2e343e5415ce347 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:07:54.340862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc77ecd-e15f-4f29-8681-9525cf67bc38 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:54.340862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc77ecd-e15f-4f29-8681-9525cf67bc38 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:54.340862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc77ecd-e15f-4f29-8681-9525cf67bc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:54.365224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fdc77ecd-e15f-4f29-8681-9525cf67bc38 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:54.369941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fdc77ecd-e15f-4f29-8681-9525cf67bc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:54.492866 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1734/3494] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:54.499946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcff64f-2243-49d7-93bc-34cae1f997ad None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:54.500235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcff64f-2243-49d7-93bc-34cae1f997ad None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:54.500571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcff64f-2243-49d7-93bc-34cae1f997ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:54.523727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-edcff64f-2243-49d7-93bc-34cae1f997ad None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:54.525418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-edcff64f-2243-49d7-93bc-34cae1f997ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:54.535925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1761/3495] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:54.540141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc77ecd-e15f-4f29-8681-9525cf67bc38 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:54.553199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc77ecd-e15f-4f29-8681-9525cf67bc38 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:54.569712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:54.569977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:54.570265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:54.570533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:54.571203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:07:54.571502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:54.573147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:54.604338 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1735/3496] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:07:54.609004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcff64f-2243-49d7-93bc-34cae1f997ad None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:54.624249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcff64f-2243-49d7-93bc-34cae1f997ad None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:54.639602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:54.640369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:54.641643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:54.642463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:54.645196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:54.647736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:54.733375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1762/3497] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:07:54 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:54.735078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:54.752151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f22ceb-a959-4133-aef5-3705146d0cec admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:54.971337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:54.972031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:54.972315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:54.972587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:54.973308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:54.979522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:55.052621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:55.059163 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1736/3498] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:07:54 2022] POST /identity/v3/users => generated 297 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:07:55.066305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47b4a6-24b6-4e35-9af7-a81c8607feab admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:55.113428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:55.114331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:55.114609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:55.114876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:55.115532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:55.117381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:55.153477 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1763/3499] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:07:55.157218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:55.171735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf143e4-e14e-4aaa-b1d1-e9953b7517b1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:55.188579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace6751-8982-4aa2-9d6d-b8e4188181ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:55.189180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace6751-8982-4aa2-9d6d-b8e4188181ee admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:55.189460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace6751-8982-4aa2-9d6d-b8e4188181ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:55.190076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace6751-8982-4aa2-9d6d-b8e4188181ee admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:55.190737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace6751-8982-4aa2-9d6d-b8e4188181ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:55.192777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace6751-8982-4aa2-9d6d-b8e4188181ee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:55.225778 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1737/3500] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:07:55.230084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:55.245736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8823-147d-4a6f-8570-ea563fe1984d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:55.251860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:55.252559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:55.252879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:55.253378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] PATH_INFO: `/v3/projects/7957b9f761414b969a3681f9d9a2945a/users/b4056bf8843c453a8f6ca05daef0e096/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:55.292807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7957b9f761414b969a3681f9d9a2945a, user_id=b4056bf8843c453a8f6ca05daef0e096, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:55.296933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:55.357256 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1764/3501] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:07:55 2022] PUT /identity/v3/projects/7957b9f761414b969a3681f9d9a2945a/users/b4056bf8843c453a8f6ca05daef0e096/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:07:55.363253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a384f-f1cf-44c0-bc12-47ef644e7cf3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:55.364038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a384f-f1cf-44c0-bc12-47ef644e7cf3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:55.364432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a384f-f1cf-44c0-bc12-47ef644e7cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:55.380585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:55.415638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a49a384f-f1cf-44c0-bc12-47ef644e7cf3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:07:55.417663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a49a384f-f1cf-44c0-bc12-47ef644e7cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:55.609048 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1738/3502] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:07:55.624491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:07:55.626675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:07:55.794142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65ec2-6c93-4236-98f2-3f7a1d79a358 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:07:55.808773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:07:55.809427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:07:55.809711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:07:55.810129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:07:55.830691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:07:55.832544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:07:55.854996 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1765/3503] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:03.691806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424729df-20cd-4092-a79c-6b6e5ed9463f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:03.692398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424729df-20cd-4092-a79c-6b6e5ed9463f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:03.692398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424729df-20cd-4092-a79c-6b6e5ed9463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:03.739672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-424729df-20cd-4092-a79c-6b6e5ed9463f None None] MFA Rules not processed for user `68d9297ca0884c2fa7f5283f722451ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:03.745657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-424729df-20cd-4092-a79c-6b6e5ed9463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:03.864359 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1739/3504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:04.363003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:04.376547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76a7b-b838-49b1-a42c-0578d72dac4d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:04.380518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:04.381276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:04.381547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:04.381836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:04.383977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:04.398032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:04.400981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:04.430466 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1766/3505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:10.049890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ffe20-5126-4f73-bc8b-3cc863332150 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:10.050933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ffe20-5126-4f73-bc8b-3cc863332150 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:10.051501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ffe20-5126-4f73-bc8b-3cc863332150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:10.112506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c37ffe20-5126-4f73-bc8b-3cc863332150 None None] MFA Rules not processed for user `b4056bf8843c453a8f6ca05daef0e096`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:10.121291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c37ffe20-5126-4f73-bc8b-3cc863332150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:10.234703 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1740/3506] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:10.684713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:10.688381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:10.770335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:10.998876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679296a4-6c19-4e5d-94f3-143daa9e3a32 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:11.029889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fb4b2-b708-4de5-9aff-284198168d6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:11.030562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fb4b2-b708-4de5-9aff-284198168d6d service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:11.030852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fb4b2-b708-4de5-9aff-284198168d6d service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:11.031114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fb4b2-b708-4de5-9aff-284198168d6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:11.038067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-768fb4b2-b708-4de5-9aff-284198168d6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:11.246633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fb4b2-b708-4de5-9aff-284198168d6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:11.253191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fb4b2-b708-4de5-9aff-284198168d6d service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:11.282852 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1767/3507] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:11.455797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ffe20-5126-4f73-bc8b-3cc863332150 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:11.478006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ffe20-5126-4f73-bc8b-3cc863332150 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:11.493567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402e01-e8b9-49f9-98f8-2d95d92de9eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:11.494538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402e01-e8b9-49f9-98f8-2d95d92de9eb service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:11.495051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402e01-e8b9-49f9-98f8-2d95d92de9eb service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:11.495507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402e01-e8b9-49f9-98f8-2d95d92de9eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:11.525722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402e01-e8b9-49f9-98f8-2d95d92de9eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:11.530734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402e01-e8b9-49f9-98f8-2d95d92de9eb service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:11.563836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1741/3508] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:12.122132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:12.122676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:12.122676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:12.172352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:12.173620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:12.303016 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1768/3509] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:08:14.759829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bdd55-03f8-4945-a9e7-a4ccf7f85fc2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:14.761092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bdd55-03f8-4945-a9e7-a4ccf7f85fc2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:14.763652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bdd55-03f8-4945-a9e7-a4ccf7f85fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:14.799781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1d8bdd55-03f8-4945-a9e7-a4ccf7f85fc2 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:14.802260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1d8bdd55-03f8-4945-a9e7-a4ccf7f85fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:14.811521 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1742/3510] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:16.642193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:16.645269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:16.762739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722cda7-f2a2-4f86-9862-ebcd34d4cb0f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:16.766897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:16.767560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:16.768128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:16.768613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:16.770749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:16.786115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:16.788038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:16.794348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:16.801152 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1769/3511] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:19.419042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:19.419617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:19.419617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:19.457465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:19.459468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:19.514358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:19.588020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1743/3512] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:08:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:19.608191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:19.609682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:19.694289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354665d-76a3-4104-b8c8-82bb0b48ba5f service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:19.720897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:19.722105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:19.722682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:19.723112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:19.740543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:19.743129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:19.755927 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1770/3513] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:08:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:20.407380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:20.409518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:20.444272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abe47-a7fd-4fdb-b475-3b0b58ac8962 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:20.452463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:20.453489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:20.454001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:20.455242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] PATH_INFO: `/v3/users/906b7b3d2366422cbc5da1f3080680f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:20.456950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=906b7b3d2366422cbc5da1f3080680f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:20.459805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:20.589496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906b7b3d2366422cbc5da1f3080680f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:20.594408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906b7b3d2366422cbc5da1f3080680f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:20.612411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906b7b3d2366422cbc5da1f3080680f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:20.633180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906b7b3d2366422cbc5da1f3080680f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:20.663617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7368a14a-550f-4237-b4e5-8b389cc579fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906b7b3d2366422cbc5da1f3080680f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:20.677432 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1744/3514] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:08:20 2022] DELETE /identity/v3/users/906b7b3d2366422cbc5da1f3080680f6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:21.670418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:21.689090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e573f-6104-4af9-8e5d-fca66f2748bb service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:21.864976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:21.866506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:21.866839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:21.867116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] PATH_INFO: `/v3/projects/462859c44a93480aba0df9344b789348` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:21.873014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=462859c44a93480aba0df9344b789348)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:21.878447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:21.994163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462859c44a93480aba0df9344b789348', 'request_id': 'req-d83e4dd3-039f-409d-984e-23a97366a188'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:22.009176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462859c44a93480aba0df9344b789348', 'request_id': 'req-d83e4dd3-039f-409d-984e-23a97366a188'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:22.017480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462859c44a93480aba0df9344b789348', 'request_id': 'req-d83e4dd3-039f-409d-984e-23a97366a188'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:22.040846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] The token cache is being invalidate because project 462859c44a93480aba0df9344b789348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:08:22.041146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:22.045224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d83e4dd3-039f-409d-984e-23a97366a188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:22.048255 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1771/3515] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:08:21 2022] DELETE /identity/v3/projects/462859c44a93480aba0df9344b789348 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:22.064743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab1b95-20dc-4c5d-b60d-0be003c38dc9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:22.064743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab1b95-20dc-4c5d-b60d-0be003c38dc9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:22.064743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab1b95-20dc-4c5d-b60d-0be003c38dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:22.114895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d8ab1b95-20dc-4c5d-b60d-0be003c38dc9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:22.116243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d8ab1b95-20dc-4c5d-b60d-0be003c38dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:22.318389 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1745/3516] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:22.321515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826fa2-593f-4b4f-9795-2511ba5fbb70 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:22.321840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826fa2-593f-4b4f-9795-2511ba5fbb70 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:22.322334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826fa2-593f-4b4f-9795-2511ba5fbb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:22.369766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d3826fa2-593f-4b4f-9795-2511ba5fbb70 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:22.371173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3826fa2-593f-4b4f-9795-2511ba5fbb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:22.398711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1772/3517] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:08:22.400931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab1b95-20dc-4c5d-b60d-0be003c38dc9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:22.413847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab1b95-20dc-4c5d-b60d-0be003c38dc9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:22.429355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:22.430812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:22.431101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:22.431380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:22.432067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:08:22.435273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:22.437077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:22.481909 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1746/3518] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:08:22.483528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826fa2-593f-4b4f-9795-2511ba5fbb70 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:22.494566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826fa2-593f-4b4f-9795-2511ba5fbb70 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:22.505472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:22.506119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:22.506447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:22.506669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:22.507425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:22.515127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:22.546758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:22.602651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:22.614730 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1773/3519] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:08:22 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:22.617512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb669e2-85bc-4f56-9d35-95c02f87b8af admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:22.676119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:22.876885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:22.878203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:22.878894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:22.879507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:22.880824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:22.885086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:22.928706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:22.943998 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1747/3520] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:08:22 2022] POST /identity/v3/users => generated 296 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:22.950194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b60c78-1959-43b6-98fb-841dac57a37c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:22.956195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:22.957144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:22.957438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:22.957716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:22.958803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:22.960739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:22.980757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:22.991352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1774/3521] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:08:22.997713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543c9e-3f0e-4fd6-a7fa-095a96b49827 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:23.005810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b9d4a-fd7f-4b2b-b19f-27556e0a8df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:23.006688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9d4a-fd7f-4b2b-b19f-27556e0a8df2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:23.006969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9d4a-fd7f-4b2b-b19f-27556e0a8df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:23.007264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9d4a-fd7f-4b2b-b19f-27556e0a8df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:23.007929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b9d4a-fd7f-4b2b-b19f-27556e0a8df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:23.010526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b9d4a-fd7f-4b2b-b19f-27556e0a8df2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:23.023663 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1748/3522] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:08:23.028462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:23.043876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70c190-14f5-4174-8f5e-bae5f85861b4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:23.049191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:23.050178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:23.050655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:23.051126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] PATH_INFO: `/v3/projects/79022e5ca5764576b4feb7f4cadba80c/users/2699ef01c0eb4d5783203e6b41293d97/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:23.099654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79022e5ca5764576b4feb7f4cadba80c, user_id=2699ef01c0eb4d5783203e6b41293d97, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:23.106292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:23.144909 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1775/3523] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:08:23 2022] PUT /identity/v3/projects/79022e5ca5764576b4feb7f4cadba80c/users/2699ef01c0eb4d5783203e6b41293d97/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:23.157348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:23.157861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:23.158072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:23.250199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:23.251488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:23.445604 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1749/3524] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:23.492600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:23.495891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:23.789742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5cc46-1b12-4eff-80df-460723a20456 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:23.802373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:23.803061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:23.803340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:23.803611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:23.819598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:23.821039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:23.844183 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1776/3525] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:26.848142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:26.851415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:26.895332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee8b9-475f-4294-a03c-f5229f2445c4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:26.913223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:26.916755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:26.917239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:26.917858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] PATH_INFO: `/v3/users/b4056bf8843c453a8f6ca05daef0e096` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:26.919354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4056bf8843c453a8f6ca05daef0e096)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:26.922751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:27.066602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4056bf8843c453a8f6ca05daef0e096'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.068312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4056bf8843c453a8f6ca05daef0e096'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.091014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4056bf8843c453a8f6ca05daef0e096'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.107026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4056bf8843c453a8f6ca05daef0e096'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.140701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4056bf8843c453a8f6ca05daef0e096'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.153045 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1750/3526] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:08:26 2022] DELETE /identity/v3/users/b4056bf8843c453a8f6ca05daef0e096 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:27.359738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:27.362099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:27.393248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924018fc-4d4d-4dc0-be7a-a7275cadb737 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:27.513834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:27.525363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:27.525647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:27.526343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] PATH_INFO: `/v3/users/430de2723f234fac8f09068195d8ce2c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:27.530123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=430de2723f234fac8f09068195d8ce2c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:27.532895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:27.659022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430de2723f234fac8f09068195d8ce2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.661616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430de2723f234fac8f09068195d8ce2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.677255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430de2723f234fac8f09068195d8ce2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.689624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:27.695695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430de2723f234fac8f09068195d8ce2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.696772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27ac87d3-c399-4079-8fc9-415dafbe75fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430de2723f234fac8f09068195d8ce2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.718567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:27.721149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:27.750086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e50053-6665-4181-8801-2a2fa1280aa6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:27.770768 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1777/3527] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:08:27 2022] DELETE /identity/v3/users/430de2723f234fac8f09068195d8ce2c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:27.777790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:27.779466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:27.782441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:27.783378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] PATH_INFO: `/v3/projects/7957b9f761414b969a3681f9d9a2945a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:27.786745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7957b9f761414b969a3681f9d9a2945a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:27.799476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:27.843326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:27.932102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7957b9f761414b969a3681f9d9a2945a', 'request_id': 'req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.953126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7957b9f761414b969a3681f9d9a2945a', 'request_id': 'req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.977266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7957b9f761414b969a3681f9d9a2945a', 'request_id': 'req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:27.997366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] The token cache is being invalidate because project 7957b9f761414b969a3681f9d9a2945a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:08:27.997582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:28.001077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0edd371-68a6-46a3-9bd2-3a4c3c4ad1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:28.006917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1751/3528] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:08:27 2022] DELETE /identity/v3/projects/7957b9f761414b969a3681f9d9a2945a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:08:28.012803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe891-8442-4cf7-ae9b-c726dadcde63 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:28.013108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe891-8442-4cf7-ae9b-c726dadcde63 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:28.013360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe891-8442-4cf7-ae9b-c726dadcde63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:28.056459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0cfbe891-8442-4cf7-ae9b-c726dadcde63 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:28.057843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0cfbe891-8442-4cf7-ae9b-c726dadcde63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:28.221051 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1778/3529] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:28.228351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09d87-2f8a-4aa0-b81b-efc6d15ef40c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:28.228636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09d87-2f8a-4aa0-b81b-efc6d15ef40c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:28.228970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09d87-2f8a-4aa0-b81b-efc6d15ef40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:28.289699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a8a09d87-2f8a-4aa0-b81b-efc6d15ef40c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:28.290707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8a09d87-2f8a-4aa0-b81b-efc6d15ef40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:28.324877 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1752/3530] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:28.327914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbe891-8442-4cf7-ae9b-c726dadcde63 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:28.363144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbe891-8442-4cf7-ae9b-c726dadcde63 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:28.375393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:28.376051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:28.376382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:28.376677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:28.383788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:08:28.384107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:28.386707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:28.448598 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1779/3531] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:08:28.453374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a09d87-2f8a-4aa0-b81b-efc6d15ef40c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:28.468902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a09d87-2f8a-4aa0-b81b-efc6d15ef40c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:28.483736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:28.484373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:28.484739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:28.484962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:28.485604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:28.487545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:28.593285 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1753/3532] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:08:28 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:28.598324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:28.621682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c50f2-2959-403f-8114-0c9937ff5741 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:28.660885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:28.663428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:28.773138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:28.774124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:28.774256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:28.774540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:28.776033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:28.832194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:28.888252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502566d2-2dae-4251-a2f9-245cc1401b06 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:28.911388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:28.912340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:28.912987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:28.913315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] PATH_INFO: `/v3/projects/fedcdffc12b94b489ce433d5caa5e4be` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:28.914653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fedcdffc12b94b489ce433d5caa5e4be)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:28.917544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:28.930766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1780/3533] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:08:28 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:28.940777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:28.962633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87194b-a78f-491c-a893-5e0bdcc8362d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:28.979736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:28.980387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:28.980965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:28.981252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:28.981909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:28.983544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:28.991178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fedcdffc12b94b489ce433d5caa5e4be', 'request_id': 'req-0435b781-33f3-4664-b45b-a4391a4d90c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:29.000552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fedcdffc12b94b489ce433d5caa5e4be', 'request_id': 'req-0435b781-33f3-4664-b45b-a4391a4d90c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:29.021080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fedcdffc12b94b489ce433d5caa5e4be', 'request_id': 'req-0435b781-33f3-4664-b45b-a4391a4d90c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:29.028299 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1781/3534] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:08:29.032023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:29.032163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:29.070785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] The token cache is being invalidate because project fedcdffc12b94b489ce433d5caa5e4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:08:29.071079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:29.072007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0435b781-33f3-4664-b45b-a4391a4d90c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:29.074337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1754/3535] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:08:28 2022] DELETE /identity/v3/projects/fedcdffc12b94b489ce433d5caa5e4be => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:29.079593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e381b-3c5f-4093-9290-8f3e747702d5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:29.080411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e381b-3c5f-4093-9290-8f3e747702d5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:29.081276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e381b-3c5f-4093-9290-8f3e747702d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:29.081897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d66e4-cfcf-4387-81d3-46884b483e64 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:29.178580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3c1e381b-3c5f-4093-9290-8f3e747702d5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:29.181415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3c1e381b-3c5f-4093-9290-8f3e747702d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:29.184495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:29.254069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:29.255207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:29.257804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:29.258351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:29.259596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:29.275823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:29.295403 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1782/3536] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:08:29.299336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:29.308253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:29.351884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0c13a-89b1-4e5a-bb89-52edf9253442 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:29.369640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4ac93-6c61-4f28-a678-da9f721cc1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:29.370329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4ac93-6c61-4f28-a678-da9f721cc1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:29.370602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4ac93-6c61-4f28-a678-da9f721cc1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:29.370882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4ac93-6c61-4f28-a678-da9f721cc1f1 admin admin] PATH_INFO: `/v3/projects/327b414889a04e189a7f940685a4ff39/users/f06d16c90a5d464696f8137612c35327/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:29.409938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4ac93-6c61-4f28-a678-da9f721cc1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327b414889a04e189a7f940685a4ff39, user_id=f06d16c90a5d464696f8137612c35327, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:29.412746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4ac93-6c61-4f28-a678-da9f721cc1f1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:29.438673 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1755/3537] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:29.443897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498143-20e6-4eb3-b3c1-37c62586c677 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:29.443897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498143-20e6-4eb3-b3c1-37c62586c677 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:29.443897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498143-20e6-4eb3-b3c1-37c62586c677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:29.444706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1783/3538] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:08:29 2022] PUT /identity/v3/projects/327b414889a04e189a7f940685a4ff39/users/f06d16c90a5d464696f8137612c35327/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:29.446460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:29.446904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:29.447353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:29.494285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:29.495728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:29.507113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e5498143-20e6-4eb3-b3c1-37c62586c677 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:29.509536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e5498143-20e6-4eb3-b3c1-37c62586c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:29.651886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1756/3539] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:29.652571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5498143-20e6-4eb3-b3c1-37c62586c677 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:29.674072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5498143-20e6-4eb3-b3c1-37c62586c677 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:29.695156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:29.695925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:29.696276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:29.696624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:29.697414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:08:29.697811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:29.699518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:29.718728 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1757/3540] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:08:29.729719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:29.735532 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1784/3541] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:29.749588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea111dab-3aa1-4934-9673-960e2b720069 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:29.782414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:29.783245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:29.783681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:29.783970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:29.784756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:29.787023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:29.843489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:29.845347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:29.889622 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1758/3542] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:08:29 2022] POST /identity/v3/projects => generated 395 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:29.893049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:30.037707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9d166-39ee-4949-8d64-d17c3f654509 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:30.099723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03acfa-08e0-4566-a765-eecc6b4d90c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:30.113254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:30.114215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:30.114508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:30.114863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:30.115617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:30.118259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:30.133428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:30.179142 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1759/3543] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:08:29 2022] POST /identity/v3/users => generated 298 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:30.182339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:30.216948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672f53-9e61-419c-ac93-375b21df3fe1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:30.224061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:30.226876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:30.227533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:30.233043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:30.233705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:30.234389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:30.235377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:30.236700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:30.238755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:30.241993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:30.258314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:30.263183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:30.270187 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1760/3544] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:08:30.274555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:30.300794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1da9b-293d-4b10-aeed-99ab8728e2ff admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:30.305121 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1785/3545] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:30.308981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb79b48-6c47-4e71-8c27-0e528285a0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:30.310154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb79b48-6c47-4e71-8c27-0e528285a0cf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:30.310456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb79b48-6c47-4e71-8c27-0e528285a0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:30.310900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb79b48-6c47-4e71-8c27-0e528285a0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:30.312399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb79b48-6c47-4e71-8c27-0e528285a0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:30.316197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb79b48-6c47-4e71-8c27-0e528285a0cf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:30.365007 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1761/3546] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:08:30.370190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:30.387050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4725-0581-4716-b2a4-07925c77c298 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:30.404722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d3dc5-f25b-4756-8662-3e8296a9e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:30.406008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3dc5-f25b-4756-8662-3e8296a9e513 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:30.406644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3dc5-f25b-4756-8662-3e8296a9e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:30.407264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3dc5-f25b-4756-8662-3e8296a9e513 admin admin] PATH_INFO: `/v3/projects/5388b7c5e001457c9ab07206893a86e8/users/d65756281cb34a258d31b6ead74411aa/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:30.435428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:30.442301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d3dc5-f25b-4756-8662-3e8296a9e513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5388b7c5e001457c9ab07206893a86e8, user_id=d65756281cb34a258d31b6ead74411aa, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:30.450120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d3dc5-f25b-4756-8662-3e8296a9e513 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:30.489641 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1786/3547] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:08:30 2022] PUT /identity/v3/projects/5388b7c5e001457c9ab07206893a86e8/users/d65756281cb34a258d31b6ead74411aa/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:30.500269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:30.500537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:30.500895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:30.589020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] MFA Rules not processed for user `d65756281cb34a258d31b6ead74411aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:30.590619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:30.775552 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1762/3548] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:30.786233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838e35f-df42-4c91-9c6a-398c0d73ced2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:30.786720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838e35f-df42-4c91-9c6a-398c0d73ced2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:30.787320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838e35f-df42-4c91-9c6a-398c0d73ced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:30.825540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f838e35f-df42-4c91-9c6a-398c0d73ced2 None None] MFA Rules not processed for user `d65756281cb34a258d31b6ead74411aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:30.827428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f838e35f-df42-4c91-9c6a-398c0d73ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:30.845269 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1787/3549] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:31.211142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:31.214435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:31.403286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a20aa0-2046-4019-a157-692648f05cde None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:31.410142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb073d66-7f67-4042-94b5-60192f4f88a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:31.410956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb073d66-7f67-4042-94b5-60192f4f88a5 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:31.411405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb073d66-7f67-4042-94b5-60192f4f88a5 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:31.412023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb073d66-7f67-4042-94b5-60192f4f88a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:31.423158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb073d66-7f67-4042-94b5-60192f4f88a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:31.424959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb073d66-7f67-4042-94b5-60192f4f88a5 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:31.453886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1763/3550] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:31 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:31.470533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4422-fd3a-4769-b2f1-aa5660ce2186 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:31.471241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4422-fd3a-4769-b2f1-aa5660ce2186 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:31.471241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e4422-fd3a-4769-b2f1-aa5660ce2186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:31.511314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-902e4422-fd3a-4769-b2f1-aa5660ce2186 None None] MFA Rules not processed for user `d65756281cb34a258d31b6ead74411aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:31.512582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-902e4422-fd3a-4769-b2f1-aa5660ce2186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:31.549410 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1788/3551] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:31.590069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf668ab-e0f4-4dd5-919d-169aa5b33101 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:31.590450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf668ab-e0f4-4dd5-919d-169aa5b33101 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:31.590797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf668ab-e0f4-4dd5-919d-169aa5b33101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:31.629074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ebf668ab-e0f4-4dd5-919d-169aa5b33101 None None] MFA Rules not processed for user `d65756281cb34a258d31b6ead74411aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:31.631113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ebf668ab-e0f4-4dd5-919d-169aa5b33101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:31.646669 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1764/3552] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:08:31.689051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79449-dab3-4ed6-b3fd-2ddaada4e062 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:31.689683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79449-dab3-4ed6-b3fd-2ddaada4e062 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:31.690186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc79449-dab3-4ed6-b3fd-2ddaada4e062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:31.730563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5fc79449-dab3-4ed6-b3fd-2ddaada4e062 None None] MFA Rules not processed for user `d65756281cb34a258d31b6ead74411aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:31.737582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5fc79449-dab3-4ed6-b3fd-2ddaada4e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:31.761241 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1789/3553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:31.809918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c8df-0b48-4789-8e17-79b75cedad58 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:31.810251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c8df-0b48-4789-8e17-79b75cedad58 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:31.810481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c8df-0b48-4789-8e17-79b75cedad58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:31.873189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-12c4c8df-0b48-4789-8e17-79b75cedad58 None None] MFA Rules not processed for user `d65756281cb34a258d31b6ead74411aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:31.881057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-12c4c8df-0b48-4789-8e17-79b75cedad58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:31.915046 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1765/3554] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:32.143651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc79449-dab3-4ed6-b3fd-2ddaada4e062 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:32.344458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc79449-dab3-4ed6-b3fd-2ddaada4e062 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:32.364565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:32.369407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:32.371106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:32.372316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] PATH_INFO: `/v3/users/d65756281cb34a258d31b6ead74411aa` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:32.380562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d65756281cb34a258d31b6ead74411aa)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:32.386497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:32.553266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd65756281cb34a258d31b6ead74411aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:32.560874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd65756281cb34a258d31b6ead74411aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:32.583263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd65756281cb34a258d31b6ead74411aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:32.603388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd65756281cb34a258d31b6ead74411aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:32.607345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd65756281cb34a258d31b6ead74411aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:32.688702 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1790/3555] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:08:32 2022] DELETE /identity/v3/users/d65756281cb34a258d31b6ead74411aa => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:32.695329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:32.695904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:32.696261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:32.740542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:32.743594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:32.861306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:32.943632 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1766/3556] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:08:33.088880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:33.090780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:33.214467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9abb-1e6e-488c-8d81-2d46d046c521 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:33.218914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc9ad0-f1a0-4ab3-9099-f3b681f0668a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:33.219551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc9ad0-f1a0-4ab3-9099-f3b681f0668a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:33.220036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc9ad0-f1a0-4ab3-9099-f3b681f0668a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:33.220405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc9ad0-f1a0-4ab3-9099-f3b681f0668a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:33.229838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:33.236045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc9ad0-f1a0-4ab3-9099-f3b681f0668a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:33.239369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc9ad0-f1a0-4ab3-9099-f3b681f0668a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:33.261576 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1791/3557] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:34.308435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:34.310132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:34.330145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2122a3-16fb-40e6-bfbe-57628b163295 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:34.338762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:34.339711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:34.340142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:34.341193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] PATH_INFO: `/v3/projects/5388b7c5e001457c9ab07206893a86e8` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:34.342833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5388b7c5e001457c9ab07206893a86e8)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:34.346548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:34.433210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5388b7c5e001457c9ab07206893a86e8', 'request_id': 'req-2f843f91-5024-4f39-83d3-a8b72c23553e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:34.442341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5388b7c5e001457c9ab07206893a86e8', 'request_id': 'req-2f843f91-5024-4f39-83d3-a8b72c23553e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:34.465218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5388b7c5e001457c9ab07206893a86e8', 'request_id': 'req-2f843f91-5024-4f39-83d3-a8b72c23553e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:34.479135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] The token cache is being invalidate because project 5388b7c5e001457c9ab07206893a86e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:08:34.479720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:34.480707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2f843f91-5024-4f39-83d3-a8b72c23553e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:34.482971 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1767/3558] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:08:34 2022] DELETE /identity/v3/projects/5388b7c5e001457c9ab07206893a86e8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:34.487838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9182e-cb47-43a1-9e58-a06e141c221f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:34.488339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9182e-cb47-43a1-9e58-a06e141c221f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:34.488549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9182e-cb47-43a1-9e58-a06e141c221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:34.513440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e3c9182e-cb47-43a1-9e58-a06e141c221f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:34.515237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e3c9182e-cb47-43a1-9e58-a06e141c221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:34.723707 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1792/3559] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:08:34.725041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36119b-ddea-4834-9ba8-49d5b2d048c1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:34.725314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36119b-ddea-4834-9ba8-49d5b2d048c1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:34.725572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36119b-ddea-4834-9ba8-49d5b2d048c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:34.764433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0f36119b-ddea-4834-9ba8-49d5b2d048c1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:34.765123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f36119b-ddea-4834-9ba8-49d5b2d048c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:34.789886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1768/3560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:34.793841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9182e-cb47-43a1-9e58-a06e141c221f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:34.808557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9182e-cb47-43a1-9e58-a06e141c221f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:34.838925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:34.839579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:34.839845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:34.840115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:34.840803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:08:34.841109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:34.859524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:34.906435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1793/3561] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:08:34.911444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36119b-ddea-4834-9ba8-49d5b2d048c1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:34.930512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36119b-ddea-4834-9ba8-49d5b2d048c1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:34.949792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:34.950475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:34.950820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:34.951368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:34.952238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:34.954222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:35.029096 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1769/3562] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:08:34 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:35.029839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:35.048956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079954e-4fe3-4344-9541-eb5d877c74da admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:35.258143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:35.258875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:35.259151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:35.259398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:35.260048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:35.265671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:35.347719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:35.348387 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1794/3563] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:08:35 2022] POST /identity/v3/users => generated 290 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:08:35.368810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb152be-f3b2-4e3b-81bf-f623596af1cd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:35.390660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:35.391342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:35.391620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:35.391895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:35.392554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:35.394303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:35.437456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:35.437800 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1770/3564] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:08:35.455319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608868e-ac7a-4f3c-8a43-65d86de96366 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:35.471011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442e4ad-0b44-45bd-9eef-21449fe9e992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:35.471750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442e4ad-0b44-45bd-9eef-21449fe9e992 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:35.472237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442e4ad-0b44-45bd-9eef-21449fe9e992 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:35.475113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442e4ad-0b44-45bd-9eef-21449fe9e992 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:35.476154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442e4ad-0b44-45bd-9eef-21449fe9e992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:35.477922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442e4ad-0b44-45bd-9eef-21449fe9e992 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:35.509008 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1795/3565] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:08:35.513051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:35.529176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9b457-7635-48be-be59-9f620e79ff20 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:35.535897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f8a11-0717-40c1-893b-c92e77695450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:35.536941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8a11-0717-40c1-893b-c92e77695450 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:35.537625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8a11-0717-40c1-893b-c92e77695450 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:35.538116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8a11-0717-40c1-893b-c92e77695450 admin admin] PATH_INFO: `/v3/projects/2b032e83813a429bb445b5f7ee854a4f/users/bf823d90bd69466691ba2094929a8239/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:35.566517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f8a11-0717-40c1-893b-c92e77695450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b032e83813a429bb445b5f7ee854a4f, user_id=bf823d90bd69466691ba2094929a8239, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:35.570066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f8a11-0717-40c1-893b-c92e77695450 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:35.598340 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1771/3566] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:08:35 2022] PUT /identity/v3/projects/2b032e83813a429bb445b5f7ee854a4f/users/bf823d90bd69466691ba2094929a8239/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:35.604500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:35.605133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:35.605610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:35.640501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:35.646333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:35.796638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:35.796900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:35.797133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:35.805414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:35.854787 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1796/3567] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:35.867581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:35.870137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:35.886277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] MFA Rules not processed for user `2699ef01c0eb4d5783203e6b41293d97`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:35.889953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:36.031368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:36.036545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbb811-db34-43b4-a8ff-8f426490428e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:36.053272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bdd6f-0c33-40d9-8585-ffdfaef97585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:36.055588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bdd6f-0c33-40d9-8585-ffdfaef97585 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:36.056292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bdd6f-0c33-40d9-8585-ffdfaef97585 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:36.056571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bdd6f-0c33-40d9-8585-ffdfaef97585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:36.070085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bdd6f-0c33-40d9-8585-ffdfaef97585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:36.071943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bdd6f-0c33-40d9-8585-ffdfaef97585 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:36.093727 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1797/3568] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:36.105614 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1772/3569] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:36.107771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6031c-5646-4b72-9394-44b1367e62d6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:36.108044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6031c-5646-4b72-9394-44b1367e62d6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:36.108310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6031c-5646-4b72-9394-44b1367e62d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:36.168347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5de6031c-5646-4b72-9394-44b1367e62d6 None None] MFA Rules not processed for user `2699ef01c0eb4d5783203e6b41293d97`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:36.173423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5de6031c-5646-4b72-9394-44b1367e62d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:36.199819 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1798/3570] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:36.306021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:36.307541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:36.445494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fd3d2-f2ba-4111-910d-9b539a8f9cd2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:36.454728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:36.456005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:36.456803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:36.458066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:36.468449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:36.470621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:36.493081 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1773/3571] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:41.076177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642788b8-cf35-42df-a0a6-4b0fb18c6492 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:41.076649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642788b8-cf35-42df-a0a6-4b0fb18c6492 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:41.076966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642788b8-cf35-42df-a0a6-4b0fb18c6492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:41.131222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-642788b8-cf35-42df-a0a6-4b0fb18c6492 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:41.134508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-642788b8-cf35-42df-a0a6-4b0fb18c6492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:41.447439 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1799/3572] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:41.486841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:41.488626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:41.511263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f781dd-549b-4645-ab76-be0753a74c64 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:41.511745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f781dd-549b-4645-ab76-be0753a74c64 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:41.512161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f781dd-549b-4645-ab76-be0753a74c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:41.539929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-45f781dd-549b-4645-ab76-be0753a74c64 None None] MFA Rules not processed for user `92b0186cd92744dabd001b86560b5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:41.541684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-45f781dd-549b-4645-ab76-be0753a74c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:41.686452 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1800/3573] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:08:41.764415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dea460-7355-42b9-b967-aa3302a0c191 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:41.774392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:41.775222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:41.775647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:41.776025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:41.789539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:41.792783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:41.837196 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1774/3574] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:08:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:45.344466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88f13e-2b66-4035-bbe4-c670acf6aee2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:45.345048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88f13e-2b66-4035-bbe4-c670acf6aee2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:45.345750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88f13e-2b66-4035-bbe4-c670acf6aee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:45.377111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1e88f13e-2b66-4035-bbe4-c670acf6aee2 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:45.377111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e88f13e-2b66-4035-bbe4-c670acf6aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:45.383482 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1801/3575] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:08:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:45.404091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:45.423155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be77da-468b-46db-98bb-329a0d9f4cea service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:45.438065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:45.439312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:45.439782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:45.440273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:45.454433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:45.456461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:45.490224 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1775/3576] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:08:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:08:48.961949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:48.962520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:48.962902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:49.122334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] MFA Rules not processed for user `f06d16c90a5d464696f8137612c35327`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:49.125282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:49.266746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:49.359420 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1802/3577] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:49.840327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:49.841932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:49.873028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f5c52-8bb1-4713-89c8-4dad5f84b468 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:49.900730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bd02f-10a3-46e3-a6c3-a658b7204c46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:49.906018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bd02f-10a3-46e3-a6c3-a658b7204c46 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:49.907330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bd02f-10a3-46e3-a6c3-a658b7204c46 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:49.911128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bd02f-10a3-46e3-a6c3-a658b7204c46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:49.949177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bd02f-10a3-46e3-a6c3-a658b7204c46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:49.950957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bd02f-10a3-46e3-a6c3-a658b7204c46 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:50.021690 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1776/3578] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:50.861710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:50.863454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:50.880316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a3840-0b6e-472c-805e-84dfb730c10b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:50.887593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:50.888458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:50.889030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:50.889526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] PATH_INFO: `/v3/users/f7bf3532492e43719ce03e29f9740b5f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:50.891053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bf3532492e43719ce03e29f9740b5f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:50.893715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:51.028086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bf3532492e43719ce03e29f9740b5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:51.029524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bf3532492e43719ce03e29f9740b5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:51.059682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bf3532492e43719ce03e29f9740b5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:51.075796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bf3532492e43719ce03e29f9740b5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:51.077029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bf3532492e43719ce03e29f9740b5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:51.110086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1803/3579] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:08:50 2022] DELETE /identity/v3/users/f7bf3532492e43719ce03e29f9740b5f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:51.111717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:51.112263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:51.112263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:51.170182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:51.172368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:51.244531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:51.312842 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1777/3580] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:51.344712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:51.346823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:51.686095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec880bb-f85a-4790-8d7e-0d7ab6bb2b9b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:51.694455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:51.695288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:51.695635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:51.695908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:51.720936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:51.722418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:51.744511 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1804/3581] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:54.249653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:54.251141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:54.267673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1802ce-a719-4670-bb96-d352b8620624 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:54.276763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:54.278049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:54.278474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:54.278900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] PATH_INFO: `/v3/projects/7fb866af50c14a0393c03c1463330238` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:54.287751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb866af50c14a0393c03c1463330238)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:54.294349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:54.475005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb866af50c14a0393c03c1463330238', 'request_id': 'req-33080b11-259b-456e-874d-910b3ce10d58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.489997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb866af50c14a0393c03c1463330238', 'request_id': 'req-33080b11-259b-456e-874d-910b3ce10d58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.508329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb866af50c14a0393c03c1463330238', 'request_id': 'req-33080b11-259b-456e-874d-910b3ce10d58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.530417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] The token cache is being invalidate because project 7fb866af50c14a0393c03c1463330238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:08:54.530715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.533964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33080b11-259b-456e-874d-910b3ce10d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.541140 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1778/3582] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:08:54 2022] DELETE /identity/v3/projects/7fb866af50c14a0393c03c1463330238 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:54.545686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:54.565091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f352d7-bbd6-482f-ab41-0da428b95768 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:54.674112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:54.703604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:54.704268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:54.704546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:54.705031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] PATH_INFO: `/v3/users/e6764e58f49944c48f8310960aa58d32` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:54.714210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6764e58f49944c48f8310960aa58d32)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:54.716474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:54.837276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6764e58f49944c48f8310960aa58d32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.843318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6764e58f49944c48f8310960aa58d32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.861693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6764e58f49944c48f8310960aa58d32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.881306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6764e58f49944c48f8310960aa58d32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.882309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6764e58f49944c48f8310960aa58d32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:54.924788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1805/3583] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:08:54 2022] DELETE /identity/v3/users/e6764e58f49944c48f8310960aa58d32 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:57.651530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:57.652729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:57.653258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:57.749804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] MFA Rules not processed for user `bf823d90bd69466691ba2094929a8239`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:57.755654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:57.927551 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1779/3584] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:57.938917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:57.942965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:58.153214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82469ef9-4234-443f-8c30-67d28b5d8344 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:58.166515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd2dc0-5b8c-4aed-ac1c-7f33b8913bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:58.167145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2dc0-5b8c-4aed-ac1c-7f33b8913bb7 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:58.168355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2dc0-5b8c-4aed-ac1c-7f33b8913bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:58.170012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2dc0-5b8c-4aed-ac1c-7f33b8913bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:58.209436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd2dc0-5b8c-4aed-ac1c-7f33b8913bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:58.213251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd2dc0-5b8c-4aed-ac1c-7f33b8913bb7 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:58.238974 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1806/3585] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:08:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:58.359454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:58.361315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:58.542485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:58.587665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe123c-d437-4d77-971d-dc6c04f40b15 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:58.605908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:58.606763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:58.607138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:58.607515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] PATH_INFO: `/v3/projects/881e25ee5abe4c2c91f0edd74c934a9b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:58.608979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=881e25ee5abe4c2c91f0edd74c934a9b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:58.612561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:58.777757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881e25ee5abe4c2c91f0edd74c934a9b', 'request_id': 'req-11c6c312-4ae7-4143-90a8-59cfac7febbd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:58.799519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881e25ee5abe4c2c91f0edd74c934a9b', 'request_id': 'req-11c6c312-4ae7-4143-90a8-59cfac7febbd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:58.822723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881e25ee5abe4c2c91f0edd74c934a9b', 'request_id': 'req-11c6c312-4ae7-4143-90a8-59cfac7febbd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:58.864340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] The token cache is being invalidate because project 881e25ee5abe4c2c91f0edd74c934a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:08:58.867970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:58.869239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-11c6c312-4ae7-4143-90a8-59cfac7febbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:58.873701 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1780/3586] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:08:58 2022] DELETE /identity/v3/projects/881e25ee5abe4c2c91f0edd74c934a9b => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:08:58.879232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9ae1-0fee-4322-bb86-5712f720bb32 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:58.879670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9ae1-0fee-4322-bb86-5712f720bb32 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:58.879951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9ae1-0fee-4322-bb86-5712f720bb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:58.923680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-43fb9ae1-0fee-4322-bb86-5712f720bb32 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:58.924955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-43fb9ae1-0fee-4322-bb86-5712f720bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:59.086251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:59.158861 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1807/3587] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:59.160997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ed44e-4b86-4f7b-a75f-eab1d50bb50b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:59.161538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ed44e-4b86-4f7b-a75f-eab1d50bb50b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:59.161930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ed44e-4b86-4f7b-a75f-eab1d50bb50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:59.192640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:59.193529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:59.193949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:59.222225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8c3ed44e-4b86-4f7b-a75f-eab1d50bb50b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:59.227839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8c3ed44e-4b86-4f7b-a75f-eab1d50bb50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:59.251037 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1781/3588] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:59.253009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ed44e-4b86-4f7b-a75f-eab1d50bb50b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:59.264323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] MFA Rules not processed for user `f06d16c90a5d464696f8137612c35327`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:08:59.266648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:59.281709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ed44e-4b86-4f7b-a75f-eab1d50bb50b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:59.294416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:59.318367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:59.322239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:59.323649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:59.324441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:08:59.326585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:59.331725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:59.385343 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1782/3589] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:08:59.390093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:59.422843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b2c1-0482-44a2-be2f-881bef54af68 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:59.448712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:59.450189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:59.453986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:59.454824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:59.455552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:59.467001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:59.613457 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1808/3590] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:08:59.616376 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1783/3591] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:08:59 2022] POST /identity/v3/projects => generated 395 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:59.617448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:59.621472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:59.622647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:59.661892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d28c8-53b8-49a6-b111-f5a04c6c10f8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:59.815791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:59.823071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04226f-efef-4f6b-abc8-f35e8797a52a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:08:59.825677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:59.826124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:59.826518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] PATH_INFO: `/v3/users/68d9297ca0884c2fa7f5283f722451ba` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:59.828313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d9297ca0884c2fa7f5283f722451ba)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:59.832277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:59.843760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:08:59.849280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:08:59.849562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:08:59.849832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:08:59.850477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:08:59.859482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:08:59.937705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d9297ca0884c2fa7f5283f722451ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:59.939030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d9297ca0884c2fa7f5283f722451ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:59.943793 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1784/3592] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:08:59 2022] POST /identity/v3/users => generated 298 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:08:59.949048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:08:59.961932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:08:59.981220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d9297ca0884c2fa7f5283f722451ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:08:59.999134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d9297ca0884c2fa7f5283f722451ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:00.002226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0e09a-5703-44de-8bfb-f7413b1dc637 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:00.002752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d9297ca0884c2fa7f5283f722451ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:00.028424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:00.029588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:00.031298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:00.032472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:00.033313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:00.037136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:00.074879 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1809/3593] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:08:59 2022] DELETE /identity/v3/users/68d9297ca0884c2fa7f5283f722451ba => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:00.080880 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1785/3594] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:09:00.086031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:00.121079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:00.263854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74179-ad5a-4257-9498-3665a81e0664 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:00.299261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f3352-fc87-47c1-9f22-4f19aa1a988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:00.300011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f3352-fc87-47c1-9f22-4f19aa1a988e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:00.300391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f3352-fc87-47c1-9f22-4f19aa1a988e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:00.300815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f3352-fc87-47c1-9f22-4f19aa1a988e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:00.301577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f3352-fc87-47c1-9f22-4f19aa1a988e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:00.303332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f3352-fc87-47c1-9f22-4f19aa1a988e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:00.332379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:00.333518 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1810/3595] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:00.356524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804ee14-2e3b-409e-af03-e429e73da39f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:00.374595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:00.375765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:00.376187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:00.376598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] PATH_INFO: `/v3/projects/56687e6bb37f48e4a6152179d840d29b/users/a8323bb0ff4542a5b523dceaaf7606a4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:00.402873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:00.413144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56687e6bb37f48e4a6152179d840d29b, user_id=a8323bb0ff4542a5b523dceaaf7606a4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:00.416209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:00.453241 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1786/3596] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:00 2022] PUT /identity/v3/projects/56687e6bb37f48e4a6152179d840d29b/users/a8323bb0ff4542a5b523dceaaf7606a4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:00.464525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:00.464525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:00.464525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:00.545253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] MFA Rules not processed for user `a8323bb0ff4542a5b523dceaaf7606a4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:00.546789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:00.782772 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1811/3597] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:00.786946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.005016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ab413-bf55-40d4-abd9-fa73a0acf4a5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:01.026505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:01.030501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.033910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.034414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.043063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.044347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:01.045176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:01.055639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:01.073856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d35dc-85d9-426a-9963-5ae2ffa69843 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:01.087122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:01.087557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.087850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.088135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] PATH_INFO: `/v3/projects/f8130333e2c04a3f960dc6465e07cc46` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.089286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8130333e2c04a3f960dc6465e07cc46)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:01.094204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:01.174662 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1787/3598] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:00 2022] POST /identity/v3/projects => generated 393 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:01.178364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.272322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8130333e2c04a3f960dc6465e07cc46', 'request_id': 'req-da385d6e-1e0e-4412-a93c-be0e914cf6cb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:01.295509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8130333e2c04a3f960dc6465e07cc46', 'request_id': 'req-da385d6e-1e0e-4412-a93c-be0e914cf6cb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:01.314613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8130333e2c04a3f960dc6465e07cc46', 'request_id': 'req-da385d6e-1e0e-4412-a93c-be0e914cf6cb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:01.337044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] The token cache is being invalidate because project f8130333e2c04a3f960dc6465e07cc46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:01.337395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:01.338097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da385d6e-1e0e-4412-a93c-be0e914cf6cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:01.340029 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1812/3599] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:01 2022] DELETE /identity/v3/projects/f8130333e2c04a3f960dc6465e07cc46 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:09:01.346431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae16da-f5d0-40e2-b4e8-88cddb1a194d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.346729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae16da-f5d0-40e2-b4e8-88cddb1a194d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.347130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae16da-f5d0-40e2-b4e8-88cddb1a194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.402958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-63ae16da-f5d0-40e2-b4e8-88cddb1a194d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:01.403244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968cd1f-3da8-46c8-92d5-8128199d8c27 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:01.409794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63ae16da-f5d0-40e2-b4e8-88cddb1a194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:01.467857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:01.560026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:01.560758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.561042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.561317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.562928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:01.567187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:01.578807 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1813/3600] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:01.592169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfbccf-ba1f-43e9-b92e-b84769f01d80 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.592874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfbccf-ba1f-43e9-b92e-b84769f01d80 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.593353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfbccf-ba1f-43e9-b92e-b84769f01d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.624238 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1788/3601] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:01 2022] POST /identity/v3/users => generated 303 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:09:01.625233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.628080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f7cfbccf-ba1f-43e9-b92e-b84769f01d80 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:01.629698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f7cfbccf-ba1f-43e9-b92e-b84769f01d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:01.632233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:01.639947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1814/3602] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:01.644983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfbccf-ba1f-43e9-b92e-b84769f01d80 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.664690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfbccf-ba1f-43e9-b92e-b84769f01d80 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:01.665954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996989-bb95-40bf-ad2e-1bc993b85db1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:01.676591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:01.677268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.677545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.677817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.678476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:01.679091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:01.680098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.680472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.680875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.681821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:01.684780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:01.686962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:01.695736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:01.703807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:01.714690 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1789/3603] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:01.728789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.729742 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1815/3604] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:01.734024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.756645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0fbab-3e9c-4283-b2d5-3cc861c4d8b7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:01.776595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7718ffa-da43-4138-bac7-3cd1445a4687 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:01.779092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:01.779978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.780255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.780619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.783016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:01.787201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:01.790530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:01.793823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:01.794279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:01.795554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] PATH_INFO: `/v3/projects/5ad444584cad4b6e8a07f6577983c389/users/1fee0baaeca84ecea7609a0ea238bc44/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:01.831826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad444584cad4b6e8a07f6577983c389, user_id=1fee0baaeca84ecea7609a0ea238bc44, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:01.835564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:01.867151 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1790/3605] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:01 2022] PUT /identity/v3/projects/5ad444584cad4b6e8a07f6577983c389/users/1fee0baaeca84ecea7609a0ea238bc44/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:01.869850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:01.888296 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1816/3606] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:01 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:09:01.891975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:02.063466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bc287-c0cb-44a6-ba57-506c0abaf2e3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:02.079317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:02.080220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.080373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.080630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.088560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:02.090738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:02.122210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3794908-04bf-4e92-9edd-4c2aafaa90d1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:02.128773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:02.129436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.129711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.130018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.130671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:02.140411 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1817/3607] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:01 2022] POST /identity/v3/users => generated 287 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:09:02.144368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:02.145195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:02.174240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696714-7ea1-45b2-b425-da0b1a91747e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:02.179245 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1791/3608] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:09:02.190414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:02.193323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:02.193966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.194264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.194534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.195266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:02.209091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:02.229131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeab4-feca-457f-80f6-6e64b4c5d8ae admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:02.243620 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1818/3609] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:02.249346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d590317-73e1-4502-9116-bf9742ecbed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:02.250038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d590317-73e1-4502-9116-bf9742ecbed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.250314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d590317-73e1-4502-9116-bf9742ecbed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.250582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d590317-73e1-4502-9116-bf9742ecbed9 admin admin] PATH_INFO: `/v3/projects/5ad444584cad4b6e8a07f6577983c389/users/1fee0baaeca84ecea7609a0ea238bc44/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.251711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:02.253546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d590317-73e1-4502-9116-bf9742ecbed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad444584cad4b6e8a07f6577983c389, user_id=1fee0baaeca84ecea7609a0ea238bc44, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:02.256213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d590317-73e1-4502-9116-bf9742ecbed9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:02.284780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b498-8432-4fa7-a29a-2fc864c24624 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:02.300636 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1792/3610] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:02 2022] PUT /identity/v3/projects/5ad444584cad4b6e8a07f6577983c389/users/1fee0baaeca84ecea7609a0ea238bc44/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:02.327348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.327566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.327833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.402349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] MFA Rules not processed for user `1fee0baaeca84ecea7609a0ea238bc44`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:02.406471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:02.438730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:02.439357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.439650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.440024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.440833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:02.442930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:02.444569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:02.459925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1819/3611] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:02.464727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:02.490956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6b660-8b57-4455-bbbb-bf419ab0cf4d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:02.502048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb706ea-b881-457f-be9f-57dfd2d7ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:02.513896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb706ea-b881-457f-be9f-57dfd2d7ede4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.514250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb706ea-b881-457f-be9f-57dfd2d7ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.514531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb706ea-b881-457f-be9f-57dfd2d7ede4 admin admin] PATH_INFO: `/v3/projects/e14d8dc6e28d4f07bdd215c19a9677d8/users/3cbee6fdccf24f6cb21da002e23db221/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.540043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb706ea-b881-457f-be9f-57dfd2d7ede4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14d8dc6e28d4f07bdd215c19a9677d8, user_id=3cbee6fdccf24f6cb21da002e23db221, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:02.543514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb706ea-b881-457f-be9f-57dfd2d7ede4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:02.571844 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1820/3612] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:02 2022] PUT /identity/v3/projects/e14d8dc6e28d4f07bdd215c19a9677d8/users/3cbee6fdccf24f6cb21da002e23db221/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:02.585951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afea2ba-50bf-4cd0-a61b-e01a7bc8d9f5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:02.587263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afea2ba-50bf-4cd0-a61b-e01a7bc8d9f5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:02.587560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afea2ba-50bf-4cd0-a61b-e01a7bc8d9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:02.594345 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1793/3613] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:02.708094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3afea2ba-50bf-4cd0-a61b-e01a7bc8d9f5 None None] MFA Rules not processed for user `3cbee6fdccf24f6cb21da002e23db221`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:02.709878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3afea2ba-50bf-4cd0-a61b-e01a7bc8d9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:02.915101 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1821/3614] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:03.099155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:03.103641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:03.265007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:03.313617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bc62c-fa1a-4b32-b754-8874e1380fc2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:03.412215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:03.412802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:03.413078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:03.413346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:03.494257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:03.495889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:03.509482 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1794/3615] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:03 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:03.584721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afea2ba-50bf-4cd0-a61b-e01a7bc8d9f5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:03.636524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afea2ba-50bf-4cd0-a61b-e01a7bc8d9f5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:03.647232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d579fc0-4c7a-471e-9239-bbae1ad7bc9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:03.649181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d579fc0-4c7a-471e-9239-bbae1ad7bc9c service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:03.649653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d579fc0-4c7a-471e-9239-bbae1ad7bc9c service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:03.650001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d579fc0-4c7a-471e-9239-bbae1ad7bc9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:03.661573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d579fc0-4c7a-471e-9239-bbae1ad7bc9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:03.664640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d579fc0-4c7a-471e-9239-bbae1ad7bc9c service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:03.677325 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1822/3616] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:04.752246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:04.850575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034b0bc-d3ee-4d09-aed5-b3386e8a00e4 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:04.861565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:04.862325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:04.862674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:04.863034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] PATH_INFO: `/v3/users/3cbee6fdccf24f6cb21da002e23db221` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:04.864264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cbee6fdccf24f6cb21da002e23db221)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:04.867207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:04.893610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:04.984909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cbee6fdccf24f6cb21da002e23db221'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:04.986068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cbee6fdccf24f6cb21da002e23db221'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:04.996625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cbee6fdccf24f6cb21da002e23db221'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.013180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cbee6fdccf24f6cb21da002e23db221'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.034622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cbee6fdccf24f6cb21da002e23db221'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.046879 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1795/3617] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:04 2022] DELETE /identity/v3/users/3cbee6fdccf24f6cb21da002e23db221 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:05.051523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:05.052013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:05.052141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:05.115780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:05.118380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:05.134109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:05.139140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:05.347903 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1823/3618] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:05.362314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d7b0-5337-454a-a419-380ae4f770f0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:05.369646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:05.370383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:05.370732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:05.371124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:05.373091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:05.402671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:05.404335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:05.491027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:05.493678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:05.509523 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1796/3619] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:05 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:05.546973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:05.548768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:05.560108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:05.569991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9e5a-6ffe-4f25-8a19-8b9a22da8bcb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:05.576183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f50315-cff9-405a-a442-02c0be270d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:05.576855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50315-cff9-405a-a442-02c0be270d31 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:05.577135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50315-cff9-405a-a442-02c0be270d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:05.577408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50315-cff9-405a-a442-02c0be270d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:05.588872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eadb8-597f-4ca4-b0e2-ceb43c781407 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:05.603951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:05.604782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:05.605221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:05.605692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] PATH_INFO: `/v3/users/a8323bb0ff4542a5b523dceaaf7606a4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:05.608284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8323bb0ff4542a5b523dceaaf7606a4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:05.613294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:05.628409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f50315-cff9-405a-a442-02c0be270d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:05.632633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f50315-cff9-405a-a442-02c0be270d31 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:05.676403 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1824/3620] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:05.681980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:05.741932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8323bb0ff4542a5b523dceaaf7606a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.742790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8323bb0ff4542a5b523dceaaf7606a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.760888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8323bb0ff4542a5b523dceaaf7606a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.798115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8323bb0ff4542a5b523dceaaf7606a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.839085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8323bb0ff4542a5b523dceaaf7606a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:05.848177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1797/3621] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:05 2022] DELETE /identity/v3/users/a8323bb0ff4542a5b523dceaaf7606a4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:09:05.852592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:05.853649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:05.854770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:05.904739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:05.906227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:06.013619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1825/3622] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:06.047971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:06.051181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:06.224958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643946d-4ede-4513-b559-6a45914f1d09 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:06.238724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956589c4-52f9-4646-961e-eb59112d0458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:06.242272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956589c4-52f9-4646-961e-eb59112d0458 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:06.242790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956589c4-52f9-4646-961e-eb59112d0458 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:06.243887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956589c4-52f9-4646-961e-eb59112d0458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:06.271606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956589c4-52f9-4646-961e-eb59112d0458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:06.275505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956589c4-52f9-4646-961e-eb59112d0458 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:06.293075 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1798/3623] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:07.192532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:07.194227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:07.231099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fefe45-9c92-4681-932c-63af107b98bb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:07.251544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:07.252505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:07.253035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:07.253514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] PATH_INFO: `/v3/projects/e14d8dc6e28d4f07bdd215c19a9677d8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:07.255539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14d8dc6e28d4f07bdd215c19a9677d8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:07.259129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:07.403649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14d8dc6e28d4f07bdd215c19a9677d8', 'request_id': 'req-260dbb37-db95-4299-a511-1c6050b343b0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:07.455762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14d8dc6e28d4f07bdd215c19a9677d8', 'request_id': 'req-260dbb37-db95-4299-a511-1c6050b343b0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:07.501708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14d8dc6e28d4f07bdd215c19a9677d8', 'request_id': 'req-260dbb37-db95-4299-a511-1c6050b343b0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:07.553700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] The token cache is being invalidate because project e14d8dc6e28d4f07bdd215c19a9677d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:07.554253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:07.555799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-260dbb37-db95-4299-a511-1c6050b343b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:07.565531 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1826/3624] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:07 2022] DELETE /identity/v3/projects/e14d8dc6e28d4f07bdd215c19a9677d8 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:07.571847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:07.572516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:07.572651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:07.642354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:07.643690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:07.830094 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1799/3625] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:07.838259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d78a0-fac0-4c90-b815-bcbc5fa88e67 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:07.838788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d78a0-fac0-4c90-b815-bcbc5fa88e67 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:07.840207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d78a0-fac0-4c90-b815-bcbc5fa88e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:07.845382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:07.847028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:07.869602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951304a-9ec9-4319-969e-ebeb83599cdd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:07.887467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-780d78a0-fac0-4c90-b815-bcbc5fa88e67 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:07.889775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-780d78a0-fac0-4c90-b815-bcbc5fa88e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:07.896250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:07.900500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:07.902739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:07.903023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] PATH_INFO: `/v3/projects/56687e6bb37f48e4a6152179d840d29b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:07.907899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=56687e6bb37f48e4a6152179d840d29b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:07.910178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:07.929666 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1827/3626] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:07.943761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d78a0-fac0-4c90-b815-bcbc5fa88e67 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:07.957591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d78a0-fac0-4c90-b815-bcbc5fa88e67 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:07.976330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:07.976998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:07.977275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:07.977549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:07.978410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:07.978710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:07.980417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:07.981241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:07.998653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56687e6bb37f48e4a6152179d840d29b', 'request_id': 'req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.000217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1828/3627] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:08.004279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:08.014932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56687e6bb37f48e4a6152179d840d29b', 'request_id': 'req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.029299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ce8a-b695-43ad-a147-7189d0700f86 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:08.049854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56687e6bb37f48e4a6152179d840d29b', 'request_id': 'req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.073924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] The token cache is being invalidate because project 56687e6bb37f48e4a6152179d840d29b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:08.074243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.080392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.090436 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1800/3628] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:07 2022] DELETE /identity/v3/projects/56687e6bb37f48e4a6152179d840d29b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:09:08.094881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:08.100612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:08.153326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:08.170358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:08.170995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:08.171271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:08.176353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:08.178752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:08.183743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:08.276204 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1829/3629] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:08 2022] POST /identity/v3/projects => generated 379 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:08.280883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:08.282450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:08.343518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a936fd7-c38e-4b9f-8ee7-fa3d3058e8fd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:08.457078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90be733-5b66-492d-a88c-38f5dceb4f2c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:08.486497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:08.487089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:08.487372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:08.487644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:08.493385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:08.502143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:08.522252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:08.523086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:08.523989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:08.524274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] PATH_INFO: `/v3/users/1fee0baaeca84ecea7609a0ea238bc44` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:08.526664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fee0baaeca84ecea7609a0ea238bc44)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:08.529633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:08.556816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1830/3630] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:08 2022] POST /identity/v3/users => generated 290 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:08.560561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:08.582218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143b567-2e77-4c9d-aceb-2fcf17079ac2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:08.604814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:08.605092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:08.605310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:08.605582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:08.606660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:08.607922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:08.632116 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1831/3631] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:09:08.633134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:08.666907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d39031-8946-47b3-8f5d-8ac509b9beb4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:08.670068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fee0baaeca84ecea7609a0ea238bc44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.671170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fee0baaeca84ecea7609a0ea238bc44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.685507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:08.686354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:08.686784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:08.687227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:08.687884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:08.688004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fee0baaeca84ecea7609a0ea238bc44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.702513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:08.712331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fee0baaeca84ecea7609a0ea238bc44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.736777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:08.739317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1832/3632] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:08.746649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-783790cb-f4c8-40dd-b666-7fe0a510c086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fee0baaeca84ecea7609a0ea238bc44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:08.747658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:08.750911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:08.768110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1801/3633] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:08 2022] DELETE /identity/v3/users/1fee0baaeca84ecea7609a0ea238bc44 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:08.784941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8c90-dc41-438c-a5be-34a95973971c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:08.934163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:08.936479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:08.937140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:08.937757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] PATH_INFO: `/v3/projects/2aae4a6ff6504f99952a87ec251fccc7/users/288659c6b5944cbfb952225197160fad/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:08.970476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aae4a6ff6504f99952a87ec251fccc7, user_id=288659c6b5944cbfb952225197160fad, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:08.976030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:09.003249 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1833/3634] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:08 2022] PUT /identity/v3/projects/2aae4a6ff6504f99952a87ec251fccc7/users/288659c6b5944cbfb952225197160fad/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:09.015446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1dafe-0206-4468-84b7-2b507cf12559 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:09.015802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1dafe-0206-4468-84b7-2b507cf12559 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:09.015993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1dafe-0206-4468-84b7-2b507cf12559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:09.061996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5eb1dafe-0206-4468-84b7-2b507cf12559 None None] MFA Rules not processed for user `288659c6b5944cbfb952225197160fad`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:09.063296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5eb1dafe-0206-4468-84b7-2b507cf12559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:09.179843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1802/3635] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:09.195159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:09.197418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:09.266883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:09.325275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90603f9b-90d2-4c14-9473-4000446cea3c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:09.331859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:09.332512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:09.332861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:09.333597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:09.343606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:09.346369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:09.371100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1834/3636] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:09.474956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1dafe-0206-4468-84b7-2b507cf12559 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:09.560170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:09.561348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:09.710776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1dafe-0206-4468-84b7-2b507cf12559 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:09.742622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:09.748507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:09.750102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:09.750680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] PATH_INFO: `/v3/users/288659c6b5944cbfb952225197160fad` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:09.759529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288659c6b5944cbfb952225197160fad)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:09.762079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:09.790868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b38bd2-2178-4fbd-b727-941d4fb79ba2 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:09.797746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:09.803667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:09.805964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:09.806428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] PATH_INFO: `/v3/projects/5ad444584cad4b6e8a07f6577983c389` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:09.810698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad444584cad4b6e8a07f6577983c389)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:09.815138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:09.898619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288659c6b5944cbfb952225197160fad'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.899363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288659c6b5944cbfb952225197160fad'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.907222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288659c6b5944cbfb952225197160fad'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.919373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad444584cad4b6e8a07f6577983c389', 'request_id': 'req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.922370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288659c6b5944cbfb952225197160fad'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.933408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad444584cad4b6e8a07f6577983c389', 'request_id': 'req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.943061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad444584cad4b6e8a07f6577983c389', 'request_id': 'req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.944629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-24623b2e-f42f-47a7-acb0-41e78224fb72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288659c6b5944cbfb952225197160fad'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.956424 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1803/3637] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:09 2022] DELETE /identity/v3/users/288659c6b5944cbfb952225197160fad => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:09.963959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:09.964141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] The token cache is being invalidate because project 5ad444584cad4b6e8a07f6577983c389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:09.964250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:09.964357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.964957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09b6d1f1-064a-4d3d-ba75-b0cd95c903bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:09.966509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:09.967785 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1835/3638] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:09 2022] DELETE /identity/v3/projects/5ad444584cad4b6e8a07f6577983c389 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:09.973383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782a754-1351-4cc8-9dd7-a12a682c3054 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:09.973667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782a754-1351-4cc8-9dd7-a12a682c3054 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:09.973948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782a754-1351-4cc8-9dd7-a12a682c3054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:09.988003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:10.007832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c782a754-1351-4cc8-9dd7-a12a682c3054 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:10.011293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c782a754-1351-4cc8-9dd7-a12a682c3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:10.022468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:10.026882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:10.098772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:10.163233 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1836/3639] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:10.175471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db9b72-ee01-41a6-afc7-b1f5087aaa25 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.175768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db9b72-ee01-41a6-afc7-b1f5087aaa25 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.176036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db9b72-ee01-41a6-afc7-b1f5087aaa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.208718 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1804/3640] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:10.216647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:10.218113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:10.228581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-65db9b72-ee01-41a6-afc7-b1f5087aaa25 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:10.229844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-65db9b72-ee01-41a6-afc7-b1f5087aaa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:10.254224 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1837/3641] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:10.259693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db9b72-ee01-41a6-afc7-b1f5087aaa25 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:10.281800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db9b72-ee01-41a6-afc7-b1f5087aaa25 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:10.288870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:10.289619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.289974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.290342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.291031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:10.291416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:10.293584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:10.322764 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1838/3642] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:09:10.323775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:10.337970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfd47e-2aba-4964-9bc7-73b4b587b3ac None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:10.339906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062684d3-ad48-42ad-aead-72c9a4d26ea5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:10.342163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:10.343924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.344370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.344859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.345270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:10.345972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.346312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.346635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.347438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:10.349593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:10.362961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:10.365341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:10.391416 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1805/3643] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:10.399733 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1839/3644] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:10 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:10.403792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:10.419272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d323e4-b93b-4963-89ea-8586c28a29c6 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:10.554647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:10.556708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.557229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.557769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.558859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:10.586087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:10.645263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1806/3645] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:10 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:10.649790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:10.668777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e67e1-4ee1-4794-86f2-cd7f050d4225 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:10.677976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:10.679178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.680104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.680104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.694192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:10.696390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:10.725728 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1840/3646] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:10.730369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:10.743387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301e09-1282-4329-a80c-b8de9e2c8cfd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:10.751908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc621fb-1006-41d7-8291-2cd5ab3fedf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:10.752555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc621fb-1006-41d7-8291-2cd5ab3fedf5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.752954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc621fb-1006-41d7-8291-2cd5ab3fedf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.753232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc621fb-1006-41d7-8291-2cd5ab3fedf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.754072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc621fb-1006-41d7-8291-2cd5ab3fedf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:10.755965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc621fb-1006-41d7-8291-2cd5ab3fedf5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:10.781775 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1807/3647] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:10.782906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:10.802644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de851acd-b09a-4956-979e-4933d67a3cf1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:10.813695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:10.815513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.815797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.818632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] PATH_INFO: `/v3/projects/f847bee14ff74dfbafe7c6a199f1b99f/users/b8d6f6c63fe144959139278ca287e2ec/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.853059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f847bee14ff74dfbafe7c6a199f1b99f, user_id=b8d6f6c63fe144959139278ca287e2ec, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:10.855759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:10.902149 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1841/3648] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:10 2022] PUT /identity/v3/projects/f847bee14ff74dfbafe7c6a199f1b99f/users/b8d6f6c63fe144959139278ca287e2ec/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:09:10.907592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:10.907592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:10.907950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:10.956951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:10.963531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:10.990316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:11.047599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1808/3649] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:11.080871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:11.206996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa14cd-11fb-4eaf-86a2-7eae6034ed77 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:11.221682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04f7f-d77e-4fa0-a6b1-9044a16bf542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:11.222323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04f7f-d77e-4fa0-a6b1-9044a16bf542 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:11.222709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04f7f-d77e-4fa0-a6b1-9044a16bf542 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:11.223970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04f7f-d77e-4fa0-a6b1-9044a16bf542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:11.237555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04f7f-d77e-4fa0-a6b1-9044a16bf542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:11.248536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04f7f-d77e-4fa0-a6b1-9044a16bf542 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:11.289004 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1842/3650] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:11.730247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:11.731848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:11.758173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d627195-433a-4d17-b048-4a8d4994dc6f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:11.767106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:11.768195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:11.768778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:11.769263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] PATH_INFO: `/v3/projects/2aae4a6ff6504f99952a87ec251fccc7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:11.771235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aae4a6ff6504f99952a87ec251fccc7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:11.774807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:11.975496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aae4a6ff6504f99952a87ec251fccc7', 'request_id': 'req-b260685e-9765-4093-9f24-146c7e2bd8ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:11.998573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aae4a6ff6504f99952a87ec251fccc7', 'request_id': 'req-b260685e-9765-4093-9f24-146c7e2bd8ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:12.024807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aae4a6ff6504f99952a87ec251fccc7', 'request_id': 'req-b260685e-9765-4093-9f24-146c7e2bd8ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:12.042189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] The token cache is being invalidate because project 2aae4a6ff6504f99952a87ec251fccc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:12.044225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:12.045055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b260685e-9765-4093-9f24-146c7e2bd8ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:12.047021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1809/3651] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:11 2022] DELETE /identity/v3/projects/2aae4a6ff6504f99952a87ec251fccc7 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:12.052554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ee1c-c3ee-477c-b7e5-ee13bef3a58b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.053721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ee1c-c3ee-477c-b7e5-ee13bef3a58b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.054845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1ee1c-c3ee-477c-b7e5-ee13bef3a58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.084646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b4b1ee1c-c3ee-477c-b7e5-ee13bef3a58b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:12.086951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4b1ee1c-c3ee-477c-b7e5-ee13bef3a58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:12.175179 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1843/3652] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:12.175965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51145f-015b-4676-9965-2252f7deec8b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.176302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51145f-015b-4676-9965-2252f7deec8b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.176530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51145f-015b-4676-9965-2252f7deec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.204291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3a51145f-015b-4676-9965-2252f7deec8b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:12.206242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3a51145f-015b-4676-9965-2252f7deec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:12.218927 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1810/3653] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:12.223952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1ee1c-c3ee-477c-b7e5-ee13bef3a58b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:12.246614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1ee1c-c3ee-477c-b7e5-ee13bef3a58b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:12.253533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:12.254171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.254499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.254920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.255516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:12.255796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:12.261211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:12.280352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:12.295262 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1844/3654] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:12.296283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51145f-015b-4676-9965-2252f7deec8b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:12.310938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51145f-015b-4676-9965-2252f7deec8b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:12.328561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:12.329829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.330097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.330377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.335136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:12.341107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:12.408412 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1811/3655] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:12 2022] POST /identity/v3/projects => generated 395 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:12.409654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:12.429760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e633c-eeed-4c58-a89a-48af23169f5a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:12.522248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:12.523063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.523341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.523827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.524484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:12.526591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:12.559746 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1845/3656] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:12 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:12.564168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:12.592227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18a020-3d21-4a57-96ed-417a8c488000 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:12.604922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:12.604922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.604922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.604922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.605426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:12.605426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:12.643965 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1812/3657] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:12.644855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:12.657605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4865f-c7a0-4f5e-a1a7-932d4ad292ea admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:12.662849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4985541-db60-4557-91ac-2e865ef1ba98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:12.664402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4985541-db60-4557-91ac-2e865ef1ba98 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.664731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4985541-db60-4557-91ac-2e865ef1ba98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.665078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4985541-db60-4557-91ac-2e865ef1ba98 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.665881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4985541-db60-4557-91ac-2e865ef1ba98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:12.667532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4985541-db60-4557-91ac-2e865ef1ba98 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:12.696777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1846/3658] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:09:12.700957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:12.716572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34931f1f-f299-4d16-a18f-d1469899dea5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:12.729101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:12.731366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.732057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.732633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] PATH_INFO: `/v3/projects/bf5cfea42dc94baeaaf5f1eaea296ff0/users/fd1d5ed67db946ce86f719a23ae682c3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.762933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5cfea42dc94baeaaf5f1eaea296ff0, user_id=fd1d5ed67db946ce86f719a23ae682c3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:12.767923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:12.808306 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1813/3659] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:12 2022] PUT /identity/v3/projects/bf5cfea42dc94baeaaf5f1eaea296ff0/users/fd1d5ed67db946ce86f719a23ae682c3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:12.822590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:12.822590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:12.822590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:12.868236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] MFA Rules not processed for user `fd1d5ed67db946ce86f719a23ae682c3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:12.869890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:12.979777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1847/3660] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:12 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:13.086581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:13.089164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:13.159959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:13.207301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:13.209698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:13.236437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d05fb-0557-4f4e-a8b3-ef015d85d877 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:13.250092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:13.254670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:13.254862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:13.255080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:13.258794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:13.300144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:13.301723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:13.316643 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1814/3661] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:13.317822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:13.319437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:13.347626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66170f-e17e-4007-af93-73f6c891a4df None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:13.362347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:13.363115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:13.363491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:13.363934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] PATH_INFO: `/v3/users/bf823d90bd69466691ba2094929a8239` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:13.365250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf823d90bd69466691ba2094929a8239)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:13.368516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:13.401509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:13.428118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e67872-4d2e-4673-ad03-67756b3cd567 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:13.439326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6434a08-9760-43a8-b4c5-27213db7e0aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:13.440150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6434a08-9760-43a8-b4c5-27213db7e0aa service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:13.445583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6434a08-9760-43a8-b4c5-27213db7e0aa service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:13.446065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6434a08-9760-43a8-b4c5-27213db7e0aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:13.475893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6434a08-9760-43a8-b4c5-27213db7e0aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:13.477734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6434a08-9760-43a8-b4c5-27213db7e0aa service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:13.497873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf823d90bd69466691ba2094929a8239'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:13.498730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf823d90bd69466691ba2094929a8239'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:13.508675 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1815/3662] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:13 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:13.519042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf823d90bd69466691ba2094929a8239'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:13.529102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf823d90bd69466691ba2094929a8239'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:13.531006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf823d90bd69466691ba2094929a8239'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:13.574743 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1848/3663] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:13 2022] DELETE /identity/v3/users/bf823d90bd69466691ba2094929a8239 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:13.584078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:13.584362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:13.584635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:13.635045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] MFA Rules not processed for user `fd1d5ed67db946ce86f719a23ae682c3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:13.639877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:13.718224 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1816/3664] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:14.285449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:14.287293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:14.346602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:14.347944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:14.448324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940bd9c-37c5-4cd4-a340-9c6aef15e924 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:14.463616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:14.464280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:14.464800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e152994-b44f-430a-8663-406c9b2577cc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:14.464948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:14.465163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] PATH_INFO: `/v3/projects/2b032e83813a429bb445b5f7ee854a4f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:14.466523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b032e83813a429bb445b5f7ee854a4f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:14.468449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:14.472157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:14.473226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:14.473680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:14.474044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] PATH_INFO: `/v3/users/fd1d5ed67db946ce86f719a23ae682c3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:14.476087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1d5ed67db946ce86f719a23ae682c3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:14.479752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:14.509737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:14.585033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1d5ed67db946ce86f719a23ae682c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.598102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1d5ed67db946ce86f719a23ae682c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.630071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b032e83813a429bb445b5f7ee854a4f', 'request_id': 'req-88a8b1f4-fa59-46b8-9e19-c72af65db016'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.641659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1d5ed67db946ce86f719a23ae682c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.644198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b032e83813a429bb445b5f7ee854a4f', 'request_id': 'req-88a8b1f4-fa59-46b8-9e19-c72af65db016'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.656788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1d5ed67db946ce86f719a23ae682c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.658547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b032e83813a429bb445b5f7ee854a4f', 'request_id': 'req-88a8b1f4-fa59-46b8-9e19-c72af65db016'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.673414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] The token cache is being invalidate because project 2b032e83813a429bb445b5f7ee854a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:14.675081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dedc169-401a-4569-9779-2330fdcf8137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1d5ed67db946ce86f719a23ae682c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.675357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.676156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88a8b1f4-fa59-46b8-9e19-c72af65db016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:14.682304 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1849/3665] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:14 2022] DELETE /identity/v3/projects/2b032e83813a429bb445b5f7ee854a4f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:14.684257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dacbb8-e11b-4f58-b2da-9df2431e63ac None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:14.684723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dacbb8-e11b-4f58-b2da-9df2431e63ac None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:14.684964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dacbb8-e11b-4f58-b2da-9df2431e63ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:14.696294 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1817/3666] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:14 2022] DELETE /identity/v3/users/fd1d5ed67db946ce86f719a23ae682c3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:14.700627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b4538-c178-4975-b600-fee710bce458 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:14.701071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b4538-c178-4975-b600-fee710bce458 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:14.701449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b4538-c178-4975-b600-fee710bce458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:14.720755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-85dacbb8-e11b-4f58-b2da-9df2431e63ac None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:14.724234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85dacbb8-e11b-4f58-b2da-9df2431e63ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:14.743160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c86b4538-c178-4975-b600-fee710bce458 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:14.744636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c86b4538-c178-4975-b600-fee710bce458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:14.750798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:14.822555 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1850/3667] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:14.830688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5716bdb-86c7-4c18-ac8c-73868781d2e5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:14.831089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5716bdb-86c7-4c18-ac8c-73868781d2e5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:14.831653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5716bdb-86c7-4c18-ac8c-73868781d2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:14.853929 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1818/3668] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:14.857542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b4538-c178-4975-b600-fee710bce458 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:14.859128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c86b4538-c178-4975-b600-fee710bce458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:14.861379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e5716bdb-86c7-4c18-ac8c-73868781d2e5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:14.863393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e5716bdb-86c7-4c18-ac8c-73868781d2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:14.881032 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1851/3669] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:14.885502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5716bdb-86c7-4c18-ac8c-73868781d2e5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:14.913752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5716bdb-86c7-4c18-ac8c-73868781d2e5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:14.932474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:14.940895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:14.941609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:14.941934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:14.942585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:14.942896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:14.945056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:14.955674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b4538-c178-4975-b600-fee710bce458 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:14.963537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:14.966240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:14.966601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:14.966940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:14.986946 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1852/3670] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:14.988068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:14.989582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:14.992486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:15.005752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b2022-92a2-4e8e-9906-32d98d7a043f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:15.006234 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1819/3671] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:09:15.010724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:15.011342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:15.011619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:15.011891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:15.012577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:15.014550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:15.048709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1853/3672] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:14 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:09:15.052808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:15.064598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d096fc4-7eb6-42f0-af5e-0bec808f6bc3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:15.096710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:15.146969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:15.147749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:15.148220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:15.148542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:15.149623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:15.154872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:15.182156 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1820/3673] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:15 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:15.187105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:15.199827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f1293-011a-4cc3-9608-0baa5632fa01 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:15.205453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:15.206613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:15.206959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:15.207288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:15.208453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:15.209686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:15.234917 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1854/3674] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:09:15.236413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:15.251395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511ec40-eb86-4431-b8cb-0467e9f698ec admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:15.261017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18e58c-2d13-4741-a94c-981cf6c83e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:15.261794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e58c-2d13-4741-a94c-981cf6c83e2b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:15.262070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e58c-2d13-4741-a94c-981cf6c83e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:15.262337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e58c-2d13-4741-a94c-981cf6c83e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:15.267245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18e58c-2d13-4741-a94c-981cf6c83e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:15.269264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18e58c-2d13-4741-a94c-981cf6c83e2b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:15.290889 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1821/3675] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:15.294941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:15.305732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc624f9-732d-4c18-8392-b1567dcf7697 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:15.321898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f77b5-7995-4387-a861-48a741a040df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:15.323027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f77b5-7995-4387-a861-48a741a040df admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:15.323971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f77b5-7995-4387-a861-48a741a040df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:15.324614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f77b5-7995-4387-a861-48a741a040df admin admin] PATH_INFO: `/v3/projects/1f83c8276958489e84b0907523036097/users/d8cc9d27cf6944aa9f5ef81d7fb3c428/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:15.359029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f77b5-7995-4387-a861-48a741a040df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f83c8276958489e84b0907523036097, user_id=d8cc9d27cf6944aa9f5ef81d7fb3c428, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:15.362237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f77b5-7995-4387-a861-48a741a040df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:15.395879 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1855/3676] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:15 2022] PUT /identity/v3/projects/1f83c8276958489e84b0907523036097/users/d8cc9d27cf6944aa9f5ef81d7fb3c428/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:15.406194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:15.406722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:15.406880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:15.468603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] MFA Rules not processed for user `d8cc9d27cf6944aa9f5ef81d7fb3c428`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:15.469445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:15.589357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1822/3677] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:15.600362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:15.600647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:15.601012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:15.634464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] MFA Rules not processed for user `d8cc9d27cf6944aa9f5ef81d7fb3c428`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:15.635307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:15.660917 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1856/3678] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:16.109041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:16.113713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:16.202235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:16.204484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:16.286186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb89ee8-8f1e-4635-9ab9-305afb5e62b8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:16.311178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:16.311911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:16.312234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:16.312551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:16.336306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:16.339779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0807f1-b06b-4f62-b407-7dba4d04c165 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:16.340312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:16.345783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41d48-edf5-48e7-bbb7-4ffa0af98855 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:16.346660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41d48-edf5-48e7-bbb7-4ffa0af98855 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:16.347085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41d48-edf5-48e7-bbb7-4ffa0af98855 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:16.347487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41d48-edf5-48e7-bbb7-4ffa0af98855 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:16.349906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dab41d48-edf5-48e7-bbb7-4ffa0af98855 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:16.369131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab41d48-edf5-48e7-bbb7-4ffa0af98855 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:16.371569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab41d48-edf5-48e7-bbb7-4ffa0af98855 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:16.374195 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1823/3679] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:16.376146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:16.378722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:16.418928 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1857/3680] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:09:16.451948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:16.470704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:16.471448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:16.471997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:16.502859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:16.504824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:16.557379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72f3fc-67fd-4b4e-ab23-50890ead598b service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:16.570494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:16.576228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:16.576934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:16.577218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:16.581051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] PATH_INFO: `/v3/projects/bf5cfea42dc94baeaaf5f1eaea296ff0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:16.589835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf5cfea42dc94baeaaf5f1eaea296ff0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:16.595638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:16.688575 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1858/3681] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:16.690034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:16.693414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:16.850622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf5cfea42dc94baeaaf5f1eaea296ff0', 'request_id': 'req-4e6dc2aa-738c-444d-9197-4ae27d4d921c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:16.880267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf5cfea42dc94baeaaf5f1eaea296ff0', 'request_id': 'req-4e6dc2aa-738c-444d-9197-4ae27d4d921c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:16.918978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf5cfea42dc94baeaaf5f1eaea296ff0', 'request_id': 'req-4e6dc2aa-738c-444d-9197-4ae27d4d921c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:16.943449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4615dd-a140-4427-8fd3-2546c2f24def None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:16.958532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] The token cache is being invalidate because project bf5cfea42dc94baeaaf5f1eaea296ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:16.959201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:16.961955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4e6dc2aa-738c-444d-9197-4ae27d4d921c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:16.969071 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1824/3682] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:16 2022] DELETE /identity/v3/projects/bf5cfea42dc94baeaaf5f1eaea296ff0 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:16.979815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b0d0-6ffa-4b90-b85e-f4a02e542a64 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:16.980107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b0d0-6ffa-4b90-b85e-f4a02e542a64 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:16.980387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b0d0-6ffa-4b90-b85e-f4a02e542a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:17.095400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3684b0d0-6ffa-4b90-b85e-f4a02e542a64 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:17.100937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3684b0d0-6ffa-4b90-b85e-f4a02e542a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:17.155845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:17.156633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:17.160460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:17.160747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:17.170845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:17.375127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:17.376769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:17.378810 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1825/3683] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:17.387619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960f18b-69fb-4990-b691-3283f8d92467 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:17.387905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960f18b-69fb-4990-b691-3283f8d92467 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:17.388182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960f18b-69fb-4990-b691-3283f8d92467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:17.417112 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1859/3684] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:09:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:17.453737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0960f18b-69fb-4990-b691-3283f8d92467 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:17.455007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0960f18b-69fb-4990-b691-3283f8d92467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:17.486836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1826/3685] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:17.491253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:17.521782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700befe-162f-4fad-9a9c-cc1378c1e6a1 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:17.548722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:17.549546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:17.550000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:17.550429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:17.551227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:17.551675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:17.557845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:17.584458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:17.619157 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1860/3686] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:17.620344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960f18b-69fb-4990-b691-3283f8d92467 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:17.637461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960f18b-69fb-4990-b691-3283f8d92467 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:17.650382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3b5d8-5eab-4550-a304-677b6c7d032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:17.651022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3b5d8-5eab-4550-a304-677b6c7d032d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:17.651300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3b5d8-5eab-4550-a304-677b6c7d032d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:17.654391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3b5d8-5eab-4550-a304-677b6c7d032d admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:17.655021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3b5d8-5eab-4550-a304-677b6c7d032d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:17.662898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3b5d8-5eab-4550-a304-677b6c7d032d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:17.771583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1827/3687] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:17 2022] POST /identity/v3/projects => generated 389 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:17.775895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:17.792148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139387-75ab-4cb4-a054-8350ce537e4c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:17.890262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:17.890549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:17.890824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:17.953559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:17.963384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:17.994967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:17.995595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:17.995866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:17.996133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:17.996816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:17.998992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:18.023062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:18.087996 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1861/3688] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:17 2022] POST /identity/v3/users => generated 295 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:09:18.095229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:18.123835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c21b8-5428-4212-b7db-e8aeee2cd769 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:18.136961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:18.137302 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1828/3689] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:09:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:18.138192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:18.138670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:18.139087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:18.139924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:18.151659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:18.154875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:18.156625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:18.206426 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1862/3690] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:18.213114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:18.245735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e021b-b785-4450-b050-bca1e2d9297a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:18.258992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:18.260990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:18.261145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:18.261506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:18.262444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:18.267916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:18.300973 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1863/3691] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:18.305711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:18.325719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa26c2b-b255-40a0-99e6-7ef7a4f610e6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:18.338939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:18.339923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:18.340369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:18.340852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] PATH_INFO: `/v3/projects/f86d6a96f893464f81868717f0531bd3/users/04a5f2b1d0134e529f25c227954b5f20/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:18.347370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953206e-4963-449d-ab91-8798b2c8d63a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:18.355179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1f6a9-c8b5-4684-9234-624410b5c746 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:18.356177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f6a9-c8b5-4684-9234-624410b5c746 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:18.356617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f6a9-c8b5-4684-9234-624410b5c746 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:18.357076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f6a9-c8b5-4684-9234-624410b5c746 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:18.377688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1f6a9-c8b5-4684-9234-624410b5c746 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:18.380010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1f6a9-c8b5-4684-9234-624410b5c746 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:18.384929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86d6a96f893464f81868717f0531bd3, user_id=04a5f2b1d0134e529f25c227954b5f20, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:18.389253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:18.403483 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1829/3692] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:09:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:18.423684 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1864/3693] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:18 2022] PUT /identity/v3/projects/f86d6a96f893464f81868717f0531bd3/users/04a5f2b1d0134e529f25c227954b5f20/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:18.429922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:18.429922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:18.429922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:18.499345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:18.505024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:18.809398 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1830/3694] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:18.858301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:18.860082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:19.020825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d1e90-7129-43ae-8110-4848bca0b102 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:19.035462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4118-9115-4a54-8364-5e7e100d7caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:19.036889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4118-9115-4a54-8364-5e7e100d7caf service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:19.037318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4118-9115-4a54-8364-5e7e100d7caf service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:19.037789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4118-9115-4a54-8364-5e7e100d7caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:19.063263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab4118-9115-4a54-8364-5e7e100d7caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:19.066926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab4118-9115-4a54-8364-5e7e100d7caf service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:19.086294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1865/3695] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:20.771084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:20.773057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:20.917715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:20.934075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0955b-3bd7-4f69-9f88-b66223fe915e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:20.945754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ea7f4-8b64-467f-80d7-c56fbc64adfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:20.948578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea7f4-8b64-467f-80d7-c56fbc64adfe service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:20.949368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea7f4-8b64-467f-80d7-c56fbc64adfe service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:20.950252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea7f4-8b64-467f-80d7-c56fbc64adfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:20.954096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-109ea7f4-8b64-467f-80d7-c56fbc64adfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:20.974440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ea7f4-8b64-467f-80d7-c56fbc64adfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:20.977281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ea7f4-8b64-467f-80d7-c56fbc64adfe service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:21.008592 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1831/3696] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:22.239100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:22.239674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:22.239674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:22.306128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:22.311767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:22.407842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204950e-58a3-4029-bc2f-72946b23a0c4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:22.408739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204950e-58a3-4029-bc2f-72946b23a0c4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:22.409015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204950e-58a3-4029-bc2f-72946b23a0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:22.464790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:22.495900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b204950e-58a3-4029-bc2f-72946b23a0c4 None None] MFA Rules not processed for user `b8d6f6c63fe144959139278ca287e2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:22.497562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b204950e-58a3-4029-bc2f-72946b23a0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:22.500283 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1866/3697] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:22.683922 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1832/3698] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:22.685192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:22.687111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:22.714192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae839e-4de4-47aa-bfb0-4b06ba04ac5a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:22.750245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:22.751655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:22.752432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:22.753965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:22.756903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:22.764019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:22.832948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1867/3699] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:22 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:22.841224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204950e-58a3-4029-bc2f-72946b23a0c4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:22.895992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204950e-58a3-4029-bc2f-72946b23a0c4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:23.055926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:23.056623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:23.056947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:23.057228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:23.058029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:23.060172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:23.118809 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1833/3700] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:22 2022] POST /identity/v3/users => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:23.122112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:23.135953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e3296-a430-4d0c-915d-d80529739811 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:23.157385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:23.158023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:23.158652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:23.158912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:23.159647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:23.161798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:23.182305 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1868/3701] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:23.187162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:23.206111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024301-44a6-4a57-aa07-bd186a1fa090 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:23.217927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:23.220690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:23.220690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:23.220690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] PATH_INFO: `/v3/projects/688812897dc54f40ad95e885b40a23f5/users/31e653b281b64ade86e15660158ff07e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:23.260192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688812897dc54f40ad95e885b40a23f5, user_id=31e653b281b64ade86e15660158ff07e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:23.263188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:23.307173 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1834/3702] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:23 2022] PUT /identity/v3/projects/688812897dc54f40ad95e885b40a23f5/users/31e653b281b64ade86e15660158ff07e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:23.312546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:23.339878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba37e9-1ba1-47bd-b9a0-57210bb0aef4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:23.578953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b26adb-d1ce-4f86-9273-ff6faec4e5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:23.579882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26adb-d1ce-4f86-9273-ff6faec4e5dc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:23.580208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26adb-d1ce-4f86-9273-ff6faec4e5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:23.580482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26adb-d1ce-4f86-9273-ff6faec4e5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:23.582313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b26adb-d1ce-4f86-9273-ff6faec4e5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:23.583942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b26adb-d1ce-4f86-9273-ff6faec4e5dc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:23.614686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:23.615790 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1869/3703] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:23.629079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9cab2-ece3-417d-a122-22dbc5521162 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:23.635132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:23.635932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:23.636316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:23.636713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] PATH_INFO: `/v3/projects/688812897dc54f40ad95e885b40a23f5/users/31e653b281b64ade86e15660158ff07e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:23.638735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688812897dc54f40ad95e885b40a23f5, user_id=31e653b281b64ade86e15660158ff07e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:23.642926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:23.662444 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1835/3704] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:23 2022] PUT /identity/v3/projects/688812897dc54f40ad95e885b40a23f5/users/31e653b281b64ade86e15660158ff07e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:33.601671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=47,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:33.653486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:36.115570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab837d-9cef-40de-80a2-bac9578d1bae None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:36.116550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab837d-9cef-40de-80a2-bac9578d1bae None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:36.117069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab837d-9cef-40de-80a2-bac9578d1bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:36.178599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f0ab837d-9cef-40de-80a2-bac9578d1bae None None] MFA Rules not processed for user `04a5f2b1d0134e529f25c227954b5f20`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:36.183724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f0ab837d-9cef-40de-80a2-bac9578d1bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:36.375337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:36.384731 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1870/3705] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:36.556293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39512ae-6fbb-4d3a-b88d-6b7ed6b0e6c3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:36.568465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:36.568992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:36.569267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:36.569583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:36.570240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:36.575158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:36.670330 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1836/3706] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:36 2022] POST /identity/v3/projects => generated 389 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:09:36.671784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab837d-9cef-40de-80a2-bac9578d1bae None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:36.733773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab837d-9cef-40de-80a2-bac9578d1bae None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:36.893862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:36.894557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:36.897423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:36.897768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:36.899504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:36.901521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:36.980295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:36.983049 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1871/3707] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:36 2022] POST /identity/v3/users => generated 301 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:36.993661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87cc82-f550-480d-8283-0db2ad1ff27e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:37.010526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:37.012143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:37.012306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:37.012526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:37.013999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:37.018462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:37.035427 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1837/3708] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:37.036579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:37.048940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d6c4-7593-4bb7-92dc-629825cbe1e3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:37.055171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:37.056364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:37.056837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:37.057410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] PATH_INFO: `/v3/projects/5f0d32ee9bd342c0949d28a270e3fb7e/users/95be68bcca8a4fa3a2d0a0d28173aa85/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:37.083007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0d32ee9bd342c0949d28a270e3fb7e, user_id=95be68bcca8a4fa3a2d0a0d28173aa85, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:37.087989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:37.122352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1872/3709] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:37 2022] PUT /identity/v3/projects/5f0d32ee9bd342c0949d28a270e3fb7e/users/95be68bcca8a4fa3a2d0a0d28173aa85/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:37.123079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:37.145105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c0411-396c-4630-8420-e10154b8e491 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:37.279410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:37.280095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:37.280495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:37.280884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:37.281571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:37.283078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:37.313263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1838/3710] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:37.314236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:37.335073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059cab2-92a4-4f0e-bcfe-257b077697ec admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:37.349168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541e442-1974-449a-8b29-a06c5066321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:37.350172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541e442-1974-449a-8b29-a06c5066321d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:37.350600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541e442-1974-449a-8b29-a06c5066321d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:37.351096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541e442-1974-449a-8b29-a06c5066321d admin admin] PATH_INFO: `/v3/projects/5f0d32ee9bd342c0949d28a270e3fb7e/users/95be68bcca8a4fa3a2d0a0d28173aa85/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:37.353755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541e442-1974-449a-8b29-a06c5066321d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0d32ee9bd342c0949d28a270e3fb7e, user_id=95be68bcca8a4fa3a2d0a0d28173aa85, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:37.373260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541e442-1974-449a-8b29-a06c5066321d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:37.391956 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1873/3711] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:37 2022] PUT /identity/v3/projects/5f0d32ee9bd342c0949d28a270e3fb7e/users/95be68bcca8a4fa3a2d0a0d28173aa85/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:39.195303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:39.198741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:39.221888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6f431-6bbd-4e5b-ace4-d1fea5e98601 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:39.347444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:39.349341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:39.349626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:39.349905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] PATH_INFO: `/v3/users/b4e0f20e52074142a3b525bafbb30c97` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:39.354591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e0f20e52074142a3b525bafbb30c97)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:39.359230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:39.391670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:39.496889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e0f20e52074142a3b525bafbb30c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:39.499325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e0f20e52074142a3b525bafbb30c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:39.516682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e0f20e52074142a3b525bafbb30c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:39.539425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e0f20e52074142a3b525bafbb30c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:39.563814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e0f20e52074142a3b525bafbb30c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:39.581049 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1839/3712] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:39 2022] DELETE /identity/v3/users/b4e0f20e52074142a3b525bafbb30c97 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:39.585714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:39.586779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:39.587256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:39.644371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:39.646051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:39.750732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:39.845951 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1874/3713] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:39.904040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:39.905747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:40.080274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c9890-2003-468b-ac8d-28055400faac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:40.091128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:40.091972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:40.092976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:40.093434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:40.122136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:40.125812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:40.155930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1840/3714] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:41.405614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:41.407793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:41.428161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab192e-6bce-44bd-afa9-ae35fda8817c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:41.440279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:41.441538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:41.443185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:41.444213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] PATH_INFO: `/v3/projects/52efcec2c8a14640a8c3df1dc576bbf1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:41.446841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52efcec2c8a14640a8c3df1dc576bbf1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:41.451238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:41.587303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52efcec2c8a14640a8c3df1dc576bbf1', 'request_id': 'req-268ad40e-0b3f-47b2-9660-3f75d2355626'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:41.626594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52efcec2c8a14640a8c3df1dc576bbf1', 'request_id': 'req-268ad40e-0b3f-47b2-9660-3f75d2355626'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:41.637104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52efcec2c8a14640a8c3df1dc576bbf1', 'request_id': 'req-268ad40e-0b3f-47b2-9660-3f75d2355626'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:41.668263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] The token cache is being invalidate because project 52efcec2c8a14640a8c3df1dc576bbf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:41.668855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:41.670387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:41.672460 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1875/3715] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:41 2022] DELETE /identity/v3/projects/52efcec2c8a14640a8c3df1dc576bbf1 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:41.676417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:41.690091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7833cb2-021c-4327-a24b-43aa77b7c191 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:41.824247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:41.825147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:41.825367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:41.825600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] PATH_INFO: `/v3/users/566ef52c26d044f8902be6f32e2b51c3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:41.827993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] RBAC: Authorizing `identity:delete_user(user_id=566ef52c26d044f8902be6f32e2b51c3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:41.833013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:41.930773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:41.951897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566ef52c26d044f8902be6f32e2b51c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:41.953869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566ef52c26d044f8902be6f32e2b51c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:41.972256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566ef52c26d044f8902be6f32e2b51c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:42.000354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566ef52c26d044f8902be6f32e2b51c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:42.032266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-643b819e-5b58-4dff-a0da-42836c2cd49a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566ef52c26d044f8902be6f32e2b51c3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:42.053525 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1841/3716] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:41 2022] DELETE /identity/v3/users/566ef52c26d044f8902be6f32e2b51c3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:43.762382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:43.764733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:43.792758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ad40e-0b3f-47b2-9660-3f75d2355626 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:43.923687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:43.924460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:43.924978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:43.925900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] PATH_INFO: `/v3/projects/89a2927af7e8412fa3fd454fbd375459` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:43.929192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a2927af7e8412fa3fd454fbd375459)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:43.931297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:44.052073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a2927af7e8412fa3fd454fbd375459', 'request_id': 'req-3cb132d4-3d30-40aa-b108-62f351d13cee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:44.082678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a2927af7e8412fa3fd454fbd375459', 'request_id': 'req-3cb132d4-3d30-40aa-b108-62f351d13cee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:44.093354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a2927af7e8412fa3fd454fbd375459', 'request_id': 'req-3cb132d4-3d30-40aa-b108-62f351d13cee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:44.098824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:44.119865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] The token cache is being invalidate because project 89a2927af7e8412fa3fd454fbd375459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:44.120170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:44.121100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3cb132d4-3d30-40aa-b108-62f351d13cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:44.127330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1876/3717] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:43 2022] DELETE /identity/v3/projects/89a2927af7e8412fa3fd454fbd375459 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:44.135728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6553b5-809c-44f2-b3c5-de54405cffc3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:44.136280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6553b5-809c-44f2-b3c5-de54405cffc3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:44.136280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6553b5-809c-44f2-b3c5-de54405cffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:44.179481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9c6553b5-809c-44f2-b3c5-de54405cffc3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:44.180644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c6553b5-809c-44f2-b3c5-de54405cffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:44.336748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1842/3718] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:44.345706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b91-c00c-4daf-aa35-df070330ea9f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:44.346005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b91-c00c-4daf-aa35-df070330ea9f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:44.346446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b91-c00c-4daf-aa35-df070330ea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:44.400019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-756f1b91-c00c-4daf-aa35-df070330ea9f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:44.401690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-756f1b91-c00c-4daf-aa35-df070330ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:44.421384 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1877/3719] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:44.423270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6553b5-809c-44f2-b3c5-de54405cffc3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:44.462940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6553b5-809c-44f2-b3c5-de54405cffc3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:44.482337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:44.483257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:44.483743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:44.484465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:44.485371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:44.485902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:44.487719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:44.518131 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1843/3720] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:44.520299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1b91-c00c-4daf-aa35-df070330ea9f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:44.537082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1b91-c00c-4daf-aa35-df070330ea9f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:44.552952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:44.553554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:44.553827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:44.554364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:44.555469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:44.557437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:44.626178 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1878/3721] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:44 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:44.628210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:44.659066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5ce56-9f17-4420-af64-aa0d9d1e7e0a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:44.874316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:44.875130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:44.875416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:44.875698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:44.876359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:44.878330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:44.948776 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1844/3722] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:44 2022] POST /identity/v3/users => generated 286 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:44.952854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:44.983916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713d6eb-2ab1-4dd5-bd0c-1dacab7a0d30 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:45.004183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:45.005008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:45.005264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:45.005524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:45.006185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:45.007969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:45.036999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:45.042600 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1879/3723] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:45.051225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba352c4-2bf6-4df7-a759-23a914d0cad8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:45.065429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ddf6e-8358-4204-b418-0990b457cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:45.066076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ddf6e-8358-4204-b418-0990b457cd93 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:45.066365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ddf6e-8358-4204-b418-0990b457cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:45.067088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ddf6e-8358-4204-b418-0990b457cd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:45.067750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ddf6e-8358-4204-b418-0990b457cd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:45.069636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ddf6e-8358-4204-b418-0990b457cd93 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:45.096372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:45.104977 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1845/3724] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:09:45.109994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc77b6b-053d-4417-892b-b1dcc1431fb3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:45.126657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:45.127472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:45.127670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:45.127978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] PATH_INFO: `/v3/projects/ddb7c4c03e254178b0e9991d54b04f41/users/9c748c35ea164fd391630a67c123cf2d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:45.194986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb7c4c03e254178b0e9991d54b04f41, user_id=9c748c35ea164fd391630a67c123cf2d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:45.209111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:45.247970 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1880/3725] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:45 2022] PUT /identity/v3/projects/ddb7c4c03e254178b0e9991d54b04f41/users/9c748c35ea164fd391630a67c123cf2d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:09:45.267684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d4e29-c008-43cf-8fce-1b3f45628259 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:45.268169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d4e29-c008-43cf-8fce-1b3f45628259 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:45.268305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d4e29-c008-43cf-8fce-1b3f45628259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:45.326576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-276d4e29-c008-43cf-8fce-1b3f45628259 None None] MFA Rules not processed for user `9c748c35ea164fd391630a67c123cf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:45.327893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-276d4e29-c008-43cf-8fce-1b3f45628259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:45.347868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:45.484848 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1846/3726] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:45.493379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:45.499210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:45.678503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df0499-93c7-4d50-a0c4-744404718f53 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:45.686052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cbf81-9e0e-4285-9b4b-275a062d7a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:45.687086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cbf81-9e0e-4285-9b4b-275a062d7a5f service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:45.687363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cbf81-9e0e-4285-9b4b-275a062d7a5f service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:45.688936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cbf81-9e0e-4285-9b4b-275a062d7a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:45.708488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cbf81-9e0e-4285-9b4b-275a062d7a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:45.711903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cbf81-9e0e-4285-9b4b-275a062d7a5f service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:45.732772 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1881/3727] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:48.408972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d4e29-c008-43cf-8fce-1b3f45628259 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:48.621960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d4e29-c008-43cf-8fce-1b3f45628259 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:48.639478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:48.641060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:48.642244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:48.643077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] PATH_INFO: `/v3/users/9c748c35ea164fd391630a67c123cf2d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:48.647257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c748c35ea164fd391630a67c123cf2d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:48.651715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:48.778459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c748c35ea164fd391630a67c123cf2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:48.780477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c748c35ea164fd391630a67c123cf2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:48.809560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c748c35ea164fd391630a67c123cf2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:48.830556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c748c35ea164fd391630a67c123cf2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:48.849563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81f99c10-c523-4daa-969d-59a5be6a7ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c748c35ea164fd391630a67c123cf2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:48.867401 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1847/3728] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:48 2022] DELETE /identity/v3/users/9c748c35ea164fd391630a67c123cf2d => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:48.869425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:48.869908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:48.870147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:48.891500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:48.892101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:48.892326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:48.939119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:48.942581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:48.978622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] MFA Rules not processed for user `31e653b281b64ade86e15660158ff07e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:48.982814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:49.030749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:49.108994 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1882/3729] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:49.176246 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1848/3730] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:49.192964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:49.194538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:49.202351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:49.206619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:49.262083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:49.354540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd95cc-6e55-4343-b7e2-65dab53b19d7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:49.363642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:49.365370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:49.367134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:49.367651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:49.392864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:49.398466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1ed50-4b53-4e23-a344-8d9a637e3835 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:49.401628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:49.425924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:49.428108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:49.428108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:49.428108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:49.431072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:49.443840 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1849/3731] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:49.597184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:49.602194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:49.631570 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1883/3732] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:50.289002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:50.291787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:50.313144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e071a0e-73bc-43a9-b8fe-8a54e1a06ebc service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:50.327233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:50.328034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:50.328321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:50.328601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] PATH_INFO: `/v3/users/2699ef01c0eb4d5783203e6b41293d97` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:50.330066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2699ef01c0eb4d5783203e6b41293d97)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:50.348119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:50.424325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2699ef01c0eb4d5783203e6b41293d97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:50.425385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2699ef01c0eb4d5783203e6b41293d97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:50.438419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2699ef01c0eb4d5783203e6b41293d97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:50.455929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2699ef01c0eb4d5783203e6b41293d97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:50.487252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03e61756-940a-4202-8335-288bcaca2dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2699ef01c0eb4d5783203e6b41293d97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:50.521775 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1850/3733] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:09:50 2022] DELETE /identity/v3/users/2699ef01c0eb4d5783203e6b41293d97 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:51.100255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:51.112507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc2a00-93c0-4fc0-a8dc-73dc2bb3dda7 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:51.270750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:51.271629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:51.272017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:51.272416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] PATH_INFO: `/v3/projects/79022e5ca5764576b4feb7f4cadba80c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:51.273866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79022e5ca5764576b4feb7f4cadba80c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:51.276076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:51.382300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79022e5ca5764576b4feb7f4cadba80c', 'request_id': 'req-6c06edae-b6f2-4e81-9d22-84238c730f18'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.412714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79022e5ca5764576b4feb7f4cadba80c', 'request_id': 'req-6c06edae-b6f2-4e81-9d22-84238c730f18'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.418779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:51.443921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79022e5ca5764576b4feb7f4cadba80c', 'request_id': 'req-6c06edae-b6f2-4e81-9d22-84238c730f18'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.468853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] The token cache is being invalidate because project 79022e5ca5764576b4feb7f4cadba80c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:51.469083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.470109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6c06edae-b6f2-4e81-9d22-84238c730f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.474509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1884/3734] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:51 2022] DELETE /identity/v3/projects/79022e5ca5764576b4feb7f4cadba80c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:51.480849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6ac0-483f-403d-9061-b8ba26bd2fc8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:51.481340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6ac0-483f-403d-9061-b8ba26bd2fc8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:51.481446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6ac0-483f-403d-9061-b8ba26bd2fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:51.518778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8fda6ac0-483f-403d-9061-b8ba26bd2fc8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:51.520109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8fda6ac0-483f-403d-9061-b8ba26bd2fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:51.569292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:51.653291 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1851/3735] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:51.661247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:51.661780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:51.662126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:51.703964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:51.706102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:51.722028 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1885/3736] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:51.723991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6ac0-483f-403d-9061-b8ba26bd2fc8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:51.738080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6ac0-483f-403d-9061-b8ba26bd2fc8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:51.743545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:51.745205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:51.754129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:51.754763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:51.755043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:51.755314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:51.755985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:51.762446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:51.772151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:51.774727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ace556-f318-4aee-8099-4d52d1ec2431 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:51.784464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:51.785618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:51.786163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:51.786684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] PATH_INFO: `/v3/projects/ddb7c4c03e254178b0e9991d54b04f41` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:51.788975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb7c4c03e254178b0e9991d54b04f41)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:51.791260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:51.814426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1852/3737] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:51.815663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:51.837499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50a13-40ea-42ec-8057-af7409781726 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:51.846331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:51.847673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:51.848405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:51.855082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:51.861975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:51.867683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:51.886069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb7c4c03e254178b0e9991d54b04f41', 'request_id': 'req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.907727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb7c4c03e254178b0e9991d54b04f41', 'request_id': 'req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.920828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb7c4c03e254178b0e9991d54b04f41', 'request_id': 'req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.938856 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1853/3738] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:51 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:51.943321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:51.951050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] The token cache is being invalidate because project ddb7c4c03e254178b0e9991d54b04f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:09:51.952066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.954999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4835c0e6-a1e0-43ca-88af-8db5fffdb0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:09:51.963026 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1886/3739] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:51 2022] DELETE /identity/v3/projects/ddb7c4c03e254178b0e9991d54b04f41 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:09:51.965289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402b0dd-86f5-4386-b5d9-9202a296f8df None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:51.965619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402b0dd-86f5-4386-b5d9-9202a296f8df None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:51.966098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402b0dd-86f5-4386-b5d9-9202a296f8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.029051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9402b0dd-86f5-4386-b5d9-9202a296f8df None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:52.035064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9402b0dd-86f5-4386-b5d9-9202a296f8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:52.118655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914deba-906a-4269-8ad0-f3f3ab201697 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.125075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.127503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.127746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.128209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.129467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.131545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.157156 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1887/3740] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:52.165631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35458754-39d1-46cd-a237-dc5405b18f6f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.168026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35458754-39d1-46cd-a237-dc5405b18f6f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.168380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35458754-39d1-46cd-a237-dc5405b18f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.175251 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1854/3741] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:51 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:52.177337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.181696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:52.211000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f304e-f01d-47a3-803f-08735801e47b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.212144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-35458754-39d1-46cd-a237-dc5405b18f6f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:52.216218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-35458754-39d1-46cd-a237-dc5405b18f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:52.224030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.225541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.226646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.228166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.228995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.230760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.243218 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1888/3742] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:52.244224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35458754-39d1-46cd-a237-dc5405b18f6f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.270171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1855/3743] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:52.270822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35458754-39d1-46cd-a237-dc5405b18f6f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.273263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.286766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde681c-5703-4513-8859-e0d58c1e8d53 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.287991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.288806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.289088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.289360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.290314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:09:52.290700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.293328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.294154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.294723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.295127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.295947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.296124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.298586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.321802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1856/3744] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:52.327784 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1889/3745] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:09:52.329863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.331383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.354749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ec1e4-1980-487b-adb7-8e69ab288a3d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.358773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3b6a6-6f0e-4e19-b5ea-67b6c11d965e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.367891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.368539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.368843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.369114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.369758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.371710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.388419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759c0f9-df97-48f1-ba1e-8fc13cb7356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.389444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c0f9-df97-48f1-ba1e-8fc13cb7356b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.389726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c0f9-df97-48f1-ba1e-8fc13cb7356b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.390009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c0f9-df97-48f1-ba1e-8fc13cb7356b admin admin] PATH_INFO: `/v3/projects/c24fd5e915af4b6c872d69b99e7375ce/users/b11313762b324174a9a5f362d1cf4bdd/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.430617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c0f9-df97-48f1-ba1e-8fc13cb7356b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24fd5e915af4b6c872d69b99e7375ce, user_id=b11313762b324174a9a5f362d1cf4bdd, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.454283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c0f9-df97-48f1-ba1e-8fc13cb7356b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.458687 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1890/3746] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:09:52 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:52.465320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.497910 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1857/3747] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:52 2022] PUT /identity/v3/projects/c24fd5e915af4b6c872d69b99e7375ce/users/b11313762b324174a9a5f362d1cf4bdd/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:52.502835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.502835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.503416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.546001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:52.564239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:52.565704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:52.634359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:52.654356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2446297-4f0b-440a-828a-83d7be58467a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.700564 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1858/3748] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:52.723982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.725559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:52.824282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.826468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.826851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.827273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.828110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.831353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.849046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4bd87-f1c5-4d48-86cd-b74e8b4bd94f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.857112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.857821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.858068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.858362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.879758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1891/3749] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:09:52 2022] POST /identity/v3/users => generated 284 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:09:52.879878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.881345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.891672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.909907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1859/3750] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:09:52.914889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f953a2-f7d1-4c7e-a66c-b942575645c1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.926138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.927077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.927419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.928085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.933820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.937386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:52.960317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1892/3751] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:52.964288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:52.975597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5eb0f6-3c98-4160-aa23-1e772bf1bf3b service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:52.993051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53946560-56fd-4bd0-b874-d61752fe4d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:52.993687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946560-56fd-4bd0-b874-d61752fe4d7d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:52.993973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946560-56fd-4bd0-b874-d61752fe4d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:52.994307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946560-56fd-4bd0-b874-d61752fe4d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:52.994972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53946560-56fd-4bd0-b874-d61752fe4d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:52.997233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53946560-56fd-4bd0-b874-d61752fe4d7d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:53.034680 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1860/3752] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:09:53.041606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:53.055208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13ed0d-874f-48a8-9377-8d420d126a82 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:53.065391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e93709-b8f2-4044-8de6-be3d3e1659db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:53.068575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93709-b8f2-4044-8de6-be3d3e1659db admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:53.069295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93709-b8f2-4044-8de6-be3d3e1659db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:53.070820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93709-b8f2-4044-8de6-be3d3e1659db admin admin] PATH_INFO: `/v3/projects/a74a11e81faa4ca59f7c5662a3e8321f/users/a83af7f06aa94823989c466abc0ffa3e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:53.092472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e93709-b8f2-4044-8de6-be3d3e1659db admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74a11e81faa4ca59f7c5662a3e8321f, user_id=a83af7f06aa94823989c466abc0ffa3e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:53.095685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e93709-b8f2-4044-8de6-be3d3e1659db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:53.135639 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1893/3753] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:09:53 2022] PUT /identity/v3/projects/a74a11e81faa4ca59f7c5662a3e8321f/users/a83af7f06aa94823989c466abc0ffa3e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:09:53.146210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:53.147929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:53.148976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:53.177015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:53.177475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:53.179404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:53.201148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:53.203199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:53.239640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] MFA Rules not processed for user `95be68bcca8a4fa3a2d0a0d28173aa85`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:53.241083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:53.346554 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1861/3754] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:53.358725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:53.360361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:53.402902 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1894/3755] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:09:53.422140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:53.423419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:53.439793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:53.502191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083c48-4da8-4bf4-a0fc-f944ba2173ed None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:53.512234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a5df-5a5e-416f-9383-f9a7e4b9a68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:53.512905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a5df-5a5e-416f-9383-f9a7e4b9a68a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:53.513999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a5df-5a5e-416f-9383-f9a7e4b9a68a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:53.516534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a5df-5a5e-416f-9383-f9a7e4b9a68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:53.540571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc623cf-70c1-458a-a1e8-fb255bf06a3e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:53.545710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:53.546533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:53.547178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:53.547595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:53.550024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:53.555166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a5df-5a5e-416f-9383-f9a7e4b9a68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:53.561600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a5df-5a5e-416f-9383-f9a7e4b9a68a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:53.596798 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1862/3756] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:53.600810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97622ef5-08d3-409d-a169-2c2988c762c7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:53.600810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97622ef5-08d3-409d-a169-2c2988c762c7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:53.600810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97622ef5-08d3-409d-a169-2c2988c762c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:53.636506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-97622ef5-08d3-409d-a169-2c2988c762c7 None None] MFA Rules not processed for user `b8d6f6c63fe144959139278ca287e2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:09:53.640394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-97622ef5-08d3-409d-a169-2c2988c762c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:53.646243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:53.685161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:53.688930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:53.743710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1895/3757] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:09:53.759086 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1863/3758] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:09:55.147133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:55.148807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:55.301077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e845370-0093-4848-86f9-1e661dc9569a service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:55.316386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:55.317451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:55.317733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:55.318080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:55.323712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:55.360573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:55.362391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:55.397077 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1896/3759] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:09:59.216519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97622ef5-08d3-409d-a169-2c2988c762c7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:09:59.230569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97622ef5-08d3-409d-a169-2c2988c762c7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:09:59.246514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:09:59.247407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:09:59.248223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:09:59.249298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:09:59.256008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:09:59.298355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:09:59.303692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:09:59.335931 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1864/3760] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:09:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:00.216314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:00.231984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e890a-001d-45e4-94e8-a10a0da3f55d service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:00.237199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd57d45-437f-4e3f-9f39-59b34a7e7ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:00.237827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd57d45-437f-4e3f-9f39-59b34a7e7ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:00.238111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd57d45-437f-4e3f-9f39-59b34a7e7ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:00.238370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd57d45-437f-4e3f-9f39-59b34a7e7ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:00.246137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ebd57d45-437f-4e3f-9f39-59b34a7e7ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:00.350617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:00.398420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd57d45-437f-4e3f-9f39-59b34a7e7ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:00.400572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd57d45-437f-4e3f-9f39-59b34a7e7ef9 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:00.476237 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1897/3761] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:04.320120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:04.321688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:04.346003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c373d-a840-4642-be3f-5567c4704b6b service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:04.353807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:04.354905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:04.355844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:04.356228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] PATH_INFO: `/v3/users/d8cc9d27cf6944aa9f5ef81d7fb3c428` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:04.358295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8cc9d27cf6944aa9f5ef81d7fb3c428)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:04.361228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:04.418207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:04.418795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:04.418900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:04.450392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8cc9d27cf6944aa9f5ef81d7fb3c428'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:04.453579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8cc9d27cf6944aa9f5ef81d7fb3c428'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:04.467984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:04.469967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:04.470722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8cc9d27cf6944aa9f5ef81d7fb3c428'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:04.495138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8cc9d27cf6944aa9f5ef81d7fb3c428'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:04.512482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf0351a9-b589-4063-bbf0-b04eefba0d59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8cc9d27cf6944aa9f5ef81d7fb3c428'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:04.539159 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1865/3762] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:04 2022] DELETE /identity/v3/users/d8cc9d27cf6944aa9f5ef81d7fb3c428 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:04.543605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:04.544360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:04.545494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:04.653648 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1898/3763] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:10:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:10:04.689167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:04.690869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:04.699725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:04.702122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:04.854268 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1866/3764] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:04.881181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4218c39-fe90-4676-8f39-3955aebfdac2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:05.073048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977854-18d8-4377-a8ba-fe615bee54da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:05.075958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977854-18d8-4377-a8ba-fe615bee54da service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:05.076364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977854-18d8-4377-a8ba-fe615bee54da service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:05.076872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977854-18d8-4377-a8ba-fe615bee54da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:05.136048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:05.158358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:05.161667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:05.221099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:05.243682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977854-18d8-4377-a8ba-fe615bee54da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:05.246904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977854-18d8-4377-a8ba-fe615bee54da service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:05.292468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1899/3765] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:10:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:05.424396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f64f7b-5e10-4a85-942a-6ee8970d2cba None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:05.449811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:05.454754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:05.456908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:05.460987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:05.509802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:05.516833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:05.567843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1867/3766] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:10:07.712353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:07.712974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:07.713720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:07.812704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] MFA Rules not processed for user `a83af7f06aa94823989c466abc0ffa3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:07.814576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:08.034685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1900/3767] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:08.277822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:08.279946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:08.396590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:08.398615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:08.495868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c0114-0407-48e6-80c7-be60cd67bc88 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:08.516800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:08.518446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:08.518697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d9c4-db23-4c88-a1a1-cbb3144fa3e6 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:08.520034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:08.521083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] PATH_INFO: `/v3/projects/1f83c8276958489e84b0907523036097` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:08.526837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f83c8276958489e84b0907523036097)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:08.535679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:08.545903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a95de-019d-4ff3-8f0f-a726c41656d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:08.547433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a95de-019d-4ff3-8f0f-a726c41656d9 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:08.549706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a95de-019d-4ff3-8f0f-a726c41656d9 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:08.550420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a95de-019d-4ff3-8f0f-a726c41656d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:08.580130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a95de-019d-4ff3-8f0f-a726c41656d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:08.590391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a95de-019d-4ff3-8f0f-a726c41656d9 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:08.614176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1868/3768] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:08.674151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f83c8276958489e84b0907523036097', 'request_id': 'req-303da40c-9ef0-4200-ac45-4e7be0e00ddd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:08.690888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f83c8276958489e84b0907523036097', 'request_id': 'req-303da40c-9ef0-4200-ac45-4e7be0e00ddd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:08.709929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f83c8276958489e84b0907523036097', 'request_id': 'req-303da40c-9ef0-4200-ac45-4e7be0e00ddd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:08.727615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] The token cache is being invalidate because project 1f83c8276958489e84b0907523036097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:08.728253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:08.729230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-303da40c-9ef0-4200-ac45-4e7be0e00ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:08.735084 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1901/3769] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:08 2022] DELETE /identity/v3/projects/1f83c8276958489e84b0907523036097 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:08.736878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99599b3d-5267-48b2-9ce0-d5edd58bd607 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:08.737748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99599b3d-5267-48b2-9ce0-d5edd58bd607 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:08.739086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99599b3d-5267-48b2-9ce0-d5edd58bd607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:08.794465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-99599b3d-5267-48b2-9ce0-d5edd58bd607 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:08.799033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99599b3d-5267-48b2-9ce0-d5edd58bd607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:08.943661 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1869/3770] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:09.052203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ad81-5643-4126-902d-f85b8ad8d66b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:09.053471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ad81-5643-4126-902d-f85b8ad8d66b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:09.054032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ad81-5643-4126-902d-f85b8ad8d66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:09.105474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e0e5ad81-5643-4126-902d-f85b8ad8d66b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:09.108191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e0e5ad81-5643-4126-902d-f85b8ad8d66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:09.146633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1902/3771] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:09.150740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99599b3d-5267-48b2-9ce0-d5edd58bd607 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:09.169863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99599b3d-5267-48b2-9ce0-d5edd58bd607 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:09.183332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:09.184530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:09.185072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:09.187110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:09.187867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:10:09.188239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:09.190172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:09.232729 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1870/3772] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:09.238131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5ad81-5643-4126-902d-f85b8ad8d66b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:09.254967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5ad81-5643-4126-902d-f85b8ad8d66b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:09.275071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:09.275504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:09.275777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:09.276072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:09.293456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:09.300412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:09.495060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:09.496574 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1903/3773] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:09 2022] POST /identity/v3/projects => generated 387 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:09.513034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd7d1f-49f6-4fbf-a001-82dd60326603 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:09.625403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:09.682404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:09.684534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:09.684844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:09.687228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:09.687944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:09.691035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:09.752292 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1871/3774] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:09 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:09.754035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:09.782350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf747a-600d-462e-af77-901d3f5979ad admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:09.805565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:09.806220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:09.806496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:09.806759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:09.811508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:09.813901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:09.834899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:09.880850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1904/3775] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:09.885075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:09.904199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a001c-e0b7-45f3-86d6-d87b3805e936 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:09.932579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f31ce-4860-4de7-8bc7-dd96385fbe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:09.933276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f31ce-4860-4de7-8bc7-dd96385fbe18 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:09.937202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f31ce-4860-4de7-8bc7-dd96385fbe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:09.937396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f31ce-4860-4de7-8bc7-dd96385fbe18 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:09.938483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f31ce-4860-4de7-8bc7-dd96385fbe18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:09.948707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f31ce-4860-4de7-8bc7-dd96385fbe18 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:10.005968 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1872/3776] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:10.010258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:10.032029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb898c1-b7ce-43ec-85e9-429bf354caa3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:10.062512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47914e15-a476-4538-940e-8e330bef5bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:10.063298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47914e15-a476-4538-940e-8e330bef5bba admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:10.063554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47914e15-a476-4538-940e-8e330bef5bba admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:10.065695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47914e15-a476-4538-940e-8e330bef5bba admin admin] PATH_INFO: `/v3/projects/c7046e6f98c54197b8f6a2a660d5ef40/users/93ce045acbbd4ccb858709feef47fdbe/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:10.131037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47914e15-a476-4538-940e-8e330bef5bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7046e6f98c54197b8f6a2a660d5ef40, user_id=93ce045acbbd4ccb858709feef47fdbe, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:10.141882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47914e15-a476-4538-940e-8e330bef5bba admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:10.171704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1905/3777] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:10 2022] PUT /identity/v3/projects/c7046e6f98c54197b8f6a2a660d5ef40/users/93ce045acbbd4ccb858709feef47fdbe/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:10.176861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:10.177230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:10.177402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:10.223661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:10.225860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:10.398188 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1873/3778] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:10.423101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:10.423384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:10.423660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:10.436839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:10.439019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:10.517190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] MFA Rules not processed for user `b11313762b324174a9a5f362d1cf4bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:10.518492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:10.631275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a3c26-b3e7-41fa-ac1c-6d4fcd77ecab None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:10.642034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f1a7e-9c9b-43ab-b1b9-b898a8706f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:10.643018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f1a7e-9c9b-43ab-b1b9-b898a8706f5a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:10.644610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f1a7e-9c9b-43ab-b1b9-b898a8706f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:10.645380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f1a7e-9c9b-43ab-b1b9-b898a8706f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:10.692730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f1a7e-9c9b-43ab-b1b9-b898a8706f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:10.692730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f1a7e-9c9b-43ab-b1b9-b898a8706f5a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:10.709852 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1874/3779] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:10.764847 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1906/3780] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:10.775492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e44db1-542c-44aa-8717-7ee64d5b2991 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:10.775704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e44db1-542c-44aa-8717-7ee64d5b2991 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:10.776156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e44db1-542c-44aa-8717-7ee64d5b2991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:10.822286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-44e44db1-542c-44aa-8717-7ee64d5b2991 None None] MFA Rules not processed for user `b11313762b324174a9a5f362d1cf4bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:10.824382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-44e44db1-542c-44aa-8717-7ee64d5b2991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:10.849006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1875/3781] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:10.860734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:10.862457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:11.043819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1437b-87e8-4ec4-b3aa-ee180d1e6526 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:11.050180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f66f8-03c5-42db-a3d4-8488e249464e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:11.051317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f66f8-03c5-42db-a3d4-8488e249464e service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:11.051804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f66f8-03c5-42db-a3d4-8488e249464e service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:11.052464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f66f8-03c5-42db-a3d4-8488e249464e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:11.063126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f66f8-03c5-42db-a3d4-8488e249464e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:11.065376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f66f8-03c5-42db-a3d4-8488e249464e service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:11.094848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1907/3782] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:12.809965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457903ac-b3dd-497d-82cc-3abfba0dcd4a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:12.809965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457903ac-b3dd-497d-82cc-3abfba0dcd4a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:12.810722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457903ac-b3dd-497d-82cc-3abfba0dcd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:12.859886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-457903ac-b3dd-497d-82cc-3abfba0dcd4a None None] MFA Rules not processed for user `a83af7f06aa94823989c466abc0ffa3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:12.863662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-457903ac-b3dd-497d-82cc-3abfba0dcd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:13.087644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:13.167835 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1876/3783] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:21.073712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:23.117243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:23.675734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0660-68b1-4706-ad1f-5a78910a9ce1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:23.676991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0660-68b1-4706-ad1f-5a78910a9ce1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:23.677423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0660-68b1-4706-ad1f-5a78910a9ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:23.701276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c04b0660-68b1-4706-ad1f-5a78910a9ce1 None None] MFA Rules not processed for user `04a5f2b1d0134e529f25c227954b5f20`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:23.702840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c04b0660-68b1-4706-ad1f-5a78910a9ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:23.839002 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1908/3784] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:23.926734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:23.927308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:23.927782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:23.960329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] MFA Rules not processed for user `04a5f2b1d0134e529f25c227954b5f20`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:23.962186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:23.970346 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1877/3785] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:25.721975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:25.722830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:25.723259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:25.765892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] MFA Rules not processed for user `93ce045acbbd4ccb858709feef47fdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:25.767606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:25.887214 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1909/3786] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:27.272391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:27.274607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:27.293386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a5c5-9910-4a4f-8f5a-7fbe22b31449 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:27.303174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7e05-5473-4617-8289-2295dac0f1d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:27.304142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7e05-5473-4617-8289-2295dac0f1d9 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:27.304632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7e05-5473-4617-8289-2295dac0f1d9 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:27.305091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7e05-5473-4617-8289-2295dac0f1d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:27.307059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-213f7e05-5473-4617-8289-2295dac0f1d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:27.416578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7e05-5473-4617-8289-2295dac0f1d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:27.419007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7e05-5473-4617-8289-2295dac0f1d9 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:27.433690 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1878/3787] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:28.419335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:28.420796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:28.439136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9a35b-f0cb-4d66-b15e-1ca406019f34 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:28.444157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f3fd4-ebb4-48c1-94b8-7c2999551fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:28.444976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f3fd4-ebb4-48c1-94b8-7c2999551fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:28.445571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f3fd4-ebb4-48c1-94b8-7c2999551fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:28.446014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f3fd4-ebb4-48c1-94b8-7c2999551fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:28.448170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e08f3fd4-ebb4-48c1-94b8-7c2999551fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:28.469616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f3fd4-ebb4-48c1-94b8-7c2999551fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:28.471402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f3fd4-ebb4-48c1-94b8-7c2999551fe9 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:28.491291 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1910/3788] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:32.425643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:32.426152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:32.426260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:32.466959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] MFA Rules not processed for user `7f7c5adbac4f48b6bc141e0f888f2496`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:32.470116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:32.489644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1879/3789] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:34.414488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d984-6717-4081-84fe-a2de1184d769 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:34.415111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d984-6717-4081-84fe-a2de1184d769 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:34.415111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d984-6717-4081-84fe-a2de1184d769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:34.486551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2b62d984-6717-4081-84fe-a2de1184d769 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:34.487983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2b62d984-6717-4081-84fe-a2de1184d769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:34.655435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1911/3790] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:10:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:34.668763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:34.670775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:34.816235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e373471-18b3-44ee-8058-ca196a81df6f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:34.829329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:34.830135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:34.830642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:34.831102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:34.852131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:34.853871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:34.872345 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1880/3791] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:10:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:36.472494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d984-6717-4081-84fe-a2de1184d769 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:36.490129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d984-6717-4081-84fe-a2de1184d769 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:36.496964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:36.499025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:36.499572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:36.500635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] PATH_INFO: `/v3/users/93ce045acbbd4ccb858709feef47fdbe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:36.502109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ce045acbbd4ccb858709feef47fdbe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:36.504530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:36.535136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:36.577457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ce045acbbd4ccb858709feef47fdbe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:36.578634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ce045acbbd4ccb858709feef47fdbe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:36.590344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ce045acbbd4ccb858709feef47fdbe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:36.600312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ce045acbbd4ccb858709feef47fdbe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:36.601552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7abd48f-db7e-4261-b071-1263f75afdf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ce045acbbd4ccb858709feef47fdbe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:36.633865 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1912/3792] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:36 2022] DELETE /identity/v3/users/93ce045acbbd4ccb858709feef47fdbe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:36.907524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:36.920331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd5fe5-3362-471b-9d8c-a1413fd555b2 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:37.026816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:37.031768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.032096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.032374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] PATH_INFO: `/v3/projects/c7046e6f98c54197b8f6a2a660d5ef40` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.034084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7046e6f98c54197b8f6a2a660d5ef40)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:37.036045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:37.077297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:37.133651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7046e6f98c54197b8f6a2a660d5ef40', 'request_id': 'req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:37.143747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7046e6f98c54197b8f6a2a660d5ef40', 'request_id': 'req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:37.190080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7046e6f98c54197b8f6a2a660d5ef40', 'request_id': 'req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:37.210338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] The token cache is being invalidate because project c7046e6f98c54197b8f6a2a660d5ef40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:37.210871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:37.212572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-003d8b6d-06b4-46c3-a2a1-5f008a7f0ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:37.216288 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1881/3793] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:36 2022] DELETE /identity/v3/projects/c7046e6f98c54197b8f6a2a660d5ef40 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:37.222219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2082d2-c72a-47fd-ac59-a8bd2f07b29b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.223028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2082d2-c72a-47fd-ac59-a8bd2f07b29b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.223542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2082d2-c72a-47fd-ac59-a8bd2f07b29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.247744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf2082d2-c72a-47fd-ac59-a8bd2f07b29b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:37.249367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf2082d2-c72a-47fd-ac59-a8bd2f07b29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:37.356763 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1913/3794] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:37.357852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a59e3-3768-4fa2-bb50-88f4aae2133a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.358139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a59e3-3768-4fa2-bb50-88f4aae2133a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.358413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a59e3-3768-4fa2-bb50-88f4aae2133a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.425526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-277a59e3-3768-4fa2-bb50-88f4aae2133a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:37.426794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-277a59e3-3768-4fa2-bb50-88f4aae2133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:37.450576 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1882/3795] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:10:37.452358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2082d2-c72a-47fd-ac59-a8bd2f07b29b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:37.469833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2082d2-c72a-47fd-ac59-a8bd2f07b29b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:37.489205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:37.490503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.490937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.491977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.492930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:10:37.493782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:37.495626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:37.526797 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1914/3796] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:37.528027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a59e3-3768-4fa2-bb50-88f4aae2133a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:37.544383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a59e3-3768-4fa2-bb50-88f4aae2133a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:37.553146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:37.554272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.554832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.555648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.559092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:37.568613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:37.625641 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1883/3797] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:37 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:37.629854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:37.643890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9f2f3-8246-4037-9693-555eedf5e1c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:37.783333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:37.784191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.785472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.785955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.786791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:37.790076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:37.823257 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1915/3798] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:37 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:37.827182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:37.841493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f6735-5060-44f4-8eac-c4d01574e1e2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:37.848294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:37.848937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.849325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.849626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.850286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:37.852475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:37.864135 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1884/3799] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:37.868095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:37.883372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eae72a-1952-4d0f-8113-599b60513e7f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:37.896046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d4b57-676d-4c38-a786-05b804bd1ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:37.897746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d4b57-676d-4c38-a786-05b804bd1ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.898347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d4b57-676d-4c38-a786-05b804bd1ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.898790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d4b57-676d-4c38-a786-05b804bd1ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.899747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d4b57-676d-4c38-a786-05b804bd1ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:37.902189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d4b57-676d-4c38-a786-05b804bd1ed2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:37.917227 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1916/3800] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:37.921600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:37.938284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41467a35-8e54-4e56-a8c7-a681f0212fcb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:37.950456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:37.952348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:37.954344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:37.955640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] PATH_INFO: `/v3/projects/9baf52fec08a41afa2b9786c810e5074/users/fa0240e7f6df4ccf9b296fc26a14e347/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:37.985812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9baf52fec08a41afa2b9786c810e5074, user_id=fa0240e7f6df4ccf9b296fc26a14e347, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:37.988485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:38.031825 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1885/3801] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:37 2022] PUT /identity/v3/projects/9baf52fec08a41afa2b9786c810e5074/users/fa0240e7f6df4ccf9b296fc26a14e347/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:38.034639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8132d-f9f7-4366-a993-9917937ddbdb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.035124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8132d-f9f7-4366-a993-9917937ddbdb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.035571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8132d-f9f7-4366-a993-9917937ddbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.046858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:38.080851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0cc8132d-f9f7-4366-a993-9917937ddbdb None None] MFA Rules not processed for user `fa0240e7f6df4ccf9b296fc26a14e347`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:38.082646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0cc8132d-f9f7-4366-a993-9917937ddbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:38.170782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1917/3802] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:38.176038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:38.258398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deddfd-24b1-4f3d-89d5-f57ce80d0793 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:38.266138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:38.266138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.266138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.266138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.266624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:38.268529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:38.302148 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1886/3803] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:38 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:38.306901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8132d-f9f7-4366-a993-9917937ddbdb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:38.329649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8132d-f9f7-4366-a993-9917937ddbdb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:38.414524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:38.415229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.415471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.415746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.416424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:38.418824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:38.465601 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1918/3804] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:38 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:38.469717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:38.488397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389719-dca8-4aaf-85d2-d818f4c784ee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:38.497617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:38.498870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.499259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.499783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.500561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:38.503426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:38.519951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1887/3805] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:38.525375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:38.551754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c1bb6-171a-4ade-84cb-c45f5e50f3f7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:38.560278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:38.562177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.562747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.563758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] PATH_INFO: `/v3/projects/7ca6bfa51cc64f9c8c880fb23ff67ffb/users/20633be47f624a55a1a447260ba7a654/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.597746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca6bfa51cc64f9c8c880fb23ff67ffb, user_id=20633be47f624a55a1a447260ba7a654, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:38.601918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:38.633361 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1919/3806] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:38 2022] PUT /identity/v3/projects/7ca6bfa51cc64f9c8c880fb23ff67ffb/users/20633be47f624a55a1a447260ba7a654/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:38.634395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:38.644966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a7c23-3905-48ac-a01c-04345375af6a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:38.658536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:38.746859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ad6a1-7d13-4de0-a4f6-45b566fa294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:38.747486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ad6a1-7d13-4de0-a4f6-45b566fa294f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.747766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ad6a1-7d13-4de0-a4f6-45b566fa294f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.748038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ad6a1-7d13-4de0-a4f6-45b566fa294f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.748717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ad6a1-7d13-4de0-a4f6-45b566fa294f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:38.750495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ad6a1-7d13-4de0-a4f6-45b566fa294f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:38.768876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:38.772761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1888/3807] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:10:38.780423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b34ad-6ab1-4b0d-bdc9-2b90a489c94b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:38.786308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:38.786999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.787259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.787559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] PATH_INFO: `/v3/projects/7ca6bfa51cc64f9c8c880fb23ff67ffb/users/20633be47f624a55a1a447260ba7a654/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.789842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca6bfa51cc64f9c8c880fb23ff67ffb, user_id=20633be47f624a55a1a447260ba7a654, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:38.793889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:38.809973 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1920/3808] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:38 2022] PUT /identity/v3/projects/7ca6bfa51cc64f9c8c880fb23ff67ffb/users/20633be47f624a55a1a447260ba7a654/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:38.820720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a322a-c301-40f9-87de-59abe66003a1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.820720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a322a-c301-40f9-87de-59abe66003a1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.820720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a322a-c301-40f9-87de-59abe66003a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.862744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-262a322a-c301-40f9-87de-59abe66003a1 None None] MFA Rules not processed for user `20633be47f624a55a1a447260ba7a654`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:38.864146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-262a322a-c301-40f9-87de-59abe66003a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:38.971167 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1889/3809] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:38.976011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:38.987782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52894a-0e24-421a-812a-232fa58a7b4e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:38.992904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:38.993806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:38.994240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:38.994515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:38.995162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:38.997214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.039012 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1921/3810] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:10:38 2022] POST /identity/v3/projects => generated 348 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:39.056699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a322a-c301-40f9-87de-59abe66003a1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.056699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a322a-c301-40f9-87de-59abe66003a1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.117847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.118412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.118720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.118964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.119929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.121758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.124398 ncdv-master-1 devstack@keystone.service[104131]: WARNING keystone.server.flask.common [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 27 10:10:39.161192 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1890/3811] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:39 2022] POST /identity/v3/projects => generated 351 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:39.165917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.177853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333a98c-c582-4471-9afa-71afdcb4e389 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.202250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:39.272780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.273603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.273951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.274290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/users/fa0240e7f6df4ccf9b296fc26a14e347` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.275564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:get_user(user_id=fa0240e7f6df4ccf9b296fc26a14e347)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.278459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.281258 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1922/3812] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:10:39 2022] GET /identity/v3/users/fa0240e7f6df4ccf9b296fc26a14e347 => generated 288 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:10:39.286309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.304108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1c045-d853-491a-9d7f-9bba255ed2f3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.312052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.313080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.313535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.314057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.314709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.316916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.358576 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1891/3813] 10.222.0.26 () {60 vars in 1162 bytes} [Tue Sep 27 10:10:39 2022] POST /identity/v3/credentials => generated 430 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:39.363047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.374392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc0352-f8ad-46cd-b7ef-35cc0cf35f78 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.380090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.381240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.381529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.381984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials/2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.384523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.386429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.389858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.392278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.412685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1923/3814] 10.222.0.26 () {60 vars in 1358 bytes} [Tue Sep 27 10:10:39 2022] PATCH /identity/v3/credentials/2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96 => generated 437 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:39.415800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.426066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1bf1a-8cb2-4b42-b1ba-75530bbc5ca6 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.431046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.431682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.431963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.432235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials/2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.434308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.436036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.438301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1892/3815] 10.222.0.26 () {58 vars in 1335 bytes} [Tue Sep 27 10:10:39 2022] GET /identity/v3/credentials/2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96 => generated 437 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:39.443425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.454432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aace7ce-1b7d-41ce-916f-f0fceee323f2 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.459283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.459913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.460404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.460689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials/2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.462781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.464524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.485110 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1924/3816] 10.222.0.26 () {58 vars in 1338 bytes} [Tue Sep 27 10:10:39 2022] DELETE /identity/v3/credentials/2bb74fdd146797baaa8269450ca42a34ef34009197b2205ef4758e43e6581a96 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:39.490346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.498185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:39.502158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923c2f6-843d-45a1-9532-6808354cd878 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.509367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.510515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.511249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.511660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.512426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.514683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.532677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1893/3817] 10.222.0.26 () {60 vars in 1162 bytes} [Tue Sep 27 10:10:39 2022] POST /identity/v3/credentials => generated 431 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:39.537275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.548105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210f27d-7c1f-4cd3-b083-20888f3565d3 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.552647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.553268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.553540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.554129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.554770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.556518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.580385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1925/3818] 10.222.0.26 () {60 vars in 1162 bytes} [Tue Sep 27 10:10:39 2022] POST /identity/v3/credentials => generated 426 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:39.581716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.592709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b8783-b10f-4cea-a485-ee2551a4665f tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.597765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.598377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.598657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.598949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.599552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.601406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.630617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.632506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.634376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.651067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.651067 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1894/3819] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 10:10:39 2022] GET /identity/v3/credentials => generated 942 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:39.651472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.658085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1db8eb-74dd-44eb-9f6a-84bbdbe811af tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.663490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.664182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.664530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.664909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials/d4099b128465e72135972671b00fb767ac2286a5fe64a2ee23fb6bb38a213d89` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.667861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=d4099b128465e72135972671b00fb767ac2286a5fe64a2ee23fb6bb38a213d89)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.670295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.702782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.703360 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1926/3820] 10.222.0.26 () {58 vars in 1338 bytes} [Tue Sep 27 10:10:39 2022] DELETE /identity/v3/credentials/d4099b128465e72135972671b00fb767ac2286a5fe64a2ee23fb6bb38a213d89 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:39.718531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836543b0-7833-47d6-8347-7f2d2f5b849c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.727528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.728480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.728480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.729039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/credentials/07e5f655f7ceb24b7e46ad444d228f898ba37296d5fb2f7ba8113431f6cf1ead` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.731438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=07e5f655f7ceb24b7e46ad444d228f898ba37296d5fb2f7ba8113431f6cf1ead)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.733402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.775046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.775985 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1895/3821] 10.222.0.26 () {58 vars in 1338 bytes} [Tue Sep 27 10:10:39 2022] DELETE /identity/v3/credentials/07e5f655f7ceb24b7e46ad444d228f898ba37296d5fb2f7ba8113431f6cf1ead => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:39.792451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897dc7f-ca70-4e75-b0a2-556ed80efdf9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:39.799302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:39.800309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:39.800744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:39.801081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/projects/3f64afa6346f4bd59eced556449ac77f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:39.802356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3f64afa6346f4bd59eced556449ac77f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:39.805851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:39.900649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f64afa6346f4bd59eced556449ac77f', 'request_id': 'req-85c1344a-d100-4650-88ec-faa27b7c344c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:39.916153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f64afa6346f4bd59eced556449ac77f', 'request_id': 'req-85c1344a-d100-4650-88ec-faa27b7c344c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:39.927267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f64afa6346f4bd59eced556449ac77f', 'request_id': 'req-85c1344a-d100-4650-88ec-faa27b7c344c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:39.953027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] The token cache is being invalidate because project 3f64afa6346f4bd59eced556449ac77f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:39.953456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:39.954598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:39.959837 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1927/3822] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:39 2022] DELETE /identity/v3/projects/3f64afa6346f4bd59eced556449ac77f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:39.961646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:39.968721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:39.990959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995055-f4c2-4bcf-bd9c-120b0d389da9 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:40.107421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20633be47f624a55a1a447260ba7a654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:40.108086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:40.108359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:40.108777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] PATH_INFO: `/v3/projects/e00aede719f64add89a6415e4df18a09` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:40.110144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e00aede719f64add89a6415e4df18a09)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:40.112814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:40.184451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00aede719f64add89a6415e4df18a09', 'request_id': 'req-677f7039-903e-491a-9eb6-ee89ed5c0447'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.196155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00aede719f64add89a6415e4df18a09', 'request_id': 'req-677f7039-903e-491a-9eb6-ee89ed5c0447'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.235784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00aede719f64add89a6415e4df18a09', 'request_id': 'req-677f7039-903e-491a-9eb6-ee89ed5c0447'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.259833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] The token cache is being invalidate because project e00aede719f64add89a6415e4df18a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:40.260517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.261913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-677f7039-903e-491a-9eb6-ee89ed5c0447 tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.264592 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1896/3823] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:39 2022] DELETE /identity/v3/projects/e00aede719f64add89a6415e4df18a09 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:40.269019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:40.273225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:40.366875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=9,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:40.372510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1344a-d100-4650-88ec-faa27b7c344c tempest-CredentialsTestJSON-105112022 tempest-CredentialsTestJSON-105112022-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:40.443277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:40.445595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:40.445882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:40.446296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] PATH_INFO: `/v3/users/fa0240e7f6df4ccf9b296fc26a14e347` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:40.448151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa0240e7f6df4ccf9b296fc26a14e347)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:40.450312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:40.553820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa0240e7f6df4ccf9b296fc26a14e347'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.554576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa0240e7f6df4ccf9b296fc26a14e347'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.567414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa0240e7f6df4ccf9b296fc26a14e347'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.592269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa0240e7f6df4ccf9b296fc26a14e347'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.593473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa0240e7f6df4ccf9b296fc26a14e347'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:40.643889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1928/3824] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:40 2022] DELETE /identity/v3/users/fa0240e7f6df4ccf9b296fc26a14e347 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:40.647328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:40.647715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:40.648055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:40.671987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:40.675179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:40.769602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:40.780870 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1897/3825] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:10:40.802757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:40.803943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:40.890282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309951af-4e06-4cdd-a4da-83ff16d4692c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:40.909329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:40.911571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:40.911834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:40.912131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:40.938815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:40.940284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:40.958718 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1929/3826] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:42.877659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:42.879468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:43.067017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd712-c7d3-446a-8094-2285a19518b6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:43.074880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:43.075789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:43.076419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:43.076937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] PATH_INFO: `/v3/projects/9baf52fec08a41afa2b9786c810e5074` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:43.078397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9baf52fec08a41afa2b9786c810e5074)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:43.081553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:43.169646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9baf52fec08a41afa2b9786c810e5074', 'request_id': 'req-01b62508-253f-4778-801f-e2962ff391ba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.181981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9baf52fec08a41afa2b9786c810e5074', 'request_id': 'req-01b62508-253f-4778-801f-e2962ff391ba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.199683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9baf52fec08a41afa2b9786c810e5074', 'request_id': 'req-01b62508-253f-4778-801f-e2962ff391ba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.224431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] The token cache is being invalidate because project 9baf52fec08a41afa2b9786c810e5074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:43.225078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.226251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.229167 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1898/3827] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:42 2022] DELETE /identity/v3/projects/9baf52fec08a41afa2b9786c810e5074 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:43.232994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:43.266850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:43.297294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a479d-e16d-45bb-80d6-0a7e011c6395 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:43.368852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:43.369768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:43.370043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:43.370315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] PATH_INFO: `/v3/users/20633be47f624a55a1a447260ba7a654` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:43.371446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=20633be47f624a55a1a447260ba7a654)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:43.373404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:43.435704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20633be47f624a55a1a447260ba7a654'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.436867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20633be47f624a55a1a447260ba7a654'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.448958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20633be47f624a55a1a447260ba7a654'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.457866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20633be47f624a55a1a447260ba7a654'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.458603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-756fdd39-cd9d-417b-ae64-e91d8c601b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20633be47f624a55a1a447260ba7a654'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:43.479891 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1930/3828] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:43 2022] DELETE /identity/v3/users/20633be47f624a55a1a447260ba7a654 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:43.897324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:43.900383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:43.916790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b62508-253f-4778-801f-e2962ff391ba admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.033980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.034935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.035316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.035943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] PATH_INFO: `/v3/projects/7ca6bfa51cc64f9c8c880fb23ff67ffb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.037488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca6bfa51cc64f9c8c880fb23ff67ffb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.039710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.065048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:44.097582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'request_id': 'req-368b66d4-c732-4c1c-af8d-355ada9f48d1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:44.113769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'request_id': 'req-368b66d4-c732-4c1c-af8d-355ada9f48d1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:44.123508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca6bfa51cc64f9c8c880fb23ff67ffb', 'request_id': 'req-368b66d4-c732-4c1c-af8d-355ada9f48d1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:44.134906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] The token cache is being invalidate because project 7ca6bfa51cc64f9c8c880fb23ff67ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:44.135194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:44.135839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368b66d4-c732-4c1c-af8d-355ada9f48d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:44.137373 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1899/3829] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:43 2022] DELETE /identity/v3/projects/7ca6bfa51cc64f9c8c880fb23ff67ffb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:44.142554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ea3a-2560-4264-bf37-f2561c56c678 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.142774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ea3a-2560-4264-bf37-f2561c56c678 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.143048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ea3a-2560-4264-bf37-f2561c56c678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.220381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a275ea3a-2560-4264-bf37-f2561c56c678 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:44.221837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a275ea3a-2560-4264-bf37-f2561c56c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:44.269058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:44.302366 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1931/3830] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:44.311354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c8575-0bec-46fe-9fc3-804fce052a90 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.311632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c8575-0bec-46fe-9fc3-804fce052a90 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.311900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c8575-0bec-46fe-9fc3-804fce052a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.334349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-598c8575-0bec-46fe-9fc3-804fce052a90 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:44.335740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-598c8575-0bec-46fe-9fc3-804fce052a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:44.349893 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1900/3831] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:44.350114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275ea3a-2560-4264-bf37-f2561c56c678 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.361823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275ea3a-2560-4264-bf37-f2561c56c678 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.367206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.368030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.368313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.368595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.369272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:10:44.376270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.382693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.401708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1932/3832] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:44.405860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c8575-0bec-46fe-9fc3-804fce052a90 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.416358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c8575-0bec-46fe-9fc3-804fce052a90 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.421423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.421772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.422043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.422310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.422939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.424868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.465182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1901/3833] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:44 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:44.469661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.482024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc087a-0cae-42bf-bc12-fd6cff1e9343 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.542332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.543118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.543496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.543859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.544565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.546623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.570466 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1933/3834] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:44 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:44.574792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.585660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf11cf-f4f5-4ef0-8863-f7e943b85b33 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.591438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.592130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.592406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.592670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.593332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.594979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.613941 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1902/3835] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:10:44.614977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.626630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42393df-760c-4e96-bf62-c3b8f5f36d61 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.632061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ab3f6-7472-4b26-b5b2-1d250851b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.632975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab3f6-7472-4b26-b5b2-1d250851b006 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.633272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab3f6-7472-4b26-b5b2-1d250851b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.633541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ab3f6-7472-4b26-b5b2-1d250851b006 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.634195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ab3f6-7472-4b26-b5b2-1d250851b006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.635803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ab3f6-7472-4b26-b5b2-1d250851b006 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.650435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1934/3836] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:44.654706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.665697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5f49-33c9-4e67-a525-a93f8c0ac123 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.671517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.672281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.672626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.673274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] PATH_INFO: `/v3/projects/4facec1eda6e4d8697c148d89db320a8/users/1b45e1a534e741c3900c12ad44ba8aca/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.694137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4facec1eda6e4d8697c148d89db320a8, user_id=1b45e1a534e741c3900c12ad44ba8aca, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.697479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.729144 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1903/3837] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:44 2022] PUT /identity/v3/projects/4facec1eda6e4d8697c148d89db320a8/users/1b45e1a534e741c3900c12ad44ba8aca/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:44.739645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf472e-e11d-4fdd-9beb-d21a6b6a7eff None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.740197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf472e-e11d-4fdd-9beb-d21a6b6a7eff None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.740695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf472e-e11d-4fdd-9beb-d21a6b6a7eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.770324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c7bf472e-e11d-4fdd-9beb-d21a6b6a7eff None None] MFA Rules not processed for user `1b45e1a534e741c3900c12ad44ba8aca`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:44.771647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7bf472e-e11d-4fdd-9beb-d21a6b6a7eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:44.847315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1935/3838] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:44.851941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.920305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b313f03-839e-4ab7-85ef-02745a399f44 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:44.925519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:44.926143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:44.926417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:44.926684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:44.927328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:44.929491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:44.964217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1904/3839] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:44 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:44.965662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf472e-e11d-4fdd-9beb-d21a6b6a7eff None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:44.979967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf472e-e11d-4fdd-9beb-d21a6b6a7eff None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.043470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:45.075956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.076571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.077491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.077771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.078398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.080334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.111632 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1936/3840] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:44 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:45.112885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.128640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad0039-9776-49c5-a55f-a2c805e49064 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.139230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.140143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.140649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.141815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.144918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.146989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.167549 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1905/3841] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:45.169397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.184734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4264d-9d50-47ad-81f1-07498bdb633d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.193321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.194412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.194794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.195167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] PATH_INFO: `/v3/projects/a58256ff69ff4134b1d959a59dc1a1bf/users/1d1409c8718946d880d6797a587270ff/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.214972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58256ff69ff4134b1d959a59dc1a1bf, user_id=1d1409c8718946d880d6797a587270ff, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.218439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.236702 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1937/3842] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:45 2022] PUT /identity/v3/projects/a58256ff69ff4134b1d959a59dc1a1bf/users/1d1409c8718946d880d6797a587270ff/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:45.240006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.256360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee525cb1-aa51-4b96-8c85-dd55230d5bdd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.264585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:45.316282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abad3f-420a-4ada-93a9-aa0a29139133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.317739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abad3f-420a-4ada-93a9-aa0a29139133 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.318367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abad3f-420a-4ada-93a9-aa0a29139133 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.318642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abad3f-420a-4ada-93a9-aa0a29139133 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.319397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abad3f-420a-4ada-93a9-aa0a29139133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.321165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abad3f-420a-4ada-93a9-aa0a29139133 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.332172 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1906/3843] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:10:45.336348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.348987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e69592-bd52-4fdf-8257-fd61ccd315cf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.354722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.355469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.355809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.356258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] PATH_INFO: `/v3/projects/a58256ff69ff4134b1d959a59dc1a1bf/users/1d1409c8718946d880d6797a587270ff/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.358385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58256ff69ff4134b1d959a59dc1a1bf, user_id=1d1409c8718946d880d6797a587270ff, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.361155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.372016 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1938/3844] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:45 2022] PUT /identity/v3/projects/a58256ff69ff4134b1d959a59dc1a1bf/users/1d1409c8718946d880d6797a587270ff/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:45.383029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e736-5d1e-41e4-86d6-ec7fadf7bb29 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.383318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e736-5d1e-41e4-86d6-ec7fadf7bb29 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.383592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e736-5d1e-41e4-86d6-ec7fadf7bb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.415368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8151e736-5d1e-41e4-86d6-ec7fadf7bb29 None None] MFA Rules not processed for user `1d1409c8718946d880d6797a587270ff`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:45.416611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8151e736-5d1e-41e4-86d6-ec7fadf7bb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:45.505405 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1907/3845] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:45.506700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.523790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d7125-ccc3-4dda-baa7-4d1463db0ff9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.528920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.529616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.529987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.530323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.531052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.532238 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:45.532238 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:10:45.532238 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:10:45.533225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.571579 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1939/3846] 10.222.0.26 () {60 vars in 1150 bytes} [Tue Sep 27 10:10:45 2022] POST /identity/v3/domains => generated 276 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:45.578653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e736-5d1e-41e4-86d6-ec7fadf7bb29 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.588976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e736-5d1e-41e4-86d6-ec7fadf7bb29 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.673290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.673926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.674206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.674486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.675289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.689408 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:45.689408 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:10:45.689408 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:10:45.690788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.720129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1908/3847] 10.222.0.26 () {60 vars in 1150 bytes} [Tue Sep 27 10:10:45 2022] POST /identity/v3/domains => generated 277 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:45.724357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.746430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0317143-d1dd-4482-8f78-6539fcc52ee6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.820719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.821343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.821625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.821896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.822565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.824211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:45.846689 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1940/3848] 10.222.0.26 () {60 vars in 1150 bytes} [Tue Sep 27 10:10:45 2022] POST /identity/v3/domains => generated 278 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:45.848071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:45.869983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed096e3-24f3-462d-ac16-81aea15d3eda tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:45.914097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:45.989857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:45.990888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:45.991996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:45.992696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:45.995202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:45.997503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:46.040740 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1909/3849] 10.222.0.26 () {60 vars in 1150 bytes} [Tue Sep 27 10:10:45 2022] POST /identity/v3/domains => generated 279 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:46.045540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:46.075257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d1339-f614-4d2e-9ca8-7b1c29eb8ecb tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:46.165568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:46.166217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:46.166495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:46.166773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/a3e35c76584642429d1a5bf025fdecad` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:46.178294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a3e35c76584642429d1a5bf025fdecad)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:46.180145 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:46.180145 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:10:46.180145 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:10:46.181117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:46.182905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:46.218261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.239720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.292944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad', 'request_id': 'req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.307668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad', 'request_id': 'req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.320932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad', 'request_id': 'req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.335196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] The token cache is being invalidate because project a3e35c76584642429d1a5bf025fdecad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:46.335547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.336263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.350913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad', 'request_id': 'req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.352102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad', 'request_id': 'req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.356977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a3e35c76584642429d1a5bf025fdecad', 'request_id': 'req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.364216 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1941/3850] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:46 2022] DELETE /identity/v3/domains/a3e35c76584642429d1a5bf025fdecad => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:46.369836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:46.371611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:46.386845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce59167-d8c5-4387-b63f-f68a875a4638 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:46.464866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:46.465576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:46.465766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:46.466039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:46.466708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:46.469257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:46.497393 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1910/3851] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:10:46 2022] POST /identity/v3/domains => generated 249 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:46.501713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:46.516148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae90d4-0b76-4b9f-967b-ce70c333ffa8 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:46.579140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:46.580473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:46.581776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:46.581776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/5811bde5e19f4a96b36b23aaa631a865` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:46.595718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=5811bde5e19f4a96b36b23aaa631a865)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:46.597243 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:46.597243 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:10:46.597243 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:10:46.604476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:46.635504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.636394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.682360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.683331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.685686 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1942/3852] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:46 2022] PATCH /identity/v3/domains/5811bde5e19f4a96b36b23aaa631a865 => generated 250 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:46.689970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:46.691851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:46.711957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4aa11-5442-4893-87d5-c9a2f455ee9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:46.782581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:46.783300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:46.783586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:46.783863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/5811bde5e19f4a96b36b23aaa631a865` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:46.813248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=5811bde5e19f4a96b36b23aaa631a865)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:46.814598 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:46.814598 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:10:46.814598 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:10:46.816450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:46.860909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.867737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.906050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:46.941275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865', 'request_id': 'req-0182a515-ef60-4c9f-9c4e-fbc578a1582c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.947517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865', 'request_id': 'req-0182a515-ef60-4c9f-9c4e-fbc578a1582c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.957445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865', 'request_id': 'req-0182a515-ef60-4c9f-9c4e-fbc578a1582c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.971248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] The token cache is being invalidate because project 5811bde5e19f4a96b36b23aaa631a865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:46.971612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.972407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.985615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865', 'request_id': 'req-0182a515-ef60-4c9f-9c4e-fbc578a1582c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.986446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865', 'request_id': 'req-0182a515-ef60-4c9f-9c4e-fbc578a1582c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:46.990831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '5811bde5e19f4a96b36b23aaa631a865', 'request_id': 'req-0182a515-ef60-4c9f-9c4e-fbc578a1582c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.003139 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1911/3853] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:46 2022] DELETE /identity/v3/domains/5811bde5e19f4a96b36b23aaa631a865 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:47.004461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:47.015013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:47.026597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6c2b-b1b4-47c3-ad77-e4181760749c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:47.126561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:47.127189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:47.127564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:47.127841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:47.137643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:47.139899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:47.166757 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1943/3854] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:10:47 2022] POST /identity/v3/domains => generated 278 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:47.171066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:47.172520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:47.186527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182a515-ef60-4c9f-9c4e-fbc578a1582c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:47.282290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:47.283051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:47.283575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:47.283954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:47.301224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d5ae76ebfbb74c9faf6c51ddd0238dd6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:47.302348 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:47.302348 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:10:47.302348 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:10:47.303422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:47.340553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.341338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.390951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.391510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.394997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1912/3855] 10.222.0.26 () {60 vars in 1250 bytes} [Tue Sep 27 10:10:47 2022] PATCH /identity/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6 => generated 279 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:47.399050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:47.402596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:47.435515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecbb5b-96fb-46b8-b931-e07477a8ae9f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:47.556709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:47.557611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:47.557906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:47.558157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:47.572698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=d5ae76ebfbb74c9faf6c51ddd0238dd6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:47.574897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:47.576690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1944/3856] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:10:47 2022] GET /identity/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6 => generated 279 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:47.580816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:47.591221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4859b4f-65c5-4935-94a9-6421d02fee02 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:47.596225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:47.597299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:47.597594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:47.597882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:47.609698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d5ae76ebfbb74c9faf6c51ddd0238dd6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:47.610220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:47.654296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.659921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.688293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:47.777153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6', 'request_id': 'req-02e84fe8-b461-4483-870c-7d42bf95e743'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.805940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6', 'request_id': 'req-02e84fe8-b461-4483-870c-7d42bf95e743'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.825641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6', 'request_id': 'req-02e84fe8-b461-4483-870c-7d42bf95e743'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.842861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] The token cache is being invalidate because project d5ae76ebfbb74c9faf6c51ddd0238dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:47.843813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.845493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.858512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6', 'request_id': 'req-02e84fe8-b461-4483-870c-7d42bf95e743'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.859684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6', 'request_id': 'req-02e84fe8-b461-4483-870c-7d42bf95e743'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.864349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd5ae76ebfbb74c9faf6c51ddd0238dd6', 'request_id': 'req-02e84fe8-b461-4483-870c-7d42bf95e743'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:47.881269 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1913/3857] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:47 2022] DELETE /identity/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:47.881497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:47.887894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:47.913004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3917b-c322-4950-98d7-11b1f7902a4f tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:48.031438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:48.032302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:48.032596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:48.032898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:48.033538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:48.035188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:48.057234 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1945/3858] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 10:10:47 2022] GET /identity/v3/domains => generated 1349 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:48.061330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:48.072337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84fe8-b461-4483-870c-7d42bf95e743 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:48.088162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:48.090695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:48.091049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:48.091600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:48.108719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find domain: d5ae76ebfbb74c9faf6c51ddd0238dd6.: keystone.exception.DomainNotFound: Could not find domain: d5ae76ebfbb74c9faf6c51ddd0238dd6. Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d5ae76ebfbb74c9faf6c51ddd0238dd6. Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:48.166860 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:48.173231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d5ae76ebfbb74c9faf6c51ddd0238dd6. Sep 27 10:10:48.174562 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:48.175921 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1914/3859] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:48 2022] PATCH /identity/v3/domains/d5ae76ebfbb74c9faf6c51ddd0238dd6 => generated 112 bytes in 108 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:10:48.176591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:48.207471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37439688-d9c1-4357-9c9f-8fdf07376ed0 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:48.244956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:48.245605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:48.245886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:48.246160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:48.246845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:48.253002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:48.307698 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1946/3860] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:10:48 2022] POST /identity/v3/domains => generated 277 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:48.311882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:48.340453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d99ba-8a85-494c-aba0-b80c18e49e10 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:48.465512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:48.466104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:48.466386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:48.466658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:48.467324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:48.472813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:48.565073 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1915/3861] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:48 2022] POST /identity/v3/users => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:48.565405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:48.580504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3357e95-9e60-41f4-b29f-e0bd2ea9fc70 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:48.600076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:48.609691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:48.609994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:48.610371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:48.611180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:48.612308 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:48.612308 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:10:48.612308 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:10:48.613924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:48.632926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:48.650954 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1947/3862] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:10:48 2022] POST /identity/v3/groups => generated 324 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:48.654863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:48.670552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18972d-3103-4d7a-b5fd-7b380d11d720 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:48.683364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:48.684579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:48.684889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:48.685167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/5bdd89ccb04345b197f336ec88a5a71d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:48.690244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=5bdd89ccb04345b197f336ec88a5a71d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:48.690244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:48.734379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:48.736102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:48.776474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:48.777502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:48.780511 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1916/3863] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:48 2022] PATCH /identity/v3/domains/5bdd89ccb04345b197f336ec88a5a71d => generated 278 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:48.784461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:48.787248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:48.802639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78122f4e-f666-469b-92c7-b2af148074b9 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:48.887129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:48.888468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:48.889594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:48.890775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/5bdd89ccb04345b197f336ec88a5a71d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:48.923514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=5bdd89ccb04345b197f336ec88a5a71d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:48.926568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:48.981979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.167407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431c25d762614729bd8a595012a2643c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.168116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431c25d762614729bd8a595012a2643c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.179299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431c25d762614729bd8a595012a2643c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.187026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431c25d762614729bd8a595012a2643c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.187827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431c25d762614729bd8a595012a2643c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.190442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:49.227637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.312605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d', 'request_id': 'req-90d47c40-9f48-4613-b013-9372160b5402'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.335923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d', 'request_id': 'req-90d47c40-9f48-4613-b013-9372160b5402'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.346834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d', 'request_id': 'req-90d47c40-9f48-4613-b013-9372160b5402'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.372822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] The token cache is being invalidate because project 5bdd89ccb04345b197f336ec88a5a71d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:49.373115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.374223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.387077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d', 'request_id': 'req-90d47c40-9f48-4613-b013-9372160b5402'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.388197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d', 'request_id': 'req-90d47c40-9f48-4613-b013-9372160b5402'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.392267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '5bdd89ccb04345b197f336ec88a5a71d', 'request_id': 'req-90d47c40-9f48-4613-b013-9372160b5402'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:49.399521 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1948/3864] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:48 2022] DELETE /identity/v3/domains/5bdd89ccb04345b197f336ec88a5a71d => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:49.403413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.415022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f0842-1b85-415c-a0a3-67e888698c7a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.429747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=11,keystone:INSERT=4,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:49.498623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.499355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.499641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.499920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/5bdd89ccb04345b197f336ec88a5a71d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.529764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=5bdd89ccb04345b197f336ec88a5a71d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:49.532479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d.: keystone.exception.DomainNotFound: Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:49.550234 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:49.551655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:49.552981 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:49.556490 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1917/3865] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:10:49 2022] GET /identity/v3/domains/5bdd89ccb04345b197f336ec88a5a71d => generated 112 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:10:49.561119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.563094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:49.583199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d47c40-9f48-4613-b013-9372160b5402 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.588491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.589194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.589470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.589740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/users/431c25d762614729bd8a595012a2643c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.607683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:get_user(user_id=431c25d762614729bd8a595012a2643c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:49.609826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find user: 431c25d762614729bd8a595012a2643c.: keystone.exception.UserNotFound: Could not find user: 431c25d762614729bd8a595012a2643c. Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 10:10:49.633655 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:49.634880 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 431c25d762614729bd8a595012a2643c. Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:49.636739 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1949/3866] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:10:49 2022] GET /identity/v3/users/431c25d762614729bd8a595012a2643c => generated 110 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:10:49.639544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.650987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25554d4c-f4a5-456c-a09a-4b03839e5b2d tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.656149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.657333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.658263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.658616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/groups/5b888b58e3254ef4bd553d54b9eb9fd1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.667463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:get_group(group_id=5b888b58e3254ef4bd553d54b9eb9fd1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:49.668580 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:49.668580 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:10:49.668580 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:10:49.669987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find group: 5b888b58e3254ef4bd553d54b9eb9fd1.: keystone.exception.GroupNotFound: Could not find group: 5b888b58e3254ef4bd553d54b9eb9fd1. Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_group(group_id) Sep 27 10:10:49.691883 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:49.693490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 5b888b58e3254ef4bd553d54b9eb9fd1. Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:49.694634 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1918/3867] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:49 2022] GET /identity/v3/groups/5b888b58e3254ef4bd553d54b9eb9fd1 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:10:49.700061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.714350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2135a0-c8ed-4c45-af1e-0376ca9b39a5 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.719041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.719729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.720016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.720301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/groups/5b888b58e3254ef4bd553d54b9eb9fd1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.728540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find group: 5b888b58e3254ef4bd553d54b9eb9fd1.: keystone.exception.GroupNotFound: Could not find group: 5b888b58e3254ef4bd553d54b9eb9fd1. Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Sep 27 10:10:49.734100 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:49.735177 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 5b888b58e3254ef4bd553d54b9eb9fd1. Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:49.739970 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1950/3868] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:10:49 2022] DELETE /identity/v3/groups/5b888b58e3254ef4bd553d54b9eb9fd1 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:10:49.741056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.752803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311abf8-1447-4af9-8b59-7b6cdf3c5f17 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.760358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.762161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.762482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.762834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/users/431c25d762614729bd8a595012a2643c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.774161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_user(user_id=431c25d762614729bd8a595012a2643c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:49.776563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find user: 431c25d762614729bd8a595012a2643c.: keystone.exception.UserNotFound: Could not find user: 431c25d762614729bd8a595012a2643c. Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Sep 27 10:10:49.789289 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:49.790498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 431c25d762614729bd8a595012a2643c. Sep 27 10:10:49.791895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:10:49.795662 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1919/3869] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:49 2022] DELETE /identity/v3/users/431c25d762614729bd8a595012a2643c => generated 110 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:10:49.798291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.810132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc77ead-d503-4f7b-8ecb-52c7dd310f46 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.816091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.816775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.817058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.817330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/5bdd89ccb04345b197f336ec88a5a71d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.828034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d.: keystone.exception.DomainNotFound: Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:49.835184 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:49.837927 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:49.839062 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:49.840350 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1951/3870] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:49 2022] PATCH /identity/v3/domains/5bdd89ccb04345b197f336ec88a5a71d => generated 112 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:10:49.841767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.859304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43330f46-77a2-444a-bb00-ebe0a710c1f2 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.869230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.870153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.870431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.870705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.871613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:49.874826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:49.899840 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1920/3871] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 10:10:49 2022] GET /identity/v3/domains => generated 1349 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:49.909553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:49.924978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47a855-3670-49f5-aac4-cd1227572c18 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:49.932791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:49.933476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:49.933875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:49.934192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:49.935178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:10:49.935979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:49.943726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:49.988417 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1952/3872] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:10:49 2022] GET /identity/v3/domains?enabled=True => generated 1095 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:49.992310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:50.021466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18159f37-1e76-4c73-9608-444e82678f3b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:50.027857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:50.028501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:50.028853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:50.029132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:50.029905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-407282568) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:10:50.030215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:50.032071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:50.047799 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1921/3873] 10.222.0.26 () {58 vars in 1197 bytes} [Tue Sep 27 10:10:49 2022] GET /identity/v3/domains?name=tempest-test_domain-407282568 => generated 407 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:50.052845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:50.072573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43b78-86c2-4b77-b5bf-2c0a53c138ba tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:50.085755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:50.086408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:50.086826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:50.087109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/5bdd89ccb04345b197f336ec88a5a71d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:50.099770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d.: keystone.exception.DomainNotFound: Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:10:50.106652 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:10:50.107734 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 5bdd89ccb04345b197f336ec88a5a71d. Sep 27 10:10:50.108780 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:10:50.116513 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1953/3874] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:50 2022] PATCH /identity/v3/domains/5bdd89ccb04345b197f336ec88a5a71d => generated 112 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:10:50.122562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:50.151782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4c4e4-2f02-4139-8941-a23135c7c971 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:50.161444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:50.162101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:50.162371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:50.162643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/143d05869b5b474d821fd81dc4c7b960` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:50.177454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=143d05869b5b474d821fd81dc4c7b960)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:50.179553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:50.243489 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1922/3875] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:50 2022] PATCH /identity/v3/domains/143d05869b5b474d821fd81dc4c7b960 => generated 278 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:10:50.247068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:50.263770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c032-534b-4a2d-840e-e4c01dc4b010 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:50.269386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:50.270110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:50.270456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:50.270794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/143d05869b5b474d821fd81dc4c7b960` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:50.283247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=143d05869b5b474d821fd81dc4c7b960)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:50.284914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:50.314113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '143d05869b5b474d821fd81dc4c7b960'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.337418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '143d05869b5b474d821fd81dc4c7b960'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.360826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:50.391013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '143d05869b5b474d821fd81dc4c7b960', 'request_id': 'req-42e1030a-9c53-4721-a91b-c9218b797b6c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.403970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '143d05869b5b474d821fd81dc4c7b960', 'request_id': 'req-42e1030a-9c53-4721-a91b-c9218b797b6c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.411341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '143d05869b5b474d821fd81dc4c7b960', 'request_id': 'req-42e1030a-9c53-4721-a91b-c9218b797b6c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.426398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] The token cache is being invalidate because project 143d05869b5b474d821fd81dc4c7b960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:50.426692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.427375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.438174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '143d05869b5b474d821fd81dc4c7b960', 'request_id': 'req-42e1030a-9c53-4721-a91b-c9218b797b6c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.439412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '143d05869b5b474d821fd81dc4c7b960', 'request_id': 'req-42e1030a-9c53-4721-a91b-c9218b797b6c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.444461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '143d05869b5b474d821fd81dc4c7b960', 'request_id': 'req-42e1030a-9c53-4721-a91b-c9218b797b6c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.452216 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1954/3876] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:50 2022] DELETE /identity/v3/domains/143d05869b5b474d821fd81dc4c7b960 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:50.456369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:50.458321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:50.474259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9d9c9-8b8b-48c5-b354-f078579f1e24 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:50.544373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:50.545445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:50.545977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:50.546465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/22fab53da5544b67919a9ca24dbc3279` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:50.565184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=22fab53da5544b67919a9ca24dbc3279)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:50.568555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:50.575250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:50.597421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '22fab53da5544b67919a9ca24dbc3279'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.598688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '22fab53da5544b67919a9ca24dbc3279'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.642836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '22fab53da5544b67919a9ca24dbc3279'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.643572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '22fab53da5544b67919a9ca24dbc3279'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.647359 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1923/3877] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:50 2022] PATCH /identity/v3/domains/22fab53da5544b67919a9ca24dbc3279 => generated 278 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:50.651286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:50.664030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1030a-9c53-4721-a91b-c9218b797b6c tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:50.797621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:50.798491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:50.798919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:50.799350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/22fab53da5544b67919a9ca24dbc3279` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:50.813290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=22fab53da5544b67919a9ca24dbc3279)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:50.815334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:50.846370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '22fab53da5544b67919a9ca24dbc3279'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.861006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '22fab53da5544b67919a9ca24dbc3279'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.923786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22fab53da5544b67919a9ca24dbc3279', 'request_id': 'req-4d173651-5661-4151-b3cd-85ab68fac5e6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.934818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22fab53da5544b67919a9ca24dbc3279', 'request_id': 'req-4d173651-5661-4151-b3cd-85ab68fac5e6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.943429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:50.946348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22fab53da5544b67919a9ca24dbc3279', 'request_id': 'req-4d173651-5661-4151-b3cd-85ab68fac5e6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.967519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] The token cache is being invalidate because project 22fab53da5544b67919a9ca24dbc3279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:50.967806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.968610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.979302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '22fab53da5544b67919a9ca24dbc3279', 'request_id': 'req-4d173651-5661-4151-b3cd-85ab68fac5e6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.980082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '22fab53da5544b67919a9ca24dbc3279', 'request_id': 'req-4d173651-5661-4151-b3cd-85ab68fac5e6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.985841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '22fab53da5544b67919a9ca24dbc3279', 'request_id': 'req-4d173651-5661-4151-b3cd-85ab68fac5e6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:50.997863 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1955/3878] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:50 2022] DELETE /identity/v3/domains/22fab53da5544b67919a9ca24dbc3279 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:50.999397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:51.000939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:51.029206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee860bda-96f9-4b70-8405-42e442f2d6b1 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:51.113997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:51.115307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:51.115615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:51.115894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/88c39c4d1efa4970974cf5072d42fea7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:51.131759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=88c39c4d1efa4970974cf5072d42fea7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:51.133701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:51.163379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.164475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.197366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.198238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.203634 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1924/3879] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:10:50 2022] PATCH /identity/v3/domains/88c39c4d1efa4970974cf5072d42fea7 => generated 277 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:51.205039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:51.206476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:51.220043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d173651-5661-4151-b3cd-85ab68fac5e6 tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:51.287297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d1409c8718946d880d6797a587270ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a58256ff69ff4134b1d959a59dc1a1bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:51.287949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:51.288299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:51.288576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] PATH_INFO: `/v3/domains/88c39c4d1efa4970974cf5072d42fea7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:51.303264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=88c39c4d1efa4970974cf5072d42fea7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:51.305115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:51.337725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.356202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.469021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7', 'request_id': 'req-09e6a7f1-0a0f-4617-b17b-1548c142af9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.485441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7', 'request_id': 'req-09e6a7f1-0a0f-4617-b17b-1548c142af9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.492901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7', 'request_id': 'req-09e6a7f1-0a0f-4617-b17b-1548c142af9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.508852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] The token cache is being invalidate because project 88c39c4d1efa4970974cf5072d42fea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:51.509530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.510394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.522837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7', 'request_id': 'req-09e6a7f1-0a0f-4617-b17b-1548c142af9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.523711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7', 'request_id': 'req-09e6a7f1-0a0f-4617-b17b-1548c142af9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.529196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09e6a7f1-0a0f-4617-b17b-1548c142af9a tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '88c39c4d1efa4970974cf5072d42fea7', 'request_id': 'req-09e6a7f1-0a0f-4617-b17b-1548c142af9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.537346 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1956/3880] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:51 2022] DELETE /identity/v3/domains/88c39c4d1efa4970974cf5072d42fea7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:51.542291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:51.543759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:51.640172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671da60-ef45-488a-a8c9-9b99d262962b tempest-DomainsTestJSON-216647923 tempest-DomainsTestJSON-216647923-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:51.672902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:51.726081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:51.729492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:51.729919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:51.730262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] PATH_INFO: `/v3/users/1b45e1a534e741c3900c12ad44ba8aca` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:51.731604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b45e1a534e741c3900c12ad44ba8aca)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:51.733655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:51.825990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b45e1a534e741c3900c12ad44ba8aca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.828041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b45e1a534e741c3900c12ad44ba8aca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.848747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b45e1a534e741c3900c12ad44ba8aca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.871721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b45e1a534e741c3900c12ad44ba8aca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.887739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b45e1a534e741c3900c12ad44ba8aca'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:51.897574 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1925/3881] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:51 2022] DELETE /identity/v3/users/1b45e1a534e741c3900c12ad44ba8aca => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:51.899851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:51.900318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:51.900405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:51.921484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=16,keystone:SELECT=83,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:51.931703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:51.937133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:52.045660 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1957/3882] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:10:52.068513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:52.070144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:52.222796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4d49-4828-4d30-a628-835ec442ce8e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:52.229081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:52.229917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:52.230492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:52.230900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:52.254892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:52.258159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:52.271348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1926/3883] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:53.156357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:53.158265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:53.173751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18902925-3e16-4e15-86c2-88ec74218a2a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:53.179780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:53.180425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:53.180726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:53.181009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] PATH_INFO: `/v3/projects/4facec1eda6e4d8697c148d89db320a8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:53.182147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] RBAC: Authorizing `identity:delete_project(project_id=4facec1eda6e4d8697c148d89db320a8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:53.184250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:53.278834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4facec1eda6e4d8697c148d89db320a8', 'request_id': 'req-896c17c0-157d-4570-aeee-f1b499c369da'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.297289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4facec1eda6e4d8697c148d89db320a8', 'request_id': 'req-896c17c0-157d-4570-aeee-f1b499c369da'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.304682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4facec1eda6e4d8697c148d89db320a8', 'request_id': 'req-896c17c0-157d-4570-aeee-f1b499c369da'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.329403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] The token cache is being invalidate because project 4facec1eda6e4d8697c148d89db320a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:53.329630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.330354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.333662 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1958/3884] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:53 2022] DELETE /identity/v3/projects/4facec1eda6e4d8697c148d89db320a8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:53.338156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:53.352715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1d313-6a6c-4ab9-aee2-32c48dcf0639 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:53.402427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:53.521270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:53.521936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:53.522220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:53.522527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] PATH_INFO: `/v3/users/1d1409c8718946d880d6797a587270ff` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:53.527528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1409c8718946d880d6797a587270ff)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:53.530495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:53.677854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1409c8718946d880d6797a587270ff'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.678712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1409c8718946d880d6797a587270ff'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.691508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1409c8718946d880d6797a587270ff'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.709273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1409c8718946d880d6797a587270ff'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.733405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50e3fd52-9914-4bbb-aea8-6af580fb18e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1409c8718946d880d6797a587270ff'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:53.759046 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1927/3885] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:53 2022] DELETE /identity/v3/users/1d1409c8718946d880d6797a587270ff => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:54.541621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:54.543403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:54.609716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c17c0-157d-4570-aeee-f1b499c369da admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:54.719726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:54.719726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:54.720775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:54.720969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] PATH_INFO: `/v3/projects/a58256ff69ff4134b1d959a59dc1a1bf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:54.722687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58256ff69ff4134b1d959a59dc1a1bf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:54.726263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:54.788189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:54.813546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58256ff69ff4134b1d959a59dc1a1bf', 'request_id': 'req-1a7ca893-385e-4644-a8ba-148e5c731355'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:54.843385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58256ff69ff4134b1d959a59dc1a1bf', 'request_id': 'req-1a7ca893-385e-4644-a8ba-148e5c731355'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:54.856959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58256ff69ff4134b1d959a59dc1a1bf', 'request_id': 'req-1a7ca893-385e-4644-a8ba-148e5c731355'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:54.882936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] The token cache is being invalidate because project a58256ff69ff4134b1d959a59dc1a1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:54.884195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:54.885347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1a7ca893-385e-4644-a8ba-148e5c731355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:54.891093 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1959/3886] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:54 2022] DELETE /identity/v3/projects/a58256ff69ff4134b1d959a59dc1a1bf => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:54.892515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4acf5-c496-480f-87d0-b4dc64ad8505 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:54.893172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4acf5-c496-480f-87d0-b4dc64ad8505 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:54.893172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4acf5-c496-480f-87d0-b4dc64ad8505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:54.936890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-92b4acf5-c496-480f-87d0-b4dc64ad8505 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:54.938282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-92b4acf5-c496-480f-87d0-b4dc64ad8505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:55.072396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5fb4f-cb25-4161-8f87-6389a57a96ef None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.073065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5fb4f-cb25-4161-8f87-6389a57a96ef None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.073586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5fb4f-cb25-4161-8f87-6389a57a96ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.082751 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1928/3887] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:55.104873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d0d5fb4f-cb25-4161-8f87-6389a57a96ef None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:55.107531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d0d5fb4f-cb25-4161-8f87-6389a57a96ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:55.124104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4acf5-c496-480f-87d0-b4dc64ad8505 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.148114 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1960/3888] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:55.151956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4acf5-c496-480f-87d0-b4dc64ad8505 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:55.169807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:55.170842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.171296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.172303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.173339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:10:55.173855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:55.176795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:55.194838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1929/3889] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:55.199727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5fb4f-cb25-4161-8f87-6389a57a96ef None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.227102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5fb4f-cb25-4161-8f87-6389a57a96ef None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:55.237479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:55.238836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.239276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.240218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.242421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:55.247332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:55.305915 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1961/3890] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:55 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:55.306361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.317392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576ebcf-10f6-49ab-8ba4-7caa18f35af9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:55.364975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:55.425566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:55.426724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.427105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.427444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.428154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:55.430800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:55.466805 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1930/3891] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:55 2022] POST /identity/v3/users => generated 282 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:55.468420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.480411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f5c88-8e52-4b4e-a8b6-32f226513cab admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:55.485959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:55.487244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.487616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.488039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.488783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:55.490671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:55.502560 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1962/3892] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:55.507177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.519533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93630d-5504-43fd-8278-a8b8e5e9f38e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:55.524510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef260d-9627-4f4a-84a5-67788875e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:55.525327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef260d-9627-4f4a-84a5-67788875e783 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.525790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef260d-9627-4f4a-84a5-67788875e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.526216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef260d-9627-4f4a-84a5-67788875e783 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.527021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef260d-9627-4f4a-84a5-67788875e783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:55.528959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef260d-9627-4f4a-84a5-67788875e783 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:55.541201 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1931/3893] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:55.545728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.556283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9179d6-2d0b-4584-a420-447b97b224aa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:55.561362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:55.562085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.562370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.562638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] PATH_INFO: `/v3/projects/19cc9e93c3e44c8db209bab41f965d5f/users/47ccd88ce545457fb0c5f044a06d7ab9/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.581019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19cc9e93c3e44c8db209bab41f965d5f, user_id=47ccd88ce545457fb0c5f044a06d7ab9, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:55.583868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:55.601048 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1963/3894] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:55 2022] PUT /identity/v3/projects/19cc9e93c3e44c8db209bab41f965d5f/users/47ccd88ce545457fb0c5f044a06d7ab9/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:55.615296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f80d7-0e88-4874-9e76-731e641a3bf1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.615673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f80d7-0e88-4874-9e76-731e641a3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.623717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f80d7-0e88-4874-9e76-731e641a3bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.656264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a44f80d7-0e88-4874-9e76-731e641a3bf1 None None] MFA Rules not processed for user `47ccd88ce545457fb0c5f044a06d7ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:55.657517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a44f80d7-0e88-4874-9e76-731e641a3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:55.752314 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1932/3895] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:55.753600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.864401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fb311-104a-43ba-bfb9-8b12daf40b7a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:55.869929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:55.870558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:55.870834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:55.871105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:55.871902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:55.874418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:55.919212 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1964/3896] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:55 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:55.923126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f80d7-0e88-4874-9e76-731e641a3bf1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:55.940150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f80d7-0e88-4874-9e76-731e641a3bf1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.029527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.030400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.031085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.031547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.032191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.034254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.086176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1933/3897] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:55 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:56.089521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.106574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220a1ba-702b-43da-949a-3cbf17da92ad admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.112130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.112959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.113246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.113559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.114225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.115878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.137064 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1965/3898] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:56.141365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.160473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6ad0-fe81-4ac5-a753-4f58a1872746 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.166919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.167595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.167866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.168284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] PATH_INFO: `/v3/projects/3b204c772fbf4691ad2ecb8623833160/users/a5b171b12fed40ce84121a79c6ceb327/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.190631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b204c772fbf4691ad2ecb8623833160, user_id=a5b171b12fed40ce84121a79c6ceb327, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.194442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.208373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:56.221593 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1934/3899] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:56 2022] PUT /identity/v3/projects/3b204c772fbf4691ad2ecb8623833160/users/a5b171b12fed40ce84121a79c6ceb327/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:56.222130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.236465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f73f1-813e-435d-922d-198753c24b2d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.276951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:56.325511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfd503-8eac-4779-96ce-64d02b3396f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.326145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfd503-8eac-4779-96ce-64d02b3396f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.326428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfd503-8eac-4779-96ce-64d02b3396f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.326699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfd503-8eac-4779-96ce-64d02b3396f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.327519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfd503-8eac-4779-96ce-64d02b3396f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.329176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfd503-8eac-4779-96ce-64d02b3396f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.342008 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1966/3900] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:56.346088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.362620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9185b-27a9-4f87-9ef9-98cc7d551c17 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.373891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.374591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.374872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.375314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] PATH_INFO: `/v3/projects/3b204c772fbf4691ad2ecb8623833160/users/a5b171b12fed40ce84121a79c6ceb327/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.378823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b204c772fbf4691ad2ecb8623833160, user_id=a5b171b12fed40ce84121a79c6ceb327, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.383082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.404858 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1935/3901] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:56 2022] PUT /identity/v3/projects/3b204c772fbf4691ad2ecb8623833160/users/a5b171b12fed40ce84121a79c6ceb327/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:56.413480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc743bd-0a38-4b15-a1b4-99cf9c46eac3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.414019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc743bd-0a38-4b15-a1b4-99cf9c46eac3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.414472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc743bd-0a38-4b15-a1b4-99cf9c46eac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.455171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6fc743bd-0a38-4b15-a1b4-99cf9c46eac3 None None] MFA Rules not processed for user `a5b171b12fed40ce84121a79c6ceb327`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:56.457041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fc743bd-0a38-4b15-a1b4-99cf9c46eac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:56.592648 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1967/3902] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:56.597222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.631176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22670c-0f72-4ab4-affb-518ffb4b14aa admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.645602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.646115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.646398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.647558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.647558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.650724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.707650 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1936/3903] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:56 2022] POST /identity/v3/projects => generated 359 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:56.709189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc743bd-0a38-4b15-a1b4-99cf9c46eac3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.724683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc743bd-0a38-4b15-a1b4-99cf9c46eac3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.759249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.780259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:56.821952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.822674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.822953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.823301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.824001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.825965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.864706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1968/3904] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:56 2022] POST /identity/v3/users => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:56.868777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.899291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff76085-5c66-40ba-8385-311982117e06 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.908118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.909004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67178-db75-4339-8279-06bb0c7fc6ba tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.909722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.910563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.911574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/b5fb88ead40a43efb2ac55d868aa816d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.918643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.919338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.919672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.920465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] PATH_INFO: `/v3/users/f06d16c90a5d464696f8137612c35327` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.921786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06d16c90a5d464696f8137612c35327)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.925226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.935754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:get_user(user_id=b5fb88ead40a43efb2ac55d868aa816d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.938728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.941005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1969/3905] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:10:56 2022] GET /identity/v3/users/b5fb88ead40a43efb2ac55d868aa816d => generated 371 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:10:56.945497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:56.958800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b70b7-aacf-40f6-a5b7-70e8714fb84d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:56.964996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:56.965778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:56.966115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:56.966452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/b5fb88ead40a43efb2ac55d868aa816d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:56.967717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b5fb88ead40a43efb2ac55d868aa816d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:56.970026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:56.989347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06d16c90a5d464696f8137612c35327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:56.990187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06d16c90a5d464696f8137612c35327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.000052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06d16c90a5d464696f8137612c35327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.010287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:57.013347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06d16c90a5d464696f8137612c35327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.030873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06d16c90a5d464696f8137612c35327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.038612 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1937/3906] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:56 2022] DELETE /identity/v3/users/f06d16c90a5d464696f8137612c35327 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:57.057001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5fb88ead40a43efb2ac55d868aa816d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.057919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5fb88ead40a43efb2ac55d868aa816d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.068949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5fb88ead40a43efb2ac55d868aa816d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.078299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5fb88ead40a43efb2ac55d868aa816d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.079223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5fb88ead40a43efb2ac55d868aa816d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.105345 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1970/3907] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:56 2022] DELETE /identity/v3/users/b5fb88ead40a43efb2ac55d868aa816d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:57.111686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.113730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:57.171735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:57.199919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abed33-fa2d-428c-a2be-255b1e90ce25 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.272638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.274194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.274729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.275085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/ba3574047f9f4a70af4e7d4e870f0275` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.276383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ba3574047f9f4a70af4e7d4e870f0275)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.278413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.329967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3574047f9f4a70af4e7d4e870f0275', 'request_id': 'req-f34aee25-0d5a-4f81-b621-b08cf749aebe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.336890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3574047f9f4a70af4e7d4e870f0275', 'request_id': 'req-f34aee25-0d5a-4f81-b621-b08cf749aebe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.347591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3574047f9f4a70af4e7d4e870f0275', 'request_id': 'req-f34aee25-0d5a-4f81-b621-b08cf749aebe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.358846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] The token cache is being invalidate because project ba3574047f9f4a70af4e7d4e870f0275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:57.359151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.359802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:57.361388 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1938/3908] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:57 2022] DELETE /identity/v3/projects/ba3574047f9f4a70af4e7d4e870f0275 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:57.366752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.368559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:57.382702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bebd8b-c293-4abd-8aea-6a9c7a802801 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.448193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.449460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.449739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.450009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.451084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.453083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.493261 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1971/3909] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:57 2022] POST /identity/v3/projects => generated 360 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:57.497452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.499004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:57.515166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aee25-0d5a-4f81-b621-b08cf749aebe tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.595844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.596476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.596796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.597075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.597750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.600961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.633228 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1939/3910] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:57 2022] POST /identity/v3/users => generated 414 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:57.634723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.649007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259702ff-7caf-413f-9b23-5d93c6a3436f tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.655601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.656228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.656535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.657496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.657719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.658857 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:57.658857 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:10:57.658857 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:10:57.659811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.681671 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1972/3911] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:10:57 2022] POST /identity/v3/roles => generated 239 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:57.685823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.701123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8150b8-4734-4ca5-939a-4adc402a9ed8 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.726477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.727165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.727455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.727735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/a7ec2b99cfa84766ba89cad4bbd81621` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.731640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:57.746837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:get_user(user_id=a7ec2b99cfa84766ba89cad4bbd81621)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.750055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.754723 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1940/3912] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:10:57 2022] GET /identity/v3/users/a7ec2b99cfa84766ba89cad4bbd81621 => generated 414 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:57.757010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.770778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f6cdf-0dbf-4ba1-983f-0ea3aead7675 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.775168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.776529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.777051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.777398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/roles/854dd0fd6a8b4651ba4d4bb19f64e534` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.779588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:get_role(role_id=854dd0fd6a8b4651ba4d4bb19f64e534)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.780692 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:57.780692 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:10:57.780692 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:10:57.781978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.785354 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1973/3913] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:10:57 2022] GET /identity/v3/roles/854dd0fd6a8b4651ba4d4bb19f64e534 => generated 239 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:57.787970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.799290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1801e24-aaed-4145-ba44-863bead2d2f3 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.804989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.805738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.806125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.806507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.807280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.810369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.835406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.838418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:57.857770 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1941/3914] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:57 2022] POST /identity/v3/projects => generated 359 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:57.859425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:57.917870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7fdbf-5d18-45e8-891b-35a1894cd990 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.929267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:57.961437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b691b9-5075-4ab8-9b6b-c674756b78e9 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:57.968262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:57.968961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:57.969444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:57.970222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/83387d5e3adf41559c2c00fe7f1ab002` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:57.986237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:get_project(project_id=83387d5e3adf41559c2c00fe7f1ab002)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:57.990020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:57.996615 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1942/3915] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:57 2022] GET /identity/v3/projects/83387d5e3adf41559c2c00fe7f1ab002 => generated 359 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:57.997767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.025194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.025875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.026372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.026651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] PATH_INFO: `/v3/projects/327b414889a04e189a7f940685a4ff39` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.027845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327b414889a04e189a7f940685a4ff39)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.030892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.034328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc17f7e-ed09-4055-a066-a6cbd5c7d989 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.045067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.045711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.045981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.046257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/83387d5e3adf41559c2c00fe7f1ab002/users/a7ec2b99cfa84766ba89cad4bbd81621/roles/854dd0fd6a8b4651ba4d4bb19f64e534` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.048642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_grant(project_id=83387d5e3adf41559c2c00fe7f1ab002, user_id=a7ec2b99cfa84766ba89cad4bbd81621, role_id=854dd0fd6a8b4651ba4d4bb19f64e534)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.051398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.083405 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1943/3916] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:57 2022] PUT /identity/v3/projects/83387d5e3adf41559c2c00fe7f1ab002/users/a7ec2b99cfa84766ba89cad4bbd81621/roles/854dd0fd6a8b4651ba4d4bb19f64e534 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:58.087888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.119263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327b414889a04e189a7f940685a4ff39', 'request_id': 'req-c650350d-2b5b-45d1-860d-d95305dc3ab2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.145388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327b414889a04e189a7f940685a4ff39', 'request_id': 'req-c650350d-2b5b-45d1-860d-d95305dc3ab2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.160966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327b414889a04e189a7f940685a4ff39', 'request_id': 'req-c650350d-2b5b-45d1-860d-d95305dc3ab2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.199309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] The token cache is being invalidate because project 327b414889a04e189a7f940685a4ff39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:58.199770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.200714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c650350d-2b5b-45d1-860d-d95305dc3ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.207038 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1974/3917] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:57 2022] DELETE /identity/v3/projects/327b414889a04e189a7f940685a4ff39 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:58.209645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822193db-dcb4-4adf-af0f-05faa69718ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.210711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822193db-dcb4-4adf-af0f-05faa69718ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.211202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822193db-dcb4-4adf-af0f-05faa69718ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.221555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a024-141f-4d6d-ad53-31a1f7eca656 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.246285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-822193db-dcb4-4adf-af0f-05faa69718ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:58.247662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-822193db-dcb4-4adf-af0f-05faa69718ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:58.302456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.303093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.303377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.303682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.304593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.307021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.330605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1975/3918] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:58.339044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608857-784f-49e1-af88-7523168dca09 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.339559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608857-784f-49e1-af88-7523168dca09 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.339946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608857-784f-49e1-af88-7523168dca09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.346054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:58.372717 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1944/3919] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:58 2022] POST /identity/v3/projects => generated 359 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:58.375657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c3608857-784f-49e1-af88-7523168dca09 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:58.375858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.377725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3608857-784f-49e1-af88-7523168dca09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:58.377861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:58.426853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:58.457105 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1976/3920] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:58.461574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608857-784f-49e1-af88-7523168dca09 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.464687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5f01c-4e32-4666-867b-626d67608d1a tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.470957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.472205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.472205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.472439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/7fdc9ef1161b410f9c5863e9943d7402` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.472532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608857-784f-49e1-af88-7523168dca09 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.473513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:get_project(project_id=7fdc9ef1161b410f9c5863e9943d7402)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.475820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.478196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.478358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.478669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.479137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.479797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:10:58.480189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.482245 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1945/3921] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:10:58 2022] GET /identity/v3/projects/7fdc9ef1161b410f9c5863e9943d7402 => generated 359 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:10:58.482743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.483082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.496065 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1977/3922] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:10:58.496220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53acc98-0432-4615-9ae4-ce6193b6564b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.500592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.500744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.501362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.501642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.501914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/7fdc9ef1161b410f9c5863e9943d7402/users/a7ec2b99cfa84766ba89cad4bbd81621/roles/854dd0fd6a8b4651ba4d4bb19f64e534` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.504211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7fdc9ef1161b410f9c5863e9943d7402, user_id=a7ec2b99cfa84766ba89cad4bbd81621, role_id=854dd0fd6a8b4651ba4d4bb19f64e534)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.508006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.511577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adfe10-70ee-4f01-851c-c89151ca857c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.516748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.517569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.518481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.518768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.519405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.521374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.525119 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1946/3923] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:58 2022] PUT /identity/v3/projects/7fdc9ef1161b410f9c5863e9943d7402/users/a7ec2b99cfa84766ba89cad4bbd81621/roles/854dd0fd6a8b4651ba4d4bb19f64e534 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:58.529414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.557503 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1978/3924] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:10:58 2022] POST /identity/v3/projects => generated 397 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:10:58.561784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.630744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9549b73-f0c1-4466-b0ea-3bc8f278076c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.696165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62622aca-e706-4694-8d24-cc06c34e7ce2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.702335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.703098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.705162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.705570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.706370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.709187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.709840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.710157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.710436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/a7ec2b99cfa84766ba89cad4bbd81621/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.711982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=a7ec2b99cfa84766ba89cad4bbd81621)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.713559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.714202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.748929 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1979/3925] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:10:58 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:10:58.753153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.768999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f904f-bd19-48b1-8370-67363479cbf3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.772005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:58.776471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.777506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.777814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.777917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1947/3926] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:10:58 2022] GET /identity/v3/users/a7ec2b99cfa84766ba89cad4bbd81621/projects => generated 842 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:10:58.778027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.781609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.782790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.784932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.795328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22112cdb-7208-4942-a152-a435346aae15 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.796962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1980/3927] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:58 2022] GET /identity/v3/roles => generated 1650 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:10:58.801459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.802012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.803127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.803594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.804181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/7fdc9ef1161b410f9c5863e9943d7402` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.805696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7fdc9ef1161b410f9c5863e9943d7402)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.808435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.813732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b8dd8-8512-475c-9cb0-05440f0d87c4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.819348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.820006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.820287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.821204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.821551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.822923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.835467 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1981/3928] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:10:58 2022] GET /identity/v3/roles => generated 1650 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:10:58.839505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.853529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dcff8-ce58-41d8-a5b1-95bff017f618 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:58.858665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e347ff-ce6c-4008-8505-01ce9c76972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:58.859442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e347ff-ce6c-4008-8505-01ce9c76972d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.859691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e347ff-ce6c-4008-8505-01ce9c76972d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.859970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e347ff-ce6c-4008-8505-01ce9c76972d admin admin] PATH_INFO: `/v3/projects/be55c1fb4e6f4870b25e986907d41101/users/96e48f22b8e7478986f912e64df670b0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.879643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e347ff-ce6c-4008-8505-01ce9c76972d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be55c1fb4e6f4870b25e986907d41101, user_id=96e48f22b8e7478986f912e64df670b0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:58.883333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e347ff-ce6c-4008-8505-01ce9c76972d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:58.890307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fdc9ef1161b410f9c5863e9943d7402', 'request_id': 'req-6559f579-65ec-41fd-84db-6c990521a9ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.901006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fdc9ef1161b410f9c5863e9943d7402', 'request_id': 'req-6559f579-65ec-41fd-84db-6c990521a9ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.907465 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1982/3929] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:10:58 2022] PUT /identity/v3/projects/be55c1fb4e6f4870b25e986907d41101/users/96e48f22b8e7478986f912e64df670b0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:58.910158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:58.910494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:58.910874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:58.915829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fdc9ef1161b410f9c5863e9943d7402', 'request_id': 'req-6559f579-65ec-41fd-84db-6c990521a9ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.929553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] The token cache is being invalidate because project 7fdc9ef1161b410f9c5863e9943d7402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:58.929833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.930741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:58.935525 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1948/3930] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:58 2022] DELETE /identity/v3/projects/7fdc9ef1161b410f9c5863e9943d7402 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:58.935713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:10:58.936816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:58.937306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:58.939886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:59.010838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:59.026084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559f579-65ec-41fd-84db-6c990521a9ec tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:59.032562 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1983/3931] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:10:59.048359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:59.051200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:59.119632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:59.120421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:59.120649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:59.120986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/83387d5e3adf41559c2c00fe7f1ab002` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:59.122861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_project(project_id=83387d5e3adf41559c2c00fe7f1ab002)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:59.124834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:59.182718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fbd3c-8582-4548-be4c-423b04f532b6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:59.184625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:59.210465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83387d5e3adf41559c2c00fe7f1ab002', 'request_id': 'req-26ea9940-69f5-457c-abf3-0f48f0b36278'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.219950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83387d5e3adf41559c2c00fe7f1ab002', 'request_id': 'req-26ea9940-69f5-457c-abf3-0f48f0b36278'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.232121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83387d5e3adf41559c2c00fe7f1ab002', 'request_id': 'req-26ea9940-69f5-457c-abf3-0f48f0b36278'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.250102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] The token cache is being invalidate because project 83387d5e3adf41559c2c00fe7f1ab002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:59.250553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.251813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.253994 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1949/3932] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:58 2022] DELETE /identity/v3/projects/83387d5e3adf41559c2c00fe7f1ab002 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:10:59.258140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:59.259688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:59.332192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:59.333312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:59.334002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:59.334165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:59.336788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:59.417490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9940-69f5-457c-abf3-0f48f0b36278 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:59.428954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:59.430005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:59.430521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:59.430890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/roles/854dd0fd6a8b4651ba4d4bb19f64e534` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:59.432757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_role(role_id=854dd0fd6a8b4651ba4d4bb19f64e534)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:59.433825 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:10:59.433825 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:10:59.433825 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:10:59.434733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:59.442011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:59.444194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:59.459872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1984/3933] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:10:59.526646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invalidating the token cache because role 854dd0fd6a8b4651ba4d4bb19f64e534 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:10:59.527758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.529097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.533284 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1950/3934] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:59 2022] DELETE /identity/v3/roles/854dd0fd6a8b4651ba4d4bb19f64e534 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:59.537701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:59.551781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793a1df-f960-4f8a-a173-df538dd15a3d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:59.634237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:59.635244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:59.635598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:59.635943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/a7ec2b99cfa84766ba89cad4bbd81621` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:59.638411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a7ec2b99cfa84766ba89cad4bbd81621)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:59.640779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:10:59.643001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:59.705904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ec2b99cfa84766ba89cad4bbd81621'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.707058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ec2b99cfa84766ba89cad4bbd81621'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.717400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ec2b99cfa84766ba89cad4bbd81621'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.731228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ec2b99cfa84766ba89cad4bbd81621'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.732275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ec2b99cfa84766ba89cad4bbd81621'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:10:59.771636 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1985/3935] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:10:59 2022] DELETE /identity/v3/users/a7ec2b99cfa84766ba89cad4bbd81621 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:10:59.778121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:10:59.780743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:10:59.800464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902dde8-da0b-4409-8fd9-6d7dd9ecca99 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:10:59.921331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:10:59.922045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:10:59.922339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:10:59.922625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/3d38490d7c4947c2af845fbee28a3462` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:10:59.926042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3d38490d7c4947c2af845fbee28a3462)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:10:59.927906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:00.000393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:00.046615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d38490d7c4947c2af845fbee28a3462', 'request_id': 'req-c6eb9be5-4a8a-4c05-b636-170b5973ad17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.056365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d38490d7c4947c2af845fbee28a3462', 'request_id': 'req-c6eb9be5-4a8a-4c05-b636-170b5973ad17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.067806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d38490d7c4947c2af845fbee28a3462', 'request_id': 'req-c6eb9be5-4a8a-4c05-b636-170b5973ad17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.082559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] The token cache is being invalidate because project 3d38490d7c4947c2af845fbee28a3462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:00.082927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.083532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6eb9be5-4a8a-4c05-b636-170b5973ad17 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.090627 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1951/3936] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:10:59 2022] DELETE /identity/v3/projects/3d38490d7c4947c2af845fbee28a3462 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:00.092232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:00.093817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.107449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423c5db-ca01-4453-bb0a-b753cdf028d0 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:00.185710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:00.186372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.186655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.186932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/47ccd88ce545457fb0c5f044a06d7ab9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.188921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:update_user(user_id=47ccd88ce545457fb0c5f044a06d7ab9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:00.189448 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:00.189448 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:11:00.189448 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:11:00.190717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:00.250557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.267548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invalidating the token cache because user 47ccd88ce545457fb0c5f044a06d7ab9 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:00.267986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.268742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.274434 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1986/3937] 10.222.0.26 () {60 vars in 1243 bytes} [Tue Sep 27 10:11:00 2022] PATCH /identity/v3/users/47ccd88ce545457fb0c5f044a06d7ab9 => generated 295 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:00.277945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc505db-f903-43b2-bcb0-3f49c190038d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.278935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc505db-f903-43b2-bcb0-3f49c190038d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.279442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc505db-f903-43b2-bcb0-3f49c190038d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.319212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cbc505db-f903-43b2-bcb0-3f49c190038d None None] MFA Rules not processed for user `47ccd88ce545457fb0c5f044a06d7ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:00.321091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cbc505db-f903-43b2-bcb0-3f49c190038d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.329087 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1952/3938] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:00 2022] POST /identity/v3/auth/tokens => generated 347 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:00.336492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:00.338212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.359733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab4c32-fc1c-474c-9d21-986633aea8ed tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:00.364406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:00.365091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.365405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.365652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/47ccd88ce545457fb0c5f044a06d7ab9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.367131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:update_user(user_id=47ccd88ce545457fb0c5f044a06d7ab9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:00.369122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:00.405331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.415434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invalidating the token cache because user 47ccd88ce545457fb0c5f044a06d7ab9 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:00.415847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.416468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.418889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1987/3939] 10.222.0.26 () {60 vars in 1243 bytes} [Tue Sep 27 10:11:00 2022] PATCH /identity/v3/users/47ccd88ce545457fb0c5f044a06d7ab9 => generated 295 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:00.422935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc505db-f903-43b2-bcb0-3f49c190038d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:00.439659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc505db-f903-43b2-bcb0-3f49c190038d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:00.449323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:00.449323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.449323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.449840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/47ccd88ce545457fb0c5f044a06d7ab9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.466902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:update_user(user_id=47ccd88ce545457fb0c5f044a06d7ab9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:00.467948 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:00.467948 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:11:00.467948 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:11:00.468237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:00.469355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:00.470319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.507912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=12,keystone:INSERT=5,keystone:UPDATE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:00.517201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.534043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invalidating the token cache because user 47ccd88ce545457fb0c5f044a06d7ab9 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:00.534760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.535589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a02dc289-926d-4b73-9123-7b6e30bab88b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.538247 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1953/3940] 10.222.0.26 () {60 vars in 1243 bytes} [Tue Sep 27 10:11:00 2022] PATCH /identity/v3/users/47ccd88ce545457fb0c5f044a06d7ab9 => generated 295 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:00.543057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.543350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.543626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.558298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47d7e-391f-441d-be07-946f04c98b91 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:00.562485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:00.563253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.563538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.563814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.567278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.588694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] MFA Rules not processed for user `47ccd88ce545457fb0c5f044a06d7ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:00.589919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.596567 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1954/3941] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:00 2022] POST /identity/v3/auth/tokens => generated 347 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 10:11:00.597708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:00.599520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.616577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f022f-42b3-444e-9298-f0064fd9cb9d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:00.621797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:00.622189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.622469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.622746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/47ccd88ce545457fb0c5f044a06d7ab9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.623956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:update_user(user_id=47ccd88ce545457fb0c5f044a06d7ab9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:00.625906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:00.654417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:00.656561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:00.669989 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1988/3942] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:00.682611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.693898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invalidating the token cache because user 47ccd88ce545457fb0c5f044a06d7ab9 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:00.695027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.696116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.698794 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1955/3943] 10.222.0.26 () {60 vars in 1243 bytes} [Tue Sep 27 10:11:00 2022] PATCH /identity/v3/users/47ccd88ce545457fb0c5f044a06d7ab9 => generated 295 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:00.704194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:00.716959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb4ee-80b0-4497-8400-052355266e19 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:00.722427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4e9c4-b7ce-4b00-be55-4189d6ecc55e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:00.722996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4e9c4-b7ce-4b00-be55-4189d6ecc55e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.723518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4e9c4-b7ce-4b00-be55-4189d6ecc55e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.723651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4e9c4-b7ce-4b00-be55-4189d6ecc55e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.724237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4e9c4-b7ce-4b00-be55-4189d6ecc55e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:00.726960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4e9c4-b7ce-4b00-be55-4189d6ecc55e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:00.762469 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1989/3944] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:11:00 2022] POST /identity/v3/users => generated 264 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:00.766522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:00.768819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:00.785216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3063-f89e-407e-b931-65d85cfa267c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:00.791788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d711076-5777-4986-a027-8669f5283b65 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:00.792741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711076-5777-4986-a027-8669f5283b65 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:00.792969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711076-5777-4986-a027-8669f5283b65 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:00.793535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711076-5777-4986-a027-8669f5283b65 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/01553c456206448f9521641365ef0420/password` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:00.875955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d711076-5777-4986-a027-8669f5283b65 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '01553c456206448f9521641365ef0420'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:00.891972 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1956/3945] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:00 2022] POST /identity/v3/users/01553c456206448f9521641365ef0420/password => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:01.898068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023d86-5dc8-4344-8e34-3ed93331c7f4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:01.898551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023d86-5dc8-4344-8e34-3ed93331c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:01.898666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023d86-5dc8-4344-8e34-3ed93331c7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:01.931400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7e023d86-5dc8-4344-8e34-3ed93331c7f4 None None] MFA Rules not processed for user `01553c456206448f9521641365ef0420`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:01.934454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e023d86-5dc8-4344-8e34-3ed93331c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:01.938763 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1990/3946] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:01 2022] POST /identity/v3/auth/tokens => generated 329 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:01.944951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d711076-5777-4986-a027-8669f5283b65 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:01.995362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d711076-5777-4986-a027-8669f5283b65 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:02.008412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:02.033529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:02.033529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:02.033529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:02.052066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:02.083850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:02.097221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=7,keystone:UPDATE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:02.107277 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1957/3947] 10.222.0.26 () {56 vars in 1263 bytes} [Tue Sep 27 10:11:01 2022] GET /identity/v3/auth/tokens => generated 329 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:02.108449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023d86-5dc8-4344-8e34-3ed93331c7f4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:02.128025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023d86-5dc8-4344-8e34-3ed93331c7f4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:02.142257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:02.143450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:02.144229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:02.144596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/01553c456206448f9521641365ef0420` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:02.146394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_user(user_id=01553c456206448f9521641365ef0420)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:02.157072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:02.297790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01553c456206448f9521641365ef0420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:02.299290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01553c456206448f9521641365ef0420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:02.340097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01553c456206448f9521641365ef0420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:02.377453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01553c456206448f9521641365ef0420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:02.378876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01553c456206448f9521641365ef0420'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:02.417601 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1991/3948] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:02 2022] DELETE /identity/v3/users/01553c456206448f9521641365ef0420 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:02.418648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:02.430015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb69e-391a-4c9b-9ad3-a321b140fcb1 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:02.526500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:02.527569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:02.528540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:02.528943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:02.529702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:02.532062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:02.579142 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1958/3949] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:02 2022] POST /identity/v3/users => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:02.582898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:02.584474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:02.601994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688fffc-278f-4520-8a6f-39662a8386fd tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:02.629435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:02.630077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:02.630358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:02.630630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:02.631284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:02.633214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:02.676413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:02.720173 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1992/3950] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:02 2022] POST /identity/v3/projects => generated 360 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:02.723514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:02.746593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d259d13-a343-445b-ae54-1b005631e83b tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:02.938835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:02.939114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:02.939374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:02.939638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/726b49336bb3487d800051258f5faf02` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:02.965201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:update_user(user_id=726b49336bb3487d800051258f5faf02)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:02.971081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:03.055090 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1959/3951] 10.222.0.26 () {60 vars in 1244 bytes} [Tue Sep 27 10:11:02 2022] PATCH /identity/v3/users/726b49336bb3487d800051258f5faf02 => generated 523 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:11:03.059342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:03.078569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f2864-b7f5-48ad-8687-977d6757b69e tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:03.094416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:03.095295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:03.095911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:03.096196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/726b49336bb3487d800051258f5faf02` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:03.149791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:get_user(user_id=726b49336bb3487d800051258f5faf02)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:03.159039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:03.166801 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1993/3952] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:11:03 2022] GET /identity/v3/users/726b49336bb3487d800051258f5faf02 => generated 389 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:03.170794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:03.187692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d292f-f037-4df5-a919-da304f519885 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:03.202426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:03.203091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:03.203421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:03.203708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/projects/5d50cb7b1701453a8016766f425289df` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:03.205778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5d50cb7b1701453a8016766f425289df)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:03.207659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:03.306214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d50cb7b1701453a8016766f425289df', 'request_id': 'req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.327340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d50cb7b1701453a8016766f425289df', 'request_id': 'req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.356561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:UPDATE=2,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:03.379350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d50cb7b1701453a8016766f425289df', 'request_id': 'req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.401189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] The token cache is being invalidate because project 5d50cb7b1701453a8016766f425289df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:03.401729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.402786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.409683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:03.411774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:03.418035 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1960/3953] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:03 2022] DELETE /identity/v3/projects/5d50cb7b1701453a8016766f425289df => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:03.436324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b97c2-7b7c-4ccb-9b0b-29a66b9cfe39 tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:03.542295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b171b12fed40ce84121a79c6ceb327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b204c772fbf4691ad2ecb8623833160', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:03.543437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:03.543893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:03.544397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] PATH_INFO: `/v3/users/726b49336bb3487d800051258f5faf02` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:03.546021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorizing `identity:delete_user(user_id=726b49336bb3487d800051258f5faf02)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:03.548310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:03.646980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726b49336bb3487d800051258f5faf02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.647780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726b49336bb3487d800051258f5faf02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.666108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726b49336bb3487d800051258f5faf02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.674930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726b49336bb3487d800051258f5faf02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.676071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0dbd071-cc85-4e08-9f12-b9f6f626434c tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726b49336bb3487d800051258f5faf02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:03.736754 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1994/3954] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:03 2022] DELETE /identity/v3/users/726b49336bb3487d800051258f5faf02 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:03.742199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:03.744513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:03.888845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce87c7-0d38-483a-b4b5-a8a05ce7ce0d tempest-UsersV3TestJSON-1146548439 tempest-UsersV3TestJSON-1146548439-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:04.019590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:04.021038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:04.021537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:04.022247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] PATH_INFO: `/v3/users/47ccd88ce545457fb0c5f044a06d7ab9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:04.056958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ccd88ce545457fb0c5f044a06d7ab9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:04.060241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:04.215615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:04.216397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:04.226674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:04.237733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:04.298309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ccd88ce545457fb0c5f044a06d7ab9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:04.301559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:04.308103 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1961/3955] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:03 2022] DELETE /identity/v3/users/47ccd88ce545457fb0c5f044a06d7ab9 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:04.312848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:04.313406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:04.313803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:04.364544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:04.375445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:04.446868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:04.448379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:04.455471 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1995/3956] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:04.498513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:04.500504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:04.516833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:04.619228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12458b1-22d5-45f5-b56f-629c9460957c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:04.641235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328d65a-d31e-43f9-a51d-454e20a04720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:04.641953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328d65a-d31e-43f9-a51d-454e20a04720 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:04.642230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328d65a-d31e-43f9-a51d-454e20a04720 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:04.642502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328d65a-d31e-43f9-a51d-454e20a04720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:04.645672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d328d65a-d31e-43f9-a51d-454e20a04720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:04.660827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea302db3-da32-47fe-ba95-dae8adbd6033 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:04.673709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328d65a-d31e-43f9-a51d-454e20a04720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:04.675282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328d65a-d31e-43f9-a51d-454e20a04720 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:04.681992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:04.682870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:04.685223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:04.685513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:04.696066 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1962/3957] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:04.713037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:04.714712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:04.736402 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1996/3958] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:05.175144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9951-0392-4639-a157-f7ea0d89d5be None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:05.175634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9951-0392-4639-a157-f7ea0d89d5be None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:05.175634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9951-0392-4639-a157-f7ea0d89d5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:05.207568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d66b9951-0392-4639-a157-f7ea0d89d5be None None] MFA Rules not processed for user `b11313762b324174a9a5f362d1cf4bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:05.208720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d66b9951-0392-4639-a157-f7ea0d89d5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:05.287982 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1963/3959] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:06.213823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:06.215944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:06.239279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ef20-f47a-4709-9181-f959d54fc088 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:06.257603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:06.258362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:06.258645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:06.258916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] PATH_INFO: `/v3/projects/19cc9e93c3e44c8db209bab41f965d5f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:06.266255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19cc9e93c3e44c8db209bab41f965d5f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:06.269932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:06.377342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19cc9e93c3e44c8db209bab41f965d5f', 'request_id': 'req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.403543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19cc9e93c3e44c8db209bab41f965d5f', 'request_id': 'req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.412342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19cc9e93c3e44c8db209bab41f965d5f', 'request_id': 'req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.425807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] The token cache is being invalidate because project 19cc9e93c3e44c8db209bab41f965d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:06.426376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.427203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.433263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b9951-0392-4639-a157-f7ea0d89d5be None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:06.435344 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1997/3960] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:06 2022] DELETE /identity/v3/projects/19cc9e93c3e44c8db209bab41f965d5f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:06.448147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b9951-0392-4639-a157-f7ea0d89d5be None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:06.562787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:06.563780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:06.564255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:06.564737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] PATH_INFO: `/v3/users/a5b171b12fed40ce84121a79c6ceb327` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:06.565886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b171b12fed40ce84121a79c6ceb327)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:06.568819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:06.612882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:06.619179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:06.634326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e12b27-5e6c-4e18-8bb3-8f16cd952ef7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:06.641927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:06.642768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:06.643045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:06.643318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] PATH_INFO: `/v3/users/92b0186cd92744dabd001b86560b5c90` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:06.645244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b0186cd92744dabd001b86560b5c90)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:06.648583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:06.707461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b171b12fed40ce84121a79c6ceb327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.708396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b171b12fed40ce84121a79c6ceb327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.712206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:06.720537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b171b12fed40ce84121a79c6ceb327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.746488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b171b12fed40ce84121a79c6ceb327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.769110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b0186cd92744dabd001b86560b5c90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.769536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b0186cd92744dabd001b86560b5c90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.788848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b0186cd92744dabd001b86560b5c90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.802794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b0186cd92744dabd001b86560b5c90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.803557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b0186cd92744dabd001b86560b5c90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.824904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b171b12fed40ce84121a79c6ceb327'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:06.841354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1964/3961] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:06 2022] DELETE /identity/v3/users/a5b171b12fed40ce84121a79c6ceb327 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:06.843867 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1998/3962] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:06 2022] DELETE /identity/v3/users/92b0186cd92744dabd001b86560b5c90 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:07.957539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:07.959386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:08.060776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1d15-b494-4f84-b994-c855257d933c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:08.172568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:08.175704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:08.182576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:08.224912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:08.225399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.225911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.226426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] PATH_INFO: `/v3/projects/027612746ecc44f4ab8e7af121ec27a1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.238224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=027612746ecc44f4ab8e7af121ec27a1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:08.247538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:08.332864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf32555-f3df-496e-b7fa-526d431d9a5f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:08.351304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:08.352011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.352299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.352575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] PATH_INFO: `/v3/projects/3b204c772fbf4691ad2ecb8623833160` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.356079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b204c772fbf4691ad2ecb8623833160)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:08.358858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:08.390159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027612746ecc44f4ab8e7af121ec27a1', 'request_id': 'req-5e209fbb-5350-4d41-ad09-4b34f90a2d87'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.411205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027612746ecc44f4ab8e7af121ec27a1', 'request_id': 'req-5e209fbb-5350-4d41-ad09-4b34f90a2d87'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.433481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027612746ecc44f4ab8e7af121ec27a1', 'request_id': 'req-5e209fbb-5350-4d41-ad09-4b34f90a2d87'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.455666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] The token cache is being invalidate because project 027612746ecc44f4ab8e7af121ec27a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:08.455916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b204c772fbf4691ad2ecb8623833160', 'request_id': 'req-a85ecab2-755c-4b38-8527-5019797c67c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.456726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.458950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e209fbb-5350-4d41-ad09-4b34f90a2d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.465747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1965/3963] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:07 2022] DELETE /identity/v3/projects/027612746ecc44f4ab8e7af121ec27a1 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:08.473387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54f730-35a4-4eaa-a3ea-bb51aaf8d2ca None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.473631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54f730-35a4-4eaa-a3ea-bb51aaf8d2ca None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.473933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54f730-35a4-4eaa-a3ea-bb51aaf8d2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.483332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b204c772fbf4691ad2ecb8623833160', 'request_id': 'req-a85ecab2-755c-4b38-8527-5019797c67c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.507799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:08.517431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b204c772fbf4691ad2ecb8623833160', 'request_id': 'req-a85ecab2-755c-4b38-8527-5019797c67c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.545558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] The token cache is being invalidate because project 3b204c772fbf4691ad2ecb8623833160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:08.545857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.550620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a85ecab2-755c-4b38-8527-5019797c67c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:08.552733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2d54f730-35a4-4eaa-a3ea-bb51aaf8d2ca None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:08.559244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2d54f730-35a4-4eaa-a3ea-bb51aaf8d2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:08.562167 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 1999/3964] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:08 2022] DELETE /identity/v3/projects/3b204c772fbf4691ad2ecb8623833160 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:08.570039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200469ab-aef5-4064-b527-7b5bee53ba59 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.570725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200469ab-aef5-4064-b527-7b5bee53ba59 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.571012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200469ab-aef5-4064-b527-7b5bee53ba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.632710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-200469ab-aef5-4064-b527-7b5bee53ba59 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:08.634403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-200469ab-aef5-4064-b527-7b5bee53ba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:08.772781 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2000/3965] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:08.781310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8e52-7e1b-4113-8a0a-5f3c7aa92947 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.782015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8e52-7e1b-4113-8a0a-5f3c7aa92947 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.783230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8e52-7e1b-4113-8a0a-5f3c7aa92947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.786313 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1966/3966] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:08.795346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83a92-6577-4050-90d9-4d608b326131 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.795626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83a92-6577-4050-90d9-4d608b326131 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.795900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83a92-6577-4050-90d9-4d608b326131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.830336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3ba83a92-6577-4050-90d9-4d608b326131 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:08.831617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3ba83a92-6577-4050-90d9-4d608b326131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:08.838277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-321a8e52-7e1b-4113-8a0a-5f3c7aa92947 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:08.839516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-321a8e52-7e1b-4113-8a0a-5f3c7aa92947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:08.859428 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1967/3967] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:08.860316 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2001/3968] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:08.863790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83a92-6577-4050-90d9-4d608b326131 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:08.864439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a8e52-7e1b-4113-8a0a-5f3c7aa92947 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:08.879771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83a92-6577-4050-90d9-4d608b326131 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:08.885217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:08.886400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.886855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.887210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a8e52-7e1b-4113-8a0a-5f3c7aa92947 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:08.887457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.890040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:08.890569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:08.892485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:08.901058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:08.904909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.905158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.905455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.906137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:08.906438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:08.915613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:08.915819 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1968/3969] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:08.916751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:08.934586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f7d2c-a07b-4f6f-8c9c-471324fc638f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:08.940556 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2002/3970] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:08.942591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:08.946319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:08.946955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.947233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.947505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.948147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:08.950167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:08.960316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcab19-9463-43ea-b6b3-cd9540807511 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:08.966441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:08.967410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:08.967785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:08.968089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:08.969132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:08.972147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:08.987050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:09.008788 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1969/3971] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:08 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:09.010971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.015390 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2003/3972] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:08 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:09.020053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.173773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fbdb5-3c04-40bd-ac68-6f5507cfd391 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.180789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38a3c6-bd27-4013-8991-f51dd58fa1a7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.186793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.187422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.187704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.188560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.188999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.191329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.193235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.193954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.194259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.194567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.195570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.197404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.224280 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1970/3973] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:09 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:11:09.230125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.238633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2004/3974] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:09 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:11:09.244534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.257377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148448d9-6ee8-42f2-b164-392be4cc4a09 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.265211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501394a7-6c98-4884-94b3-173f33e8b33f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.268246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.269191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.269621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.270103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.270953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.272379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.273310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.273641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.273796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.274111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.274883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.277144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.287789 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2005/3975] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:09.291827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.295567 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1971/3976] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:09.300700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.305796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e3046-3311-4652-af80-3b42b64bdb66 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.311375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.312207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.312636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.313118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.313902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.315833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.326461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe5394-5f3f-473f-917a-472230c11313 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.332563 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2006/3977] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:09.334815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.335842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.336245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.336617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.337309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.337492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.339887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.357085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89992fb-0ceb-4c36-b006-117c8134457e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.357441 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1972/3978] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:09.364626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb63985-bc58-48a6-afc1-a3b6463cb8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.365813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63985-bc58-48a6-afc1-a3b6463cb8de admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.366277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63985-bc58-48a6-afc1-a3b6463cb8de admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.366646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63985-bc58-48a6-afc1-a3b6463cb8de admin admin] PATH_INFO: `/v3/projects/98fa4f91786c4a60abcf0b9cc50d7689/users/e34d4d2b42ab4046aeb36bc90b9e51cf/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.369561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.388149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c0cb8-6bf9-40b8-86b1-8b83f4c2accf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.390903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb63985-bc58-48a6-afc1-a3b6463cb8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fa4f91786c4a60abcf0b9cc50d7689, user_id=e34d4d2b42ab4046aeb36bc90b9e51cf, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.393874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afaed0-519a-4b4c-a7fb-39ae59a61a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.394622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afaed0-519a-4b4c-a7fb-39ae59a61a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.395241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afaed0-519a-4b4c-a7fb-39ae59a61a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.395576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afaed0-519a-4b4c-a7fb-39ae59a61a29 admin admin] PATH_INFO: `/v3/projects/d729807b74a241d7a17b184897aa595e/users/7d3c9ae0a3b34cd487cf069002f6d70e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.398738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb63985-bc58-48a6-afc1-a3b6463cb8de admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.418957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afaed0-519a-4b4c-a7fb-39ae59a61a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d729807b74a241d7a17b184897aa595e, user_id=7d3c9ae0a3b34cd487cf069002f6d70e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.424534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afaed0-519a-4b4c-a7fb-39ae59a61a29 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.425468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2007/3979] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:09 2022] PUT /identity/v3/projects/98fa4f91786c4a60abcf0b9cc50d7689/users/e34d4d2b42ab4046aeb36bc90b9e51cf/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:09.442160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67c008-e537-403e-8258-de81c13e134a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.442817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67c008-e537-403e-8258-de81c13e134a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.443262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67c008-e537-403e-8258-de81c13e134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.456882 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1973/3980] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:09 2022] PUT /identity/v3/projects/d729807b74a241d7a17b184897aa595e/users/7d3c9ae0a3b34cd487cf069002f6d70e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:09.461690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.462573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.463598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.485702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:09.504755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5c67c008-e537-403e-8258-de81c13e134a None None] MFA Rules not processed for user `e34d4d2b42ab4046aeb36bc90b9e51cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:09.506136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5c67c008-e537-403e-8258-de81c13e134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:09.535953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] MFA Rules not processed for user `7d3c9ae0a3b34cd487cf069002f6d70e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:09.537310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:09.599138 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2008/3981] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:09.609366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67c008-e537-403e-8258-de81c13e134a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.621559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67c008-e537-403e-8258-de81c13e134a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.634148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34d4d2b42ab4046aeb36bc90b9e51cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98fa4f91786c4a60abcf0b9cc50d7689', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.634816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.635099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.635373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] PATH_INFO: `/v3/domains/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.638176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.642246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.652559 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2009/3982] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:11:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:09.652559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.661056 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1974/3983] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:09.662359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.664082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:09.683530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:09.749079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566b677-75de-4c4f-88c6-71577593ddc1 tempest-DefaultDomainTestJSON-818078459 tempest-DefaultDomainTestJSON-818078459-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.762421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.763088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.763364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.763633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] PATH_INFO: `/v3/users/e34d4d2b42ab4046aeb36bc90b9e51cf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.766080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34d4d2b42ab4046aeb36bc90b9e51cf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.768053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.795341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85cb92-351f-49d7-ae3d-8f559966e69c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:09.805555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2e39-ffd2-4275-b769-86113ddeb81f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:09.806643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2e39-ffd2-4275-b769-86113ddeb81f service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.807814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2e39-ffd2-4275-b769-86113ddeb81f service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.808311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2e39-ffd2-4275-b769-86113ddeb81f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.822426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2e39-ffd2-4275-b769-86113ddeb81f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:09.824144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2e39-ffd2-4275-b769-86113ddeb81f service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:09.836379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1975/3984] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:09.837039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34d4d2b42ab4046aeb36bc90b9e51cf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:09.838042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34d4d2b42ab4046aeb36bc90b9e51cf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:09.860066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.860379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.860711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34d4d2b42ab4046aeb36bc90b9e51cf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:09.860881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.862829 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1976/3985] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 10:11:09.871264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:11:09.874863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34d4d2b42ab4046aeb36bc90b9e51cf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:09.875608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0a7dd21c-02ea-4fb5-9a71-8a0efe5f2fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34d4d2b42ab4046aeb36bc90b9e51cf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:09.883738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:09.899811 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2010/3986] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:09 2022] DELETE /identity/v3/users/e34d4d2b42ab4046aeb36bc90b9e51cf => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:09.905593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:09.906047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:09.906583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:09.925247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:09.928487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:09.933426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:09.934897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:09.953649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016843e-c9d3-48ec-8cce-216b8d102cdb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.010006 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2011/3987] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:10.022281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:10.025838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:10.026368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23659f48-9473-453d-b471-f23f22633196 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d3c9ae0a3b34cd487cf069002f6d70e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd729807b74a241d7a17b184897aa595e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.027102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23659f48-9473-453d-b471-f23f22633196 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.027682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23659f48-9473-453d-b471-f23f22633196 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.027948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23659f48-9473-453d-b471-f23f22633196 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] PATH_INFO: `/v3/projects/d729807b74a241d7a17b184897aa595e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.029110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23659f48-9473-453d-b471-f23f22633196 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorizing `identity:get_project(project_id=d729807b74a241d7a17b184897aa595e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.031386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23659f48-9473-453d-b471-f23f22633196 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.033844 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1977/3988] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:11:09 2022] GET /identity/v3/projects/d729807b74a241d7a17b184897aa595e => generated 365 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:11:10.123389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:10.155603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434f723-d046-42d3-98a2-49feae1cd2ee None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.164190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78777ee-930a-49c9-b55d-d197493597da None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.164883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78777ee-930a-49c9-b55d-d197493597da None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.165589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78777ee-930a-49c9-b55d-d197493597da None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.167449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06acc5e-ac1c-4e33-b832-a3a7a6dc5b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.168187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06acc5e-ac1c-4e33-b832-a3a7a6dc5b15 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.168335 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1978/3989] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:11:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 10:11:10.168534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06acc5e-ac1c-4e33-b832-a3a7a6dc5b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.169490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06acc5e-ac1c-4e33-b832-a3a7a6dc5b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.184398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78777ee-930a-49c9-b55d-d197493597da None None] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:11:10.186227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b78777ee-930a-49c9-b55d-d197493597da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:10.207756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06acc5e-ac1c-4e33-b832-a3a7a6dc5b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.209279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06acc5e-ac1c-4e33-b832-a3a7a6dc5b15 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.248401 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2012/3990] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:10.249219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:10.336145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78777ee-930a-49c9-b55d-d197493597da None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:10.347955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78777ee-930a-49c9-b55d-d197493597da None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.357710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d3c9ae0a3b34cd487cf069002f6d70e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd729807b74a241d7a17b184897aa595e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.359832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.360291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.360837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] PATH_INFO: `/v3/projects/d729807b74a241d7a17b184897aa595e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.366559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorizing `identity:get_project(project_id=d729807b74a241d7a17b184897aa595e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.369289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.380358 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1979/3991] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:11:10 2022] GET /identity/v3/projects/d729807b74a241d7a17b184897aa595e => generated 365 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:11:10.417728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8259563-e7f9-474c-83bd-e751b48369bc None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.418224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8259563-e7f9-474c-83bd-e751b48369bc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.418880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8259563-e7f9-474c-83bd-e751b48369bc None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.421134 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2013/3992] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:11:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:11:10.427550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:11:10.444953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:10.456123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9fc6b-39d5-4869-a414-cc73392a718f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.462844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d3c9ae0a3b34cd487cf069002f6d70e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd729807b74a241d7a17b184897aa595e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.463754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.464107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.464433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] PATH_INFO: `/v3/projects/d729807b74a241d7a17b184897aa595e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.465811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorizing `identity:get_project(project_id=d729807b74a241d7a17b184897aa595e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.469482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.487498 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1980/3993] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:11:10 2022] GET /identity/v3/projects/d729807b74a241d7a17b184897aa595e => generated 365 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:10.516800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd26acb-1e4f-49a4-b126-1522f78a48e4 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.517051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd26acb-1e4f-49a4-b126-1522f78a48e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.518374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd26acb-1e4f-49a4-b126-1522f78a48e4 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.520375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2014/3994] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:11:10.527003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:11:10.558953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:10.569938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad8d98-26fa-4678-a516-e598a67a87c1 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.575022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d3c9ae0a3b34cd487cf069002f6d70e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd729807b74a241d7a17b184897aa595e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.575677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.575961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.576241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] PATH_INFO: `/v3/projects/d729807b74a241d7a17b184897aa595e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.577991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorizing `identity:get_project(project_id=d729807b74a241d7a17b184897aa595e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.579808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.581953 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1981/3995] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:11:10 2022] GET /identity/v3/projects/d729807b74a241d7a17b184897aa595e => generated 365 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:10.650720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae47e5-b295-4fb6-aede-dff9926c60e9 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.650994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae47e5-b295-4fb6-aede-dff9926c60e9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.651448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae47e5-b295-4fb6-aede-dff9926c60e9 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.652908 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2015/3996] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:11:10.661446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Authenticating service token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:11:10.678254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:10.689041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e441d95-9c4e-4ea4-95ef-7e556b7836dd tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.694626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d3c9ae0a3b34cd487cf069002f6d70e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd729807b74a241d7a17b184897aa595e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.696108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.696586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.697074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] PATH_INFO: `/v3/projects/d729807b74a241d7a17b184897aa595e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.698718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorizing `identity:get_project(project_id=d729807b74a241d7a17b184897aa595e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.700749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.709903 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1982/3997] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:11:10 2022] GET /identity/v3/projects/d729807b74a241d7a17b184897aa595e => generated 365 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:10.745932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae47e5-b295-4fb6-aede-dff9926c60e9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:10.747822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-30ae47e5-b295-4fb6-aede-dff9926c60e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:10.764223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:10.766914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:10.769464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae47e5-b295-4fb6-aede-dff9926c60e9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.784507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.785232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.785921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde9ce3-b252-409d-8ae8-6d8d6662757f tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:10.786238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.786542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] PATH_INFO: `/v3/users/7d3c9ae0a3b34cd487cf069002f6d70e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.789811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3c9ae0a3b34cd487cf069002f6d70e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.791777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:10.792272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.792717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.793092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.793583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] PATH_INFO: `/v3/projects/98fa4f91786c4a60abcf0b9cc50d7689` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.794923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=98fa4f91786c4a60abcf0b9cc50d7689)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:10.797123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:10.854771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98fa4f91786c4a60abcf0b9cc50d7689', 'request_id': 'req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.863147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3c9ae0a3b34cd487cf069002f6d70e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.863404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98fa4f91786c4a60abcf0b9cc50d7689', 'request_id': 'req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.865674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3c9ae0a3b34cd487cf069002f6d70e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.875217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98fa4f91786c4a60abcf0b9cc50d7689', 'request_id': 'req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.877832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3c9ae0a3b34cd487cf069002f6d70e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.887113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3c9ae0a3b34cd487cf069002f6d70e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.889336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30308b62-01d5-436a-8668-b1eb07ceb917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3c9ae0a3b34cd487cf069002f6d70e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.889810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] The token cache is being invalidate because project 98fa4f91786c4a60abcf0b9cc50d7689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:10.890692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.891431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdc3704d-5a4a-402e-bf0a-e88152a2d88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:10.893592 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1983/3998] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:10 2022] DELETE /identity/v3/projects/98fa4f91786c4a60abcf0b9cc50d7689 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:10.898334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda5e3-326f-4599-82f2-1ec6369960f5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.898915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda5e3-326f-4599-82f2-1ec6369960f5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.899195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda5e3-326f-4599-82f2-1ec6369960f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.918098 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2016/3999] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:10 2022] DELETE /identity/v3/users/7d3c9ae0a3b34cd487cf069002f6d70e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:10.922307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:10.922587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:10.922855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:10.927858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-51cda5e3-326f-4599-82f2-1ec6369960f5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:10.929582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-51cda5e3-326f-4599-82f2-1ec6369960f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:10.947036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:10.948550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:11.002981 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1984/4000] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:11.013553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f960-dce3-4c6e-8421-a9f8356ba2c4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.014137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f960-dce3-4c6e-8421-a9f8356ba2c4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.014660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f960-dce3-4c6e-8421-a9f8356ba2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.037039 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2017/4001] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:11.045831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-67b5f960-dce3-4c6e-8421-a9f8356ba2c4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:11.046891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.047751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-67b5f960-dce3-4c6e-8421-a9f8356ba2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:11.048398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:11.060052 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1985/4002] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:11.061429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f960-dce3-4c6e-8421-a9f8356ba2c4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.067752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:11.073090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f960-dce3-4c6e-8421-a9f8356ba2c4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.081681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.082827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.083191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.083915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.085435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:11.086240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:11.088952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:11.117143 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1986/4003] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:11:11.120304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.131925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8545856-6d61-4f91-8a89-d0d616d8e42c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.138486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.140758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.141197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.141496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.142405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:11.144416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:11.151491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85075e5-e9e5-47a9-82c7-3fc8ee84bddb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.156473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.157799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.158793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.159991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.177645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:11.183270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:11.195824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:11.197516 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1987/4004] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:11 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:11.202959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.214441 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2018/4005] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:11.329114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affdb5a-7b2d-4c98-a7f3-6fc6c455168b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.336947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.337474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.337754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.338033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.338677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:11.341203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:11.385761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1988/4006] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:11 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:11.390606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.411663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ca06-c2fd-4a43-9947-6f20d538b86b service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.421630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.422502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.422781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.423052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.423692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:11.425376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:11.438074 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2019/4007] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:11:11.443838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.463455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0f06-0b8d-426d-9ec2-a256c96aed46 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.468622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d1949-9414-4923-ae21-235394109883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.469412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1949-9414-4923-ae21-235394109883 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.469991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1949-9414-4923-ae21-235394109883 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.470146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1949-9414-4923-ae21-235394109883 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.471004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d1949-9414-4923-ae21-235394109883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:11.472829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d1949-9414-4923-ae21-235394109883 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:11.486828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1989/4008] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:11.488317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.499775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869aedf-5baf-4592-8e1c-197fdf280c06 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.513587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.514445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.514776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.515123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] PATH_INFO: `/v3/projects/53a09593ff74430fa95aace000e41fcf/users/d212a58e4ba2431a88cb76f30a245d37/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.533320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a09593ff74430fa95aace000e41fcf, user_id=d212a58e4ba2431a88cb76f30a245d37, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:11.536717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:11.556909 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2020/4009] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:11 2022] PUT /identity/v3/projects/53a09593ff74430fa95aace000e41fcf/users/d212a58e4ba2431a88cb76f30a245d37/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:11.568861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c1db3-643f-4419-aab7-998b19488236 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.569167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c1db3-643f-4419-aab7-998b19488236 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.569498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c1db3-643f-4419-aab7-998b19488236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.603511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-904c1db3-643f-4419-aab7-998b19488236 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:11.607588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-904c1db3-643f-4419-aab7-998b19488236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:11.685468 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1990/4010] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:11.690980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:11.703599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bb814-c10f-4149-ad87-e08e42e2eb71 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:11.713234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c95aff-a4eb-4f27-b3cb-cbadf97836ed tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:11.714004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c95aff-a4eb-4f27-b3cb-cbadf97836ed tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:11.715113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c95aff-a4eb-4f27-b3cb-cbadf97836ed tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:11.715397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c95aff-a4eb-4f27-b3cb-cbadf97836ed tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:11.766452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7c95aff-a4eb-4f27-b3cb-cbadf97836ed tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:11.776482 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2021/4011] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:11 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:12.205081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c1db3-643f-4419-aab7-998b19488236 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:12.208753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-904c1db3-643f-4419-aab7-998b19488236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:12.288404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:12.346348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c1db3-643f-4419-aab7-998b19488236 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:12.351888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:12.352790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.353226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.353508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] PATH_INFO: `/v3/projects/d729807b74a241d7a17b184897aa595e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.355233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d729807b74a241d7a17b184897aa595e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:12.357436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:12.422384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b10a8-f0dc-48a4-81d0-0bc6c3d653c1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.422942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b10a8-f0dc-48a4-81d0-0bc6c3d653c1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.423052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b10a8-f0dc-48a4-81d0-0bc6c3d653c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.439399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd729807b74a241d7a17b184897aa595e', 'request_id': 'req-b246b43c-6425-4099-ab51-de203022e2df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:12.455932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd729807b74a241d7a17b184897aa595e', 'request_id': 'req-b246b43c-6425-4099-ab51-de203022e2df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:12.477403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd729807b74a241d7a17b184897aa595e', 'request_id': 'req-b246b43c-6425-4099-ab51-de203022e2df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:12.497798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f59b10a8-f0dc-48a4-81d0-0bc6c3d653c1 None None] MFA Rules not processed for user `96e48f22b8e7478986f912e64df670b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:12.501920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f59b10a8-f0dc-48a4-81d0-0bc6c3d653c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:12.512351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] The token cache is being invalidate because project d729807b74a241d7a17b184897aa595e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:12.513156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:12.515120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b246b43c-6425-4099-ab51-de203022e2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:12.524812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1991/4012] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:12 2022] DELETE /identity/v3/projects/d729807b74a241d7a17b184897aa595e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:12.530490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917a8cc-31fe-4fa4-a42b-870ce7411e04 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.530926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917a8cc-31fe-4fa4-a42b-870ce7411e04 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.533650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917a8cc-31fe-4fa4-a42b-870ce7411e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.575117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3917a8cc-31fe-4fa4-a42b-870ce7411e04 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:12.578593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3917a8cc-31fe-4fa4-a42b-870ce7411e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:12.624744 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2022/4013] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:12.631043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.631326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.631598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.675487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] MFA Rules not processed for user `96e48f22b8e7478986f912e64df670b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:12.679135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:12.696948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2023/4014] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:12.699546 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1992/4015] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:12.706917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:12.709080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ffe74-842c-4a17-82bc-6d039729b7db None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.709646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ffe74-842c-4a17-82bc-6d039729b7db None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.711470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:12.712630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ffe74-842c-4a17-82bc-6d039729b7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.761767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d18ffe74-842c-4a17-82bc-6d039729b7db None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:12.764078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d18ffe74-842c-4a17-82bc-6d039729b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:12.788601 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1993/4016] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:12.792366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79510-f02a-4faa-a2e4-a7b865ab9734 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.792985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79510-f02a-4faa-a2e4-a7b865ab9734 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.793580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79510-f02a-4faa-a2e4-a7b865ab9734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.849144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b8e79510-f02a-4faa-a2e4-a7b865ab9734 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:12.853886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b8e79510-f02a-4faa-a2e4-a7b865ab9734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:12.867432 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1994/4017] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:12 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:12.868888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79510-f02a-4faa-a2e4-a7b865ab9734 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:12.910632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd67ca-90b1-4bb4-9778-3907a4185ee3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:12.921637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79510-f02a-4faa-a2e4-a7b865ab9734 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:12.927804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bc8cc-36d2-4f75-ace0-0849c3db8c4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:12.930063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bc8cc-36d2-4f75-ace0-0849c3db8c4a service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.930779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bc8cc-36d2-4f75-ace0-0849c3db8c4a service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.931480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bc8cc-36d2-4f75-ace0-0849c3db8c4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.944925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:12.945585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.945863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.946137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:12.947024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:12.947341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:12.948920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:12.952804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bc8cc-36d2-4f75-ace0-0849c3db8c4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:12.956273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bc8cc-36d2-4f75-ace0-0849c3db8c4a service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:12.966736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:12.984488 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2024/4018] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:12.986660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed2af-7897-4ca9-beb4-e1a7380c92d0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:12.987993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed2af-7897-4ca9-beb4-e1a7380c92d0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:12.988440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed2af-7897-4ca9-beb4-e1a7380c92d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.023114 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1995/4019] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:13.027042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.037110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-536ed2af-7897-4ca9-beb4-e1a7380c92d0 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:13.043623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-536ed2af-7897-4ca9-beb4-e1a7380c92d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:13.079720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba40236-f17c-4e65-aa3b-df86d4c5b6e6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.125912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.126557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.126840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.127114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.127952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:13.133718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:13.190702 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2025/4020] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:13.195382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ed2af-7897-4ca9-beb4-e1a7380c92d0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.212834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ed2af-7897-4ca9-beb4-e1a7380c92d0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.222613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21da9d-d27c-4700-b7f7-3db8db8fdc0d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.224039 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1996/4021] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:13 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:13.225006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21da9d-d27c-4700-b7f7-3db8db8fdc0d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.225728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21da9d-d27c-4700-b7f7-3db8db8fdc0d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.227459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21da9d-d27c-4700-b7f7-3db8db8fdc0d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.228628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.286530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-2c21da9d-d27c-4700-b7f7-3db8db8fdc0d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:13.321054 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 27 10:11:13.322330 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:13.325044 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2026/4022] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:13 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 188 bytes in 130 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 27 10:11:13.329145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21da9d-d27c-4700-b7f7-3db8db8fdc0d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.358670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21da9d-d27c-4700-b7f7-3db8db8fdc0d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.383884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db959fd2-f722-4526-ba0e-37980ceff3ee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.394089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.394725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.395011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.395280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.395923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:13.398331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:13.447876 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1997/4023] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:13 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:13.449348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.467470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5612864-39d8-4fb1-8479-8aba4fb3625d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.468203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6ba8-af91-47e6-bad1-f6dad7112ee4 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.470895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6ba8-af91-47e6-bad1-f6dad7112ee4 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.471589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6ba8-af91-47e6-bad1-f6dad7112ee4 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.472115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6ba8-af91-47e6-bad1-f6dad7112ee4 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.480444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.481110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.481387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.481659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.482318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:13.483920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:13.511558 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1998/4024] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:11:13.512624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.539043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8a232-36e9-47a9-aa80-0b1bd05aeee8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-759f6ba8-af91-47e6-bad1-f6dad7112ee4 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:13.541525 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 27 10:11:13.542523 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:13.549416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2027/4025] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:13 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 188 bytes in 220 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:11:13.550374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6ba8-af91-47e6-bad1-f6dad7112ee4 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.552132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.552813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.553084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.553377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.554033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:13.555958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:13.574502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6ba8-af91-47e6-bad1-f6dad7112ee4 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.620939 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 1999/4026] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:13.621947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:13.622986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.623645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.623920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.624189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.646739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88642040-9cc0-4c9a-9c5b-06e80db819b2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:13.657247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79f10b-d75b-4249-a85d-07226ddd5ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:13.657941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f10b-d75b-4249-a85d-07226ddd5ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.658219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f10b-d75b-4249-a85d-07226ddd5ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.658493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f10b-d75b-4249-a85d-07226ddd5ffe admin admin] PATH_INFO: `/v3/projects/cb17c2ebff4e4bd9b5844a58cd8677b1/users/28f357781a0b4818a8a9c8945f318ffe/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.693245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79f10b-d75b-4249-a85d-07226ddd5ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb17c2ebff4e4bd9b5844a58cd8677b1, user_id=28f357781a0b4818a8a9c8945f318ffe, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:13.696364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79f10b-d75b-4249-a85d-07226ddd5ffe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:13.727581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:13.735306 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2000/4027] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:13 2022] PUT /identity/v3/projects/cb17c2ebff4e4bd9b5844a58cd8677b1/users/28f357781a0b4818a8a9c8945f318ffe/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:13.742518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:13.742809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:13.743087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:13.753350 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2028/4028] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:13 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:13.796011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:13.797573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:13.927870 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2001/4029] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:13.997352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:14.000222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:14.110203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:14.149061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2cb9f-8b8a-4031-a2ac-b14b3f1d19ae tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:14.284092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb793770-f916-426f-a7c8-91ca52f504b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:14.284824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793770-f916-426f-a7c8-91ca52f504b5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:14.285142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793770-f916-426f-a7c8-91ca52f504b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:14.285433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793770-f916-426f-a7c8-91ca52f504b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:14.305902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb793770-f916-426f-a7c8-91ca52f504b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:14.307816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb793770-f916-426f-a7c8-91ca52f504b5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:14.321789 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2029/4030] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:14.464180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:14.465822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:14.491830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046694a5-8883-41f9-9eb2-fff3cafa8715 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:14.504190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:14.505553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:14.506362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:14.506999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] PATH_INFO: `/v3/users/7f7c5adbac4f48b6bc141e0f888f2496` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:14.510399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7c5adbac4f48b6bc141e0f888f2496)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:14.512626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:14.633411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7c5adbac4f48b6bc141e0f888f2496'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:14.636130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7c5adbac4f48b6bc141e0f888f2496'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:14.651347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7c5adbac4f48b6bc141e0f888f2496'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:14.656027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:14.672542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7c5adbac4f48b6bc141e0f888f2496'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:14.691895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-506acfe2-02b5-44e4-8f01-867d50318080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7c5adbac4f48b6bc141e0f888f2496'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:14.712840 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2002/4031] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:14 2022] DELETE /identity/v3/users/7f7c5adbac4f48b6bc141e0f888f2496 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:14.760447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44033c-7c8e-455b-8044-8b93174517cb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:14.761048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44033c-7c8e-455b-8044-8b93174517cb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:14.761517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44033c-7c8e-455b-8044-8b93174517cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:14.787648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1f44033c-7c8e-455b-8044-8b93174517cb None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:14.789582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1f44033c-7c8e-455b-8044-8b93174517cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:14.794488 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2030/4032] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:14 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:14.800191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49cd4d-9153-4d15-b650-817431727abd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:14.801049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49cd4d-9153-4d15-b650-817431727abd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:14.801578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49cd4d-9153-4d15-b650-817431727abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:14.857425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1f49cd4d-9153-4d15-b650-817431727abd None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:14.859175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f49cd4d-9153-4d15-b650-817431727abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:14.888594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8f381-db25-4456-b949-c388b6f51615 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:14.889782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8f381-db25-4456-b949-c388b6f51615 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:14.890269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8f381-db25-4456-b949-c388b6f51615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:15.002064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3ab8f381-db25-4456-b949-c388b6f51615 None None] MFA Rules not processed for user `28f357781a0b4818a8a9c8945f318ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:15.003211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ab8f381-db25-4456-b949-c388b6f51615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:15.036520 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2003/4033] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:15.037926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49cd4d-9153-4d15-b650-817431727abd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:15.059293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49cd4d-9153-4d15-b650-817431727abd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:15.070108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:15.072783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:15.074507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:15.074902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:15.253846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2031/4034] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:15.258271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:15.258559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:15.258827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:15.279036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:15.294941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] MFA Rules not processed for user `28f357781a0b4818a8a9c8945f318ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:15.296277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:15.298713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2004/4035] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:15 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:15.306653 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2032/4036] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:15.510966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:15.513427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:15.556729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:15.558773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:15.660983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2f65-cd81-4805-b3d9-5015474560b2 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:15.670566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed74ac-93be-4933-8227-5b5278eb525c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:15.671597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed74ac-93be-4933-8227-5b5278eb525c service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:15.672190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed74ac-93be-4933-8227-5b5278eb525c service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:15.672637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed74ac-93be-4933-8227-5b5278eb525c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:15.686611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed74ac-93be-4933-8227-5b5278eb525c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:15.689111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed74ac-93be-4933-8227-5b5278eb525c service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:15.714651 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2005/4037] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:15.723780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723249d-7982-4e34-a84b-ea250970c29c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:15.753528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:15.755749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:15.756038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:15.756313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] PATH_INFO: `/v3/projects/6c681ff8fe354f97abb5fe0d5181e6f1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:15.760373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c681ff8fe354f97abb5fe0d5181e6f1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:15.773937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:15.825443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:15.890287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c681ff8fe354f97abb5fe0d5181e6f1', 'request_id': 'req-830cc03f-15ea-4deb-bb78-6c32ce8a0414'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:15.906996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c681ff8fe354f97abb5fe0d5181e6f1', 'request_id': 'req-830cc03f-15ea-4deb-bb78-6c32ce8a0414'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:15.916194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c681ff8fe354f97abb5fe0d5181e6f1', 'request_id': 'req-830cc03f-15ea-4deb-bb78-6c32ce8a0414'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:15.930623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] The token cache is being invalidate because project 6c681ff8fe354f97abb5fe0d5181e6f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:15.931134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:15.931896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-830cc03f-15ea-4deb-bb78-6c32ce8a0414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:15.934361 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2033/4038] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:15 2022] DELETE /identity/v3/projects/6c681ff8fe354f97abb5fe0d5181e6f1 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:15.939137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165731ae-058b-4786-8591-71e8c349d7d2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:15.939419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165731ae-058b-4786-8591-71e8c349d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:15.939692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165731ae-058b-4786-8591-71e8c349d7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:15.978541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-165731ae-058b-4786-8591-71e8c349d7d2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:15.979804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-165731ae-058b-4786-8591-71e8c349d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:16.102972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:16.228855 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2006/4039] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:16.238445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73073733-998b-4cd2-aa3d-8b7894849cdf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.238751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73073733-998b-4cd2-aa3d-8b7894849cdf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.239027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73073733-998b-4cd2-aa3d-8b7894849cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.306221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310866e9-546d-4360-bac8-b2ee48642cc3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.306974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310866e9-546d-4360-bac8-b2ee48642cc3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.307402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310866e9-546d-4360-bac8-b2ee48642cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.345369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-73073733-998b-4cd2-aa3d-8b7894849cdf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:16.347678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-73073733-998b-4cd2-aa3d-8b7894849cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:16.384561 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2034/4040] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:16.386516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73073733-998b-4cd2-aa3d-8b7894849cdf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:16.394754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-310866e9-546d-4360-bac8-b2ee48642cc3 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:16.399386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73073733-998b-4cd2-aa3d-8b7894849cdf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:16.400401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-310866e9-546d-4360-bac8-b2ee48642cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:16.413385 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2007/4041] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:16 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:16.415235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08fd73-7e45-4628-8bf3-a78e4d723e5f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.416737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:16.418614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08fd73-7e45-4628-8bf3-a78e4d723e5f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.419082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.419573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08fd73-7e45-4628-8bf3-a78e4d723e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.420075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.421835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.422735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:16.424006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:16.429420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:16.478165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2035/4042] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:16.480097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:16.487071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1f08fd73-7e45-4628-8bf3-a78e4d723e5f None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:16.488333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f08fd73-7e45-4628-8bf3-a78e4d723e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:16.507700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7812594-da2e-4504-8895-ede0f3b44dff admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:16.522597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:16.522883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.523172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.523447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.524136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:16.531613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:16.628506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2036/4043] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:16 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:16.629797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:16.664061 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2008/4044] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:16.668196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08fd73-7e45-4628-8bf3-a78e4d723e5f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:16.696364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08fd73-7e45-4628-8bf3-a78e4d723e5f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:16.800002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cea32a-aaf3-406c-8c01-681e40136b46 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:16.810450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:16.811781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.812295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.812822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.815008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:16.817713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:16.842099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2635df7-3022-4ac4-be64-f8b56a0b7518 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:16.842991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2635df7-3022-4ac4-be64-f8b56a0b7518 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.843365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2635df7-3022-4ac4-be64-f8b56a0b7518 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.843883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2635df7-3022-4ac4-be64-f8b56a0b7518 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.864569 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2037/4045] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:16 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:16.868793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:16.886079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122964c-464a-42f8-a481-6a6669b3e0b5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:16.904098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:16.905455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.905950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.906429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.907357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:16.909441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:16.940524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2635df7-3022-4ac4-be64-f8b56a0b7518 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:16.951407 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2038/4046] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:11:16.955720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:16.969800 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2009/4047] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:16 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:16.984890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb3b5f-e6bf-479d-9edf-03894a731066 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:16.992856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:16.993632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:16.994397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:16.994811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:16.996834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:16.998630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:17.014427 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2039/4048] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:17.019285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2635df7-3022-4ac4-be64-f8b56a0b7518 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:17.035868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2635df7-3022-4ac4-be64-f8b56a0b7518 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:17.044138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcb3a8-e0a5-4965-996a-fb2e4c23e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:17.045887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcb3a8-e0a5-4965-996a-fb2e4c23e95e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:17.046334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcb3a8-e0a5-4965-996a-fb2e4c23e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:17.046767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcb3a8-e0a5-4965-996a-fb2e4c23e95e admin admin] PATH_INFO: `/v3/projects/9c60d0e3692444e39346d2c296c32ac2/users/e4772d35d57f478d929eea70fd11e147/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:17.109503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcb3a8-e0a5-4965-996a-fb2e4c23e95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c60d0e3692444e39346d2c296c32ac2, user_id=e4772d35d57f478d929eea70fd11e147, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:17.111784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:17.114748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:17.114984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcb3a8-e0a5-4965-996a-fb2e4c23e95e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:17.143266 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2010/4049] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:17 2022] PUT /identity/v3/projects/9c60d0e3692444e39346d2c296c32ac2/users/e4772d35d57f478d929eea70fd11e147/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:17.148830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:17.149178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:17.149440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:17.192549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:17.194261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:17.206528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:17.243518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:17.290454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f338ea-cab0-4746-93b3-c9c38f812575 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:17.365604 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2011/4050] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:17.371188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:17.373833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:17.461999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:17.463195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:17.463672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:17.464005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:17.470446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:17.489351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3e979-8fb0-4478-ae38-79f8118e2816 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:17.498631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9d8f7-2c6b-4ef4-9abd-cd463412d6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:17.499412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9d8f7-2c6b-4ef4-9abd-cd463412d6be service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:17.499793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9d8f7-2c6b-4ef4-9abd-cd463412d6be service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:17.500256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9d8f7-2c6b-4ef4-9abd-cd463412d6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:17.503660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:17.507259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:17.521437 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2040/4051] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:17.522478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9d8f7-2c6b-4ef4-9abd-cd463412d6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:17.522765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:17.526770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9d8f7-2c6b-4ef4-9abd-cd463412d6be service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:17.538265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:17.552873 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2012/4052] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:17.582093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44e536-516d-41f1-8bf3-3d4d74f9a169 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:17.594608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:17.595385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:17.595706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:17.596077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] PATH_INFO: `/v3/users/28f357781a0b4818a8a9c8945f318ffe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:17.597627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f357781a0b4818a8a9c8945f318ffe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:17.618683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:17.720259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f357781a0b4818a8a9c8945f318ffe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:17.721669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f357781a0b4818a8a9c8945f318ffe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:17.749427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f357781a0b4818a8a9c8945f318ffe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:17.762682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f357781a0b4818a8a9c8945f318ffe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:17.766202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15ecff2f-db30-457f-bfdb-39b87e390f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f357781a0b4818a8a9c8945f318ffe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:17.816881 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2041/4053] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:17 2022] DELETE /identity/v3/users/28f357781a0b4818a8a9c8945f318ffe => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:17.973455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f64d2-29b0-4115-826d-d33c57b6545a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:17.974047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f64d2-29b0-4115-826d-d33c57b6545a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:17.974047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f64d2-29b0-4115-826d-d33c57b6545a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.010854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-089f64d2-29b0-4115-826d-d33c57b6545a None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:18.015259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-089f64d2-29b0-4115-826d-d33c57b6545a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:18.042668 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2013/4054] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:17 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:18.046447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:18.046780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:18.047219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.079572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:18.080837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:18.170233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:18.252802 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2042/4055] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:18.254361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f64d2-29b0-4115-826d-d33c57b6545a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:18.271257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f64d2-29b0-4115-826d-d33c57b6545a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:18.289919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a6fdd-790b-4019-a23f-c3a50f3aae83 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:18.294458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a6fdd-790b-4019-a23f-c3a50f3aae83 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:18.296216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a6fdd-790b-4019-a23f-c3a50f3aae83 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:18.297009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a6fdd-790b-4019-a23f-c3a50f3aae83 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.354872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:18.356343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:18.471502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d2a6fdd-790b-4019-a23f-c3a50f3aae83 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:18.498717 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2014/4056] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:18 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:18.523705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8cd68-0436-4689-9d02-1741fb0a87e2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:18.539032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:18.540707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:18.541262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:18.542012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] PATH_INFO: `/v3/projects/cb17c2ebff4e4bd9b5844a58cd8677b1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.546108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb17c2ebff4e4bd9b5844a58cd8677b1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:18.548610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:18.622233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb17c2ebff4e4bd9b5844a58cd8677b1', 'request_id': 'req-0178c203-28d4-4900-89dd-6ca2acfa5ea1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:18.634612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb17c2ebff4e4bd9b5844a58cd8677b1', 'request_id': 'req-0178c203-28d4-4900-89dd-6ca2acfa5ea1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:18.643072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb17c2ebff4e4bd9b5844a58cd8677b1', 'request_id': 'req-0178c203-28d4-4900-89dd-6ca2acfa5ea1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:18.658103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] The token cache is being invalidate because project cb17c2ebff4e4bd9b5844a58cd8677b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:18.659799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:18.660733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0178c203-28d4-4900-89dd-6ca2acfa5ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:18.662898 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2043/4057] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:18 2022] DELETE /identity/v3/projects/cb17c2ebff4e4bd9b5844a58cd8677b1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:18.667802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac33f-86c8-4d54-a1a0-8df1b8fe66ad None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:18.668144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac33f-86c8-4d54-a1a0-8df1b8fe66ad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:18.668483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac33f-86c8-4d54-a1a0-8df1b8fe66ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.688832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-27bac33f-86c8-4d54-a1a0-8df1b8fe66ad None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:18.690301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27bac33f-86c8-4d54-a1a0-8df1b8fe66ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:18.758859 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2015/4058] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:18.766690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769da198-7a5a-4a3c-a5db-d0fb9ab4d410 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:18.766975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769da198-7a5a-4a3c-a5db-d0fb9ab4d410 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:18.767264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769da198-7a5a-4a3c-a5db-d0fb9ab4d410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.792491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-769da198-7a5a-4a3c-a5db-d0fb9ab4d410 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:18.793895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-769da198-7a5a-4a3c-a5db-d0fb9ab4d410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:18.803814 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2044/4059] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:18.808269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bac33f-86c8-4d54-a1a0-8df1b8fe66ad None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:18.828319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bac33f-86c8-4d54-a1a0-8df1b8fe66ad None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:18.842386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:18.843222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:18.843851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:18.844191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.845214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:18.845752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:18.850853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:18.878491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2016/4060] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:18.882676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769da198-7a5a-4a3c-a5db-d0fb9ab4d410 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:18.894037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769da198-7a5a-4a3c-a5db-d0fb9ab4d410 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:18.900709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:18.901806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:18.901915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:18.902159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:18.902862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:18.904981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:18.941786 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2045/4061] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:18 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:18.946626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:18.959089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392f6c-911e-46f7-8ee6-cb854000f420 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:18.971529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:19.076561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:19.077662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:19.078669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:19.079021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:19.080309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:19.088905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:19.149787 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2017/4062] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:18 2022] POST /identity/v3/users => generated 282 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:19.151348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:19.166850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdbc2c-6799-44a0-8203-0aa6eb749da5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:19.177588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:19.178056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:19.178438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:19.178624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:19.181085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:19.182841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:19.208605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2046/4063] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:19.212739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:19.226923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046439c9-9cdc-4193-a4d1-3b842581e0ed admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:19.239376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bfaaf-a332-43a7-bb6d-813d7a346382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:19.240113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bfaaf-a332-43a7-bb6d-813d7a346382 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:19.240748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bfaaf-a332-43a7-bb6d-813d7a346382 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:19.241252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bfaaf-a332-43a7-bb6d-813d7a346382 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:19.242618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bfaaf-a332-43a7-bb6d-813d7a346382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:19.246859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bfaaf-a332-43a7-bb6d-813d7a346382 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:19.271414 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2018/4064] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:19.275509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:19.292140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7724b-4a32-4720-924f-fea67c6fa25b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:19.306335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ca4c8-b398-41ac-bf86-1348ae044d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:19.307439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ca4c8-b398-41ac-bf86-1348ae044d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:19.307961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ca4c8-b398-41ac-bf86-1348ae044d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:19.308491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ca4c8-b398-41ac-bf86-1348ae044d0b admin admin] PATH_INFO: `/v3/projects/4dd13c6da37d4d0aa58db41f6722c823/users/a12e7d80aab0420da7046cda0b553a44/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:19.350140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ca4c8-b398-41ac-bf86-1348ae044d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dd13c6da37d4d0aa58db41f6722c823, user_id=a12e7d80aab0420da7046cda0b553a44, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:19.354018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ca4c8-b398-41ac-bf86-1348ae044d0b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:19.385944 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2047/4065] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:19 2022] PUT /identity/v3/projects/4dd13c6da37d4d0aa58db41f6722c823/users/a12e7d80aab0420da7046cda0b553a44/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:19.387385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:19.387674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:19.387950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:19.438263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] MFA Rules not processed for user `a12e7d80aab0420da7046cda0b553a44`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:19.439592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:19.505133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988981b-1d75-45a6-a378-c47d34993098 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:19.505353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988981b-1d75-45a6-a378-c47d34993098 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:19.505631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988981b-1d75-45a6-a378-c47d34993098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:19.526927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:19.539199 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2019/4066] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:19.552363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:19.553846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:19.557583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5988981b-1d75-45a6-a378-c47d34993098 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:19.558902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5988981b-1d75-45a6-a378-c47d34993098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:19.565259 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2048/4067] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:19 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:19.699502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e46d2-c527-4135-877c-f2458706ed11 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:19.705581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f2b2e-4330-47f8-97cc-9bfb219bcc00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:19.706415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f2b2e-4330-47f8-97cc-9bfb219bcc00 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:19.706760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f2b2e-4330-47f8-97cc-9bfb219bcc00 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:19.707072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f2b2e-4330-47f8-97cc-9bfb219bcc00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:19.724352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f2b2e-4330-47f8-97cc-9bfb219bcc00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:19.727651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f2b2e-4330-47f8-97cc-9bfb219bcc00 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:19.759384 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2020/4068] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:20.569496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3333e-2c60-43e2-a596-a3c5c3d7a823 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:20.570279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3333e-2c60-43e2-a596-a3c5c3d7a823 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:20.570279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3333e-2c60-43e2-a596-a3c5c3d7a823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:20.616032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-08e3333e-2c60-43e2-a596-a3c5c3d7a823 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:20.618206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08e3333e-2c60-43e2-a596-a3c5c3d7a823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:20.698503 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2049/4069] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:20.699748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d036-eefb-4f2c-b918-fbeadecd6cac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:20.700209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d036-eefb-4f2c-b918-fbeadecd6cac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:20.700402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d036-eefb-4f2c-b918-fbeadecd6cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:20.738916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4340d036-eefb-4f2c-b918-fbeadecd6cac None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:20.740128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4340d036-eefb-4f2c-b918-fbeadecd6cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:20.756011 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2021/4070] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:20 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:20.765924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3f632-3342-4bc7-9561-a4962702f809 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:20.766254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3f632-3342-4bc7-9561-a4962702f809 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:20.766820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3f632-3342-4bc7-9561-a4962702f809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:20.860471 ncdv-master-1 devstack@keystone.service[104132]: WARNING keystone.server.flask.application [None req-3aa3f632-3342-4bc7-9561-a4962702f809 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 27 10:11:20.863603 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2050/4071] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:20 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 100 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 27 10:11:20.868926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e4e3b-2c4e-4779-8b7d-06f6085c81ac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:20.869460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e4e3b-2c4e-4779-8b7d-06f6085c81ac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:20.869893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e4e3b-2c4e-4779-8b7d-06f6085c81ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:20.905267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:20.931765 ncdv-master-1 devstack@keystone.service[104131]: WARNING keystone.server.flask.application [None req-5a3e4e3b-2c4e-4779-8b7d-06f6085c81ac None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 27 10:11:20.939419 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2022/4072] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:20 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 72 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 27 10:11:20.940829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc594-5dda-484c-860a-79dac03163ca None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:20.941189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc594-5dda-484c-860a-79dac03163ca None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:20.941535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc594-5dda-484c-860a-79dac03163ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:20.971707 ncdv-master-1 devstack@keystone.service[104132]: WARNING keystone.server.flask.application [None req-118dc594-5dda-484c-860a-79dac03163ca None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 27 10:11:20.986671 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2051/4073] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:20 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 47 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 27 10:11:25.979216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5ecf4-baa1-4cd6-b6f4-9880298c8df9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:25.980163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5ecf4-baa1-4cd6-b6f4-9880298c8df9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:25.980448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5ecf4-baa1-4cd6-b6f4-9880298c8df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:26.034447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8ad5ecf4-baa1-4cd6-b6f4-9880298c8df9 None None] MFA Rules not processed for user `e4772d35d57f478d929eea70fd11e147`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:26.035716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8ad5ecf4-baa1-4cd6-b6f4-9880298c8df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:26.172708 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2023/4074] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:26.176387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc594-5dda-484c-860a-79dac03163ca None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:26.179378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-118dc594-5dda-484c-860a-79dac03163ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:26.316535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc594-5dda-484c-860a-79dac03163ca None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:26.326979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:26.328068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:26.328519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:26.329021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:26.329895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:26.332231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:26.392750 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2052/4075] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:26 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:26.400730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5ecf4-baa1-4cd6-b6f4-9880298c8df9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:26.414224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5ecf4-baa1-4cd6-b6f4-9880298c8df9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:26.503107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:26.503861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:26.504209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:26.504847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:26.505575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:26.508555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:26.565421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2024/4076] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:26 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:26.571105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:26.592486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7b8ca-29db-429c-bfcf-2179b037eb2b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:26.602584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:26.604586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:26.607115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:26.608307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:26.611206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:26.617413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:26.635330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2053/4077] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:26.636056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:26.650410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9a70b-9cd9-4ebc-bc71-fc1cb0625692 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:26.659774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:26.660507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:26.660867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:26.661146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] PATH_INFO: `/v3/projects/f8c8e28396c5494c9d0e82a3bec39844/users/79ad6239ed5d4b0d91b35d961ec79f50/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:26.703596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c8e28396c5494c9d0e82a3bec39844, user_id=79ad6239ed5d4b0d91b35d961ec79f50, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:26.706405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:26.743521 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2025/4078] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:26 2022] PUT /identity/v3/projects/f8c8e28396c5494c9d0e82a3bec39844/users/79ad6239ed5d4b0d91b35d961ec79f50/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:26.747343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:26.762244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8264b-0ca8-461c-8226-05e7edf62c2d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:26.859244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:26.896237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:26.897307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:26.897818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:26.898594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:26.898594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:26.901513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:26.920692 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2054/4079] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:26.924921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:26.938538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc979d82-75dd-4219-aa29-069380f23c58 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:26.945162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334c12-7622-442a-88d9-78a33d120c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:26.946525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334c12-7622-442a-88d9-78a33d120c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:26.946525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334c12-7622-442a-88d9-78a33d120c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:26.946940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334c12-7622-442a-88d9-78a33d120c17 admin admin] PATH_INFO: `/v3/projects/f8c8e28396c5494c9d0e82a3bec39844/users/79ad6239ed5d4b0d91b35d961ec79f50/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:26.949709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87334c12-7622-442a-88d9-78a33d120c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c8e28396c5494c9d0e82a3bec39844, user_id=79ad6239ed5d4b0d91b35d961ec79f50, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:26.954117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87334c12-7622-442a-88d9-78a33d120c17 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:26.968704 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2026/4080] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:26 2022] PUT /identity/v3/projects/f8c8e28396c5494c9d0e82a3bec39844/users/79ad6239ed5d4b0d91b35d961ec79f50/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:30.295156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:30.315180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8dbaf-b19c-4209-9ee9-308f3480a80f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:30.493982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:30.494920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:30.495857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:30.496451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] PATH_INFO: `/v3/users/a12e7d80aab0420da7046cda0b553a44` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:30.498501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a12e7d80aab0420da7046cda0b553a44)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:30.504453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:30.635562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a12e7d80aab0420da7046cda0b553a44'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:30.636237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a12e7d80aab0420da7046cda0b553a44'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:30.663896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a12e7d80aab0420da7046cda0b553a44'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:30.674788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a12e7d80aab0420da7046cda0b553a44'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:30.675841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a12e7d80aab0420da7046cda0b553a44'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:30.712522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2055/4081] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:30 2022] DELETE /identity/v3/users/a12e7d80aab0420da7046cda0b553a44 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:30.717669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7118359-dd66-4ca3-bee4-f9cace54efa2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:30.718195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7118359-dd66-4ca3-bee4-f9cace54efa2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:30.718464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7118359-dd66-4ca3-bee4-f9cace54efa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:30.751067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b7118359-dd66-4ca3-bee4-f9cace54efa2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:30.753097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b7118359-dd66-4ca3-bee4-f9cace54efa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:30.801599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:31.115020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2027/4082] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:31.233519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:31.235195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:31.386108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24922a-d7be-4786-8b39-e1ddf612f7d4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:31.394231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc4c5f-224c-4bb3-ab46-d8ad9c6a06ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:31.397032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4c5f-224c-4bb3-ab46-d8ad9c6a06ae service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:31.397384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4c5f-224c-4bb3-ab46-d8ad9c6a06ae service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:31.400843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4c5f-224c-4bb3-ab46-d8ad9c6a06ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:31.421997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc4c5f-224c-4bb3-ab46-d8ad9c6a06ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:31.423602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc4c5f-224c-4bb3-ab46-d8ad9c6a06ae service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:31.455204 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2056/4083] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:31.548880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7118359-dd66-4ca3-bee4-f9cace54efa2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:31.562475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7118359-dd66-4ca3-bee4-f9cace54efa2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:31.576406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:31.578246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:31.578646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:31.583451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:31.589143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:31.757442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:31.761174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:31.782709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2028/4084] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:32.002897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:32.006736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:32.006736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:32.013066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:32.070093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:32.071911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:32.076602 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2057/4085] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:32 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:32.081705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:32.083432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:32.228719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1bdec-b1e2-4213-87c3-3002b8ee1993 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:32.235700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17e5d3-99b9-4441-b345-a2d77ef9bf6d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:32.236374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17e5d3-99b9-4441-b345-a2d77ef9bf6d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:32.236719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17e5d3-99b9-4441-b345-a2d77ef9bf6d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:32.237004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17e5d3-99b9-4441-b345-a2d77ef9bf6d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:32.314554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9f17e5d3-99b9-4441-b345-a2d77ef9bf6d tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:32.341623 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2029/4086] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:32 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:32.635580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:32.637914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:32.662871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6a89b-bd52-4690-94f7-43286fdf1b39 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:32.669365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:32.670557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:32.671112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:32.671728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] PATH_INFO: `/v3/projects/4dd13c6da37d4d0aa58db41f6722c823` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:32.673700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dd13c6da37d4d0aa58db41f6722c823)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:32.676082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:32.786283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dd13c6da37d4d0aa58db41f6722c823', 'request_id': 'req-e539dd16-3d64-4d10-a9d6-52e89736ff43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:32.798634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dd13c6da37d4d0aa58db41f6722c823', 'request_id': 'req-e539dd16-3d64-4d10-a9d6-52e89736ff43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:32.852426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dd13c6da37d4d0aa58db41f6722c823', 'request_id': 'req-e539dd16-3d64-4d10-a9d6-52e89736ff43'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:32.870931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] The token cache is being invalidate because project 4dd13c6da37d4d0aa58db41f6722c823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:32.871369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:32.872821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e539dd16-3d64-4d10-a9d6-52e89736ff43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:32.882756 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2058/4087] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:32 2022] DELETE /identity/v3/projects/4dd13c6da37d4d0aa58db41f6722c823 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:32.885027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d2ec-7dc3-4897-b699-df44037aac74 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:32.886129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d2ec-7dc3-4897-b699-df44037aac74 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:32.886640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d2ec-7dc3-4897-b699-df44037aac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:32.921790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e2b5d2ec-7dc3-4897-b699-df44037aac74 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:32.923429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e2b5d2ec-7dc3-4897-b699-df44037aac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:32.991690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:33.052913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2030/4088] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:33.064200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd94020-00e5-4d6b-b095-38bbd5e36406 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.065075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd94020-00e5-4d6b-b095-38bbd5e36406 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.065279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd94020-00e5-4d6b-b095-38bbd5e36406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.120019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9bd94020-00e5-4d6b-b095-38bbd5e36406 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:33.122088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9bd94020-00e5-4d6b-b095-38bbd5e36406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:33.138229 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2059/4089] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:33.138992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d2ec-7dc3-4897-b699-df44037aac74 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:33.155733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d2ec-7dc3-4897-b699-df44037aac74 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:33.172787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:33.173436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.173710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.173979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.174667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:33.174942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:33.177572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:33.215492 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2031/4090] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:11:33.215603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd94020-00e5-4d6b-b095-38bbd5e36406 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:33.226875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd94020-00e5-4d6b-b095-38bbd5e36406 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:33.239078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1cad8-0551-4a01-bee4-0d3a99f5631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:33.239709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1cad8-0551-4a01-bee4-0d3a99f5631f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.240058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1cad8-0551-4a01-bee4-0d3a99f5631f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.240229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1cad8-0551-4a01-bee4-0d3a99f5631f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.240997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1cad8-0551-4a01-bee4-0d3a99f5631f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:33.243300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1cad8-0551-4a01-bee4-0d3a99f5631f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:33.317937 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2060/4091] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:33 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:33.322523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:33.333822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3e3d9-4262-4620-b1c5-95ea6ddb91da admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:33.345464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9676ac-8180-4caa-9cc9-e5861ac9a1b6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.345796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9676ac-8180-4caa-9cc9-e5861ac9a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.346018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9676ac-8180-4caa-9cc9-e5861ac9a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.390676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9e9676ac-8180-4caa-9cc9-e5861ac9a1b6 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:33.395782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9e9676ac-8180-4caa-9cc9-e5861ac9a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:33.407593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2061/4092] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:33 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 10:11:33.409416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042cd3-ebdd-406a-98ce-945abe0a9555 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.409765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042cd3-ebdd-406a-98ce-945abe0a9555 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.410057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042cd3-ebdd-406a-98ce-945abe0a9555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.421326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e6042cd3-ebdd-406a-98ce-945abe0a9555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:33.467941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:33.468620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.469070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.469556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.470257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:33.474787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:33.533407 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2032/4093] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:33 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:33.534406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:33.551403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773988f-50f7-420d-a121-b1a3379eb91c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:33.557632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:33.558395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.558678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.558959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.559605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:33.561265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:33.580870 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2033/4094] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:33.582411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:33.594186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eac83-2a80-4d42-9559-7d184927f582 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:33.601654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:33.604063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.604510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.604829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.605634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:33.612261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:33.643188 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2034/4095] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:33.644232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-e6042cd3-ebdd-406a-98ce-945abe0a9555 None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Sep 27 10:11:33.649049 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:11:33.650689 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:33.651804 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2062/4096] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:33 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 10:11:33.654846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b25bbd-d510-4882-a146-b0fe8e90ff87 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.655079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b25bbd-d510-4882-a146-b0fe8e90ff87 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.655355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b25bbd-d510-4882-a146-b0fe8e90ff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.672477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6fcf-d7d7-4c00-b578-10feba2500da admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:33.682631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5864f71-34f1-4373-a8d3-0c7c5a34c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:33.683301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5864f71-34f1-4373-a8d3-0c7c5a34c224 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.683577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5864f71-34f1-4373-a8d3-0c7c5a34c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.686733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5864f71-34f1-4373-a8d3-0c7c5a34c224 admin admin] PATH_INFO: `/v3/projects/c1df43105f914ce494a192a957ee5015/users/8c3505534e054c2980973ac46bf5b078/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.730832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5864f71-34f1-4373-a8d3-0c7c5a34c224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1df43105f914ce494a192a957ee5015, user_id=8c3505534e054c2980973ac46bf5b078, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:33.744204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5864f71-34f1-4373-a8d3-0c7c5a34c224 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:33.771176 ncdv-master-1 devstack@keystone.service[104132]: WARNING keystone.server.flask.application [None req-11b25bbd-d510-4882-a146-b0fe8e90ff87 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 27 10:11:33.775046 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2063/4097] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:33 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 121 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 27 10:11:33.777229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b25bbd-d510-4882-a146-b0fe8e90ff87 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:33.789004 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2035/4098] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:33 2022] PUT /identity/v3/projects/c1df43105f914ce494a192a957ee5015/users/8c3505534e054c2980973ac46bf5b078/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:33.789394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-11b25bbd-d510-4882-a146-b0fe8e90ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:33.791293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8047b-ac0c-4298-ab69-09b20affd013 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:33.791572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8047b-ac0c-4298-ab69-09b20affd013 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:33.791915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8047b-ac0c-4298-ab69-09b20affd013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:33.879140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:UPDATE=2,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:33.923646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d4f8047b-ac0c-4298-ab69-09b20affd013 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:33.924899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d4f8047b-ac0c-4298-ab69-09b20affd013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:34.006116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b25bbd-d510-4882-a146-b0fe8e90ff87 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:34.021863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:34.023746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:34.024293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:34.024293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:34.031042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:34.119628 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2036/4099] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:34.121086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843215-43e8-4d8d-896b-6d714806ad6f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:34.121374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843215-43e8-4d8d-896b-6d714806ad6f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:34.121650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843215-43e8-4d8d-896b-6d714806ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:34.138363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:34.162285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-79843215-43e8-4d8d-896b-6d714806ad6f None None] MFA Rules not processed for user `8c3505534e054c2980973ac46bf5b078`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:34.165572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79843215-43e8-4d8d-896b-6d714806ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:34.234505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:34.236223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:34.279701 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2064/4100] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:34.281024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:34.301309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eef0c1-63c6-4c44-96cc-ea54409c7495 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:34.309092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb3b6e-14d4-4d47-a4d9-d792510bd232 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:34.309858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb3b6e-14d4-4d47-a4d9-d792510bd232 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:34.310141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb3b6e-14d4-4d47-a4d9-d792510bd232 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:34.310410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb3b6e-14d4-4d47-a4d9-d792510bd232 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:34.335081 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2037/4101] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:34.340312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843215-43e8-4d8d-896b-6d714806ad6f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:34.542500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843215-43e8-4d8d-896b-6d714806ad6f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:34.561227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:34.561934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:34.562210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:34.562477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:34.563123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:34.565111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:34.670698 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2038/4102] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:34 2022] POST /identity/v3/projects => generated 387 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:34.678971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:34.808829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-00cb3b6e-14d4-4d47-a4d9-d792510bd232 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:34.831424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b682b7-5a4b-4f01-ad99-132b14bc8e72 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:34.843356 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2065/4103] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:34 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:34.843920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:34.844560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:34.844875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:34.845143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:34.845812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:34.847783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:34.906650 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2039/4104] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:34 2022] POST /identity/v3/users => generated 300 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:34.927847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb3b6e-14d4-4d47-a4d9-d792510bd232 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:34.952091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb3b6e-14d4-4d47-a4d9-d792510bd232 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:34.965788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:34.966898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:34.967773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:34.968676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:34.970070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:34.975027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:35.022367 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2066/4105] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:35.027193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:35.051865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1475a66-c787-44ca-b4e0-84090e20322a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:35.072273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:35.073192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.074577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.074866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] PATH_INFO: `/v3/projects/ee7b5627c16547f1b6055c43ababb050/users/b461d68e8ee845cba5e8903597e9e7f0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.109330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:35.125613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7b5627c16547f1b6055c43ababb050, user_id=b461d68e8ee845cba5e8903597e9e7f0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:35.134160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:35.185550 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2040/4106] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:35 2022] PUT /identity/v3/projects/ee7b5627c16547f1b6055c43ababb050/users/b461d68e8ee845cba5e8903597e9e7f0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:35.189416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:35.205582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534906bf-0f2a-4b6d-8f71-d2edb718aadd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:35.369679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69eac5e-8fb5-4d23-b088-e35fefc15f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:35.370649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69eac5e-8fb5-4d23-b088-e35fefc15f14 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.371104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69eac5e-8fb5-4d23-b088-e35fefc15f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.371548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69eac5e-8fb5-4d23-b088-e35fefc15f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.373651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69eac5e-8fb5-4d23-b088-e35fefc15f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:35.376918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69eac5e-8fb5-4d23-b088-e35fefc15f14 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:35.393346 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2067/4107] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:35.394509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:35.414176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e4ace-8ded-4314-85b6-ba45dc4a0088 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:35.431032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:35.431814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.431998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.432271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] PATH_INFO: `/v3/projects/ee7b5627c16547f1b6055c43ababb050/users/b461d68e8ee845cba5e8903597e9e7f0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.440919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7b5627c16547f1b6055c43ababb050, user_id=b461d68e8ee845cba5e8903597e9e7f0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:35.443810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:35.480627 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2041/4108] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:35 2022] PUT /identity/v3/projects/ee7b5627c16547f1b6055c43ababb050/users/b461d68e8ee845cba5e8903597e9e7f0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:35.483760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbc10-dd47-401b-acfa-4acdffce6b0a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.483760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbc10-dd47-401b-acfa-4acdffce6b0a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.483760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbc10-dd47-401b-acfa-4acdffce6b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.527804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:35.560054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7f2fbc10-dd47-401b-acfa-4acdffce6b0a None None] MFA Rules not processed for user `b461d68e8ee845cba5e8903597e9e7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:35.561787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7f2fbc10-dd47-401b-acfa-4acdffce6b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:35.749261 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2068/4109] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:35.754164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:35.770180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238f488-3fa3-4095-bca2-7471d52bc6c4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:35.788736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:35.795875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.796408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.797202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.798132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:35.799557 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:35.799557 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:11:35.799557 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:11:35.801869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:35.844929 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2042/4110] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:35 2022] POST /identity/v3/services => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:35.849748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8648fd4-a405-467a-9614-305c4cc43564 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.850253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8648fd4-a405-467a-9614-305c4cc43564 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.850590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8648fd4-a405-467a-9614-305c4cc43564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.882441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:35.907120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b8648fd4-a405-467a-9614-305c4cc43564 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:35.908496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8648fd4-a405-467a-9614-305c4cc43564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:35.929125 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2069/4111] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:35 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:35.934060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b9ddf-9df1-4103-b35b-cbc63c8c3b8a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.934890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b9ddf-9df1-4103-b35b-cbc63c8c3b8a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.935347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b9ddf-9df1-4103-b35b-cbc63c8c3b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.937713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fd1e1-c176-4326-851e-d81c93ef635a tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:35.956624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:35.957302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:35.957588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:35.957851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:35.958997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:35.962164 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:35.962164 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:11:35.962164 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:11:35.966989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-905b9ddf-9df1-4103-b35b-cbc63c8c3b8a None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:35.967809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:35.969842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-905b9ddf-9df1-4103-b35b-cbc63c8c3b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: On instance['enabled']: Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: On instance['enabled']: Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: 'False' Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise error Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application 'False' Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:35.993963 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:11:35.996070 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application 'False' Sep 27 10:11:35.997520 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.001693 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2043/4112] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:11:35 2022] POST /identity/v3/endpoints => generated 283 bytes in 120 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:11:36.002772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.023823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ded35-1f16-426c-af98-9e0c5a369bbb tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.037353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.038181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.038337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.038575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.039265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.043354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: On instance['enabled']: Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: On instance['enabled']: Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: 'True' Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise error Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application 'True' Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.062001 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:11:36.063755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application 'True' Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:36.065064 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2044/4113] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:11:36 2022] POST /identity/v3/endpoints => generated 281 bytes in 61 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:11:36.071019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.090333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962f039-bc44-43f9-bd49-c9aa33aa1df8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.104605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.105320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.105599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.105878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.106567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.108255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.109936 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2070/4114] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:35 2022] POST /identity/v3/auth/tokens => generated 3383 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:36.119681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b9ddf-9df1-4103-b35b-cbc63c8c3b8a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.134836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b9ddf-9df1-4103-b35b-cbc63c8c3b8a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.141157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9916e-7741-405e-9401-bf6eb3f1fc69 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.142224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9916e-7741-405e-9401-bf6eb3f1fc69 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.142780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9916e-7741-405e-9401-bf6eb3f1fc69 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.143552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9916e-7741-405e-9401-bf6eb3f1fc69 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.183708 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1450866499. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:11:36.183708 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:11:36.183708 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:11:36.226816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2045/4115] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:11:36 2022] POST /identity/v3/endpoints => generated 361 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:11:36.228368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.236070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-80c9916e-7741-405e-9401-bf6eb3f1fc69 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:36.257587 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2071/4116] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:36 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:36.262007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17639-cd57-403f-b3d4-46274ebd2d02 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.268033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.269250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.269641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.270015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/regions/tempest-region-1450866499` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.271165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1450866499)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.272463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.276376 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2046/4117] 10.222.0.26 () {58 vars in 1206 bytes} [Tue Sep 27 10:11:36 2022] GET /identity/v3/regions/tempest-region-1450866499 => generated 180 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:36.280455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9916e-7741-405e-9401-bf6eb3f1fc69 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.302261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9916e-7741-405e-9401-bf6eb3f1fc69 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.345984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.346777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.347161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.347566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints/0d41e6ce20494413badac2c823bd2518` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.357075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=0d41e6ce20494413badac2c823bd2518)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.357975 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:36.357975 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:11:36.357975 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:11:36.359181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: On instance['enabled']: Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: On instance['enabled']: Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: 'False' Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise error Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application 'False' Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:36.380395 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:11:36.382464 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application 'False' Sep 27 10:11:36.383811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:36.388753 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2072/4118] 10.222.0.26 () {60 vars in 1255 bytes} [Tue Sep 27 10:11:36 2022] PATCH /identity/v3/endpoints/0d41e6ce20494413badac2c823bd2518 => generated 283 bytes in 106 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 27 10:11:36.390090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.407914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14ba4b-0972-4599-8fbb-ebd358540140 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.422319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.423435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.423897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.424609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints/0d41e6ce20494413badac2c823bd2518` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.427046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=0d41e6ce20494413badac2c823bd2518)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.428364 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:36.428364 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:11:36.428364 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:11:36.429948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.456020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '0d41e6ce20494413badac2c823bd2518', 'request_id': 'req-9e8fa7d4-2787-4422-9e6f-085a69b52583'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:36.485917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2047/4119] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:36 2022] DELETE /identity/v3/endpoints/0d41e6ce20494413badac2c823bd2518 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:36.490252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.510875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4171d-9aa1-428e-ae62-d597393cf966 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.552771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.554905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.556885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.561810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/regions/tempest-region-1450866499` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.564456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1450866499)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.568338 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:36.568338 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:11:36.568338 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:11:36.573731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.624576 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1450866499. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:11:36.624576 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:11:36.624576 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:11:36.636274 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2073/4120] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 10:11:36 2022] DELETE /identity/v3/regions/tempest-region-1450866499 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:36.643453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.665125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa7d4-2787-4422-9e6f-085a69b52583 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.714345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.714975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.715260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.715541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.721174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.722827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.769384 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-200034371. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:11:36.769384 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:11:36.769384 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:11:36.826030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2048/4121] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:11:36 2022] POST /identity/v3/endpoints => generated 359 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:36.832359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.847053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0262ad3-d43d-4509-a743-91a54b4b57e3 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:36.879726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:36.931469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:36.932171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:36.932458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:36.932763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/regions/tempest-region-200034371` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:36.936237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-200034371)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:36.945924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:36.960193 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2074/4122] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 10:11:36 2022] GET /identity/v3/regions/tempest-region-200034371 => generated 178 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:36.961482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:36.964234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:36.973330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:37.007567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ccf5e-6285-42be-9685-3d642aef231f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:37.040439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:37.041248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.041547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.041808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints/585b20368f384e5bbf4db7904783ac78` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.063371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=585b20368f384e5bbf4db7904783ac78)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:37.065741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: On instance['enabled']: Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: On instance['enabled']: Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: 'True' Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise error Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application 'True' Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:37.079409 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:11:37.081510 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['enabled']: Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application 'True' Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:11:37.083980 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2075/4123] 10.222.0.26 () {60 vars in 1255 bytes} [Tue Sep 27 10:11:36 2022] PATCH /identity/v3/endpoints/585b20368f384e5bbf4db7904783ac78 => generated 281 bytes in 119 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:11:37.089965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:37.116041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ced9-540a-4e44-a337-24b6a88134e4 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:37.125123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=7,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:37.137577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:37.138425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.138775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.141564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/endpoints/585b20368f384e5bbf4db7904783ac78` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.144064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=585b20368f384e5bbf4db7904783ac78)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:37.145151 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:37.145151 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:11:37.145151 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:11:37.147584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:37.175208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '585b20368f384e5bbf4db7904783ac78', 'request_id': 'req-ea030a29-0d15-4202-a2b7-638f46ca555f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:37.201165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2076/4124] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:37 2022] DELETE /identity/v3/endpoints/585b20368f384e5bbf4db7904783ac78 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:37.202985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:37.214934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3cb36-07f9-41ed-997f-707a530acca8 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:37.249590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:37.250646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.251011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.251417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] PATH_INFO: `/v3/users/a83af7f06aa94823989c466abc0ffa3e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.253100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83af7f06aa94823989c466abc0ffa3e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:37.256216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:37.298699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea030a29-0d15-4202-a2b7-638f46ca555f tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:37.323680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354b045-8980-4b12-a648-a2e0e8485384 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:37.324353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354b045-8980-4b12-a648-a2e0e8485384 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.324996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354b045-8980-4b12-a648-a2e0e8485384 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.325217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354b045-8980-4b12-a648-a2e0e8485384 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/regions/tempest-region-200034371` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.325810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354b045-8980-4b12-a648-a2e0e8485384 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-200034371)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:37.327433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354b045-8980-4b12-a648-a2e0e8485384 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:37.365532 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-a354b045-8980-4b12-a648-a2e0e8485384 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-200034371. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:11:37.365532 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:11:37.365532 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:11:37.369371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83af7f06aa94823989c466abc0ffa3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:37.372016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83af7f06aa94823989c466abc0ffa3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:37.385691 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2077/4125] 10.222.0.26 () {58 vars in 1206 bytes} [Tue Sep 27 10:11:37 2022] DELETE /identity/v3/regions/tempest-region-200034371 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:37.385691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44798ae6-d846-4b44-b88f-6b1d0f58e9b1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.385691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44798ae6-d846-4b44-b88f-6b1d0f58e9b1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.385691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44798ae6-d846-4b44-b88f-6b1d0f58e9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.397212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83af7f06aa94823989c466abc0ffa3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:37.419451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83af7f06aa94823989c466abc0ffa3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:37.438579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-44798ae6-d846-4b44-b88f-6b1d0f58e9b1 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:37.442257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44798ae6-d846-4b44-b88f-6b1d0f58e9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:37.443399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83af7f06aa94823989c466abc0ffa3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:37.459999 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2049/4126] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:36 2022] DELETE /identity/v3/users/a83af7f06aa94823989c466abc0ffa3e => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:37.462297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:37.464978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:37.467176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2078/4127] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:37 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:37.470610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.471145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.471145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.546998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:37.548275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:37.611539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3d2d2-9746-432e-8783-3bfdcbee8219 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:37.672183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1f6b8-5256-47fb-b85a-09d501f958c9 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b461d68e8ee845cba5e8903597e9e7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b5627c16547f1b6055c43ababb050', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:37.684870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f6b8-5256-47fb-b85a-09d501f958c9 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.685395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f6b8-5256-47fb-b85a-09d501f958c9 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.685904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f6b8-5256-47fb-b85a-09d501f958c9 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] PATH_INFO: `/v3/services/d040a93345ce4a16966e24d2c494a45a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.687361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1f6b8-5256-47fb-b85a-09d501f958c9 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorizing `identity:delete_service(service_id=d040a93345ce4a16966e24d2c494a45a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:37.688514 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-14d1f6b8-5256-47fb-b85a-09d501f958c9 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:11:37.688514 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:11:37.688514 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:11:37.689761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1f6b8-5256-47fb-b85a-09d501f958c9 tempest-EndpointsNegativeTestJSON-1190533282 tempest-EndpointsNegativeTestJSON-1190533282-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:37.754594 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2050/4128] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:37 2022] DELETE /identity/v3/services/d040a93345ce4a16966e24d2c494a45a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:37.756677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52737766-1588-4ea7-be23-2e48a0f5e020 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:37.757885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52737766-1588-4ea7-be23-2e48a0f5e020 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:37.759221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52737766-1588-4ea7-be23-2e48a0f5e020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:37.848049 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2079/4129] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:37.853195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:37.857860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:37.859689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-52737766-1588-4ea7-be23-2e48a0f5e020 None None] MFA Rules not processed for user `79ad6239ed5d4b0d91b35d961ec79f50`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:37.866711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-52737766-1588-4ea7-be23-2e48a0f5e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:37.891519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:38.017293 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2051/4130] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:38.021987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52737766-1588-4ea7-be23-2e48a0f5e020 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.037099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738ad8-d359-47d8-b2f5-8543dd648cf2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.042151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52737766-1588-4ea7-be23-2e48a0f5e020 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.043001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.043676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.043947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.044211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] PATH_INFO: `/v3/users/8c3505534e054c2980973ac46bf5b078` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.045507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3505534e054c2980973ac46bf5b078)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.046718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd212a58e4ba2431a88cb76f30a245d37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a09593ff74430fa95aace000e41fcf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.047364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.047664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.047936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37/password` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.048144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.070535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:38.100254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.104458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3505534e054c2980973ac46bf5b078'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.105706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3505534e054c2980973ac46bf5b078'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.112123 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2052/4131] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:11:38 2022] POST /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37/password => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:38.113360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.114993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:38.115972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3505534e054c2980973ac46bf5b078'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.126964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3505534e054c2980973ac46bf5b078'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.127763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3505534e054c2980973ac46bf5b078'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.137233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c2778-cdf6-4d79-bd69-de35cc6d1402 tempest-IdentityV3UsersTest-212040216 tempest-IdentityV3UsersTest-212040216-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.143568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.144359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.144489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.144675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] PATH_INFO: `/v3/projects/a74a11e81faa4ca59f7c5662a3e8321f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.145846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74a11e81faa4ca59f7c5662a3e8321f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.149563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.155714 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2080/4132] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:37 2022] DELETE /identity/v3/users/8c3505534e054c2980973ac46bf5b078 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:38.156479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.159174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:38.214603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74a11e81faa4ca59f7c5662a3e8321f', 'request_id': 'req-e24b1638-f933-4c82-bed8-8808b93950c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.224180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74a11e81faa4ca59f7c5662a3e8321f', 'request_id': 'req-e24b1638-f933-4c82-bed8-8808b93950c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.233928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74a11e81faa4ca59f7c5662a3e8321f', 'request_id': 'req-e24b1638-f933-4c82-bed8-8808b93950c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.249332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] The token cache is being invalidate because project a74a11e81faa4ca59f7c5662a3e8321f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:38.249858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.250999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e24b1638-f933-4c82-bed8-8808b93950c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:38.256040 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2053/4133] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:38 2022] DELETE /identity/v3/projects/a74a11e81faa4ca59f7c5662a3e8321f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:38.256040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243a6c6-fdca-413b-a7db-00ac53d902d2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.256040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243a6c6-fdca-413b-a7db-00ac53d902d2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.256040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243a6c6-fdca-413b-a7db-00ac53d902d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.258206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b057b0-3191-4e00-a559-fa3318ae41e1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.278119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1243a6c6-fdca-413b-a7db-00ac53d902d2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:38.279374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1243a6c6-fdca-413b-a7db-00ac53d902d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:38.334167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2ec5d-69c1-437a-b22b-396102d9dfc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.334725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2ec5d-69c1-437a-b22b-396102d9dfc0 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.335247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2ec5d-69c1-437a-b22b-396102d9dfc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.335636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2ec5d-69c1-437a-b22b-396102d9dfc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.338186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8d2ec5d-69c1-437a-b22b-396102d9dfc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:38.338712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=9 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:38.360424 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2054/4134] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:38.381166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.382259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.383527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.407313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:38.408922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:38.425628 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2055/4135] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:38.426825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.434056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:38.447777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2ec5d-69c1-437a-b22b-396102d9dfc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.450614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2ec5d-69c1-437a-b22b-396102d9dfc0 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.455849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:38.489866 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2081/4136] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:38.492006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796acf3-1b84-4bfa-9566-3dfc5b3064b8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.492299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796acf3-1b84-4bfa-9566-3dfc5b3064b8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.492571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796acf3-1b84-4bfa-9566-3dfc5b3064b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.515998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d796acf3-1b84-4bfa-9566-3dfc5b3064b8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:38.517359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d796acf3-1b84-4bfa-9566-3dfc5b3064b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:38.531097 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2082/4137] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:38.536256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796acf3-1b84-4bfa-9566-3dfc5b3064b8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.558412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796acf3-1b84-4bfa-9566-3dfc5b3064b8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.573009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.578038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.578411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.578692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.579391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:38.579680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.581339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.596883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5802c-2319-42d2-808f-3cb69b8fe85e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.612155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.614225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.616625 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2083/4138] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:38.620916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.621384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.621854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.632792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78761e-d629-4e24-b635-12e38ee77d6e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.634791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.636506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.638336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.639164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.639657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.640124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.641228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.644367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.647535 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2056/4139] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:38.687795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.694643 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2084/4140] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:38 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:38.703576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedafe7-2a94-46f4-b38d-943746562630 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.786168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.787501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.787784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.789160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.790751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.796526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.830709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2057/4141] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:38 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:38.835436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.849736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77fed-9b27-4c8b-99a6-fa704cc4431a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.859214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.860109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.860316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.860578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.862177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.863786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.884373 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2085/4142] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:38.884821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.895828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2feb-f537-4205-a784-361feadf5b30 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.903214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.903969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.904247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.904549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:38.905310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:38.906902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:38.930137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:38.933269 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2058/4143] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:38.961993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e91d28-60a1-4c05-a5e6-f0e8ba11fa4f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:38.971208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e37e92-3305-43f3-affe-d725385bafbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:38.971907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e37e92-3305-43f3-affe-d725385bafbe admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:38.972206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e37e92-3305-43f3-affe-d725385bafbe admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:38.972507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e37e92-3305-43f3-affe-d725385bafbe admin admin] PATH_INFO: `/v3/projects/0a1ced5ea0884fc99d7644f37ca3f376/users/015d442ea03c434bb63aa9ba9b537ce8/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:39.014956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e37e92-3305-43f3-affe-d725385bafbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1ced5ea0884fc99d7644f37ca3f376, user_id=015d442ea03c434bb63aa9ba9b537ce8, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:39.017710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e37e92-3305-43f3-affe-d725385bafbe admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:39.039390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:39.042070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:39.083263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d143f-6e2a-4c14-8c42-8009ba8ac35d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:39.097483 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2086/4144] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:38 2022] PUT /identity/v3/projects/0a1ced5ea0884fc99d7644f37ca3f376/users/015d442ea03c434bb63aa9ba9b537ce8/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:39.099559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee7109-42e3-4b12-92e8-0ece6dc2982e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:39.099915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee7109-42e3-4b12-92e8-0ece6dc2982e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:39.100234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee7109-42e3-4b12-92e8-0ece6dc2982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:39.154686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8eee7109-42e3-4b12-92e8-0ece6dc2982e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:39.155967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8eee7109-42e3-4b12-92e8-0ece6dc2982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:39.237163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:39.237761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:39.237761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:39.237988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] PATH_INFO: `/v3/projects/c1df43105f914ce494a192a957ee5015` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:39.241728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1df43105f914ce494a192a957ee5015)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:39.242851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:39.245661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:39.309115 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2087/4145] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:39.310612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:39.311013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:39.311499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:39.337771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1df43105f914ce494a192a957ee5015', 'request_id': 'req-cb51de5e-e363-4671-886f-575c272a121b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.354340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1df43105f914ce494a192a957ee5015', 'request_id': 'req-cb51de5e-e363-4671-886f-575c272a121b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.364864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:39.366399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:39.372418 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2088/4146] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:39 2022] POST /identity/v3/auth/tokens => generated 352 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:39.372418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:39.374550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:39.375133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1df43105f914ce494a192a957ee5015', 'request_id': 'req-cb51de5e-e363-4671-886f-575c272a121b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.392048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] The token cache is being invalidate because project c1df43105f914ce494a192a957ee5015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:39.393033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.396233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.401235 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2059/4147] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:39 2022] DELETE /identity/v3/projects/c1df43105f914ce494a192a957ee5015 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:39.404650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:39.406560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:39.561610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51de5e-e363-4671-886f-575c272a121b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:39.575954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:39.578454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85242339-e60e-4534-92d0-c2b73441cb6f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:39.578773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:39.579154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:39.579530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] PATH_INFO: `/v3/users/b461d68e8ee845cba5e8903597e9e7f0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:39.581759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b461d68e8ee845cba5e8903597e9e7f0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:39.586114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:39.589805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5544083-1fdc-43a6-8a2d-139270a62acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:39.590560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5544083-1fdc-43a6-8a2d-139270a62acc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:39.590940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5544083-1fdc-43a6-8a2d-139270a62acc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:39.591313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5544083-1fdc-43a6-8a2d-139270a62acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:39.595299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c5544083-1fdc-43a6-8a2d-139270a62acc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:39.639059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5544083-1fdc-43a6-8a2d-139270a62acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:39.641128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5544083-1fdc-43a6-8a2d-139270a62acc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:39.660333 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2089/4148] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:39.680171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b461d68e8ee845cba5e8903597e9e7f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.683300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b461d68e8ee845cba5e8903597e9e7f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.694399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b461d68e8ee845cba5e8903597e9e7f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.709508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b461d68e8ee845cba5e8903597e9e7f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.735120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-37f79ecd-e9bc-49fc-b2c3-d614f51a7c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b461d68e8ee845cba5e8903597e9e7f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:39.748713 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2060/4149] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:39 2022] DELETE /identity/v3/users/b461d68e8ee845cba5e8903597e9e7f0 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:40.376046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:40.376542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:40.376635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:40.406973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906388b2-7ac9-4283-bf77-558c117135fd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:40.407582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906388b2-7ac9-4283-bf77-558c117135fd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:40.407582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906388b2-7ac9-4283-bf77-558c117135fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:40.415698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] MFA Rules not processed for user `d212a58e4ba2431a88cb76f30a245d37`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:40.416472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:40.437163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:40.448275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-906388b2-7ac9-4283-bf77-558c117135fd None None] MFA Rules not processed for user `96e48f22b8e7478986f912e64df670b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:40.450046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-906388b2-7ac9-4283-bf77-558c117135fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:40.456396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:40.573407 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2090/4150] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:40.575178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:40.577558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:40.615743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2061/4151] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:40.712326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eca963-699c-4089-aa57-ba9b34c83cbb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:40.713510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eca963-699c-4089-aa57-ba9b34c83cbb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:40.713964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eca963-699c-4089-aa57-ba9b34c83cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:40.762336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-06eca963-699c-4089-aa57-ba9b34c83cbb None None] MFA Rules not processed for user `96e48f22b8e7478986f912e64df670b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:40.763732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06eca963-699c-4089-aa57-ba9b34c83cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:40.769521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6b3a4-a2a0-409e-a834-e0586920a070 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:40.776081 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2062/4152] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:40.787378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:40.788837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:40.789402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:40.789776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] PATH_INFO: `/v3/users/d212a58e4ba2431a88cb76f30a245d37` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:40.799131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d212a58e4ba2431a88cb76f30a245d37)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:40.804686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:40.818576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ac7d-d046-4082-baee-d0bbc28d40da None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:40.818576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ac7d-d046-4082-baee-d0bbc28d40da None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:40.820290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ac7d-d046-4082-baee-d0bbc28d40da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:40.871126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3864ac7d-d046-4082-baee-d0bbc28d40da None None] MFA Rules not processed for user `96e48f22b8e7478986f912e64df670b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:40.872441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3864ac7d-d046-4082-baee-d0bbc28d40da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:40.894543 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2063/4153] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:40.932531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:40.932874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:40.933247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:40.958301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:40.961430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:40.978842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] MFA Rules not processed for user `96e48f22b8e7478986f912e64df670b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:40.981418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:40.989500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:40.995363 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2064/4154] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:40.997390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:40.999704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:41.007295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:41.010935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ef230903-b950-43c8-b3da-91b65cfdd4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd212a58e4ba2431a88cb76f30a245d37'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:41.023609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f1696-cb03-4f6e-8a6a-ca3af1792de3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:41.044817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:41.045571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.045925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.046254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] PATH_INFO: `/v3/projects/ee7b5627c16547f1b6055c43ababb050` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.048017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7b5627c16547f1b6055c43ababb050)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:41.052403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:41.056464 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2091/4155] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:40 2022] DELETE /identity/v3/users/d212a58e4ba2431a88cb76f30a245d37 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:41.058951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caeffc-fe44-4309-a5b2-6a1346a528ed None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.059203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caeffc-fe44-4309-a5b2-6a1346a528ed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.059476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caeffc-fe44-4309-a5b2-6a1346a528ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.083081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e4caeffc-fe44-4309-a5b2-6a1346a528ed None None] MFA Rules not processed for user `96e48f22b8e7478986f912e64df670b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:41.084301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e4caeffc-fe44-4309-a5b2-6a1346a528ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:41.125167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:41.153627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7b5627c16547f1b6055c43ababb050', 'request_id': 'req-38980b12-71a5-4ebd-bc91-50844410aa6f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:41.170457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7b5627c16547f1b6055c43ababb050', 'request_id': 'req-38980b12-71a5-4ebd-bc91-50844410aa6f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:41.185324 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2092/4156] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:41.185324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.185324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.185324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.209053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7b5627c16547f1b6055c43ababb050', 'request_id': 'req-38980b12-71a5-4ebd-bc91-50844410aa6f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:41.227217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] The token cache is being invalidate because project ee7b5627c16547f1b6055c43ababb050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:41.227508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:41.228273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38980b12-71a5-4ebd-bc91-50844410aa6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:41.233342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2065/4157] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:40 2022] DELETE /identity/v3/projects/ee7b5627c16547f1b6055c43ababb050 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:41.244101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:41.244101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:41.246809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72cd0-33fb-45ba-bd9d-05dbcf14adca None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.246809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72cd0-33fb-45ba-bd9d-05dbcf14adca None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.246809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72cd0-33fb-45ba-bd9d-05dbcf14adca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.280865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2bc72cd0-33fb-45ba-bd9d-05dbcf14adca None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:41.282195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2bc72cd0-33fb-45ba-bd9d-05dbcf14adca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:41.342725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2093/4158] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:41.361759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:41.363521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:41.447087 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2066/4159] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:41.448772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97990a90-a1e2-4d69-8e50-9c0e18baccee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.449122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97990a90-a1e2-4d69-8e50-9c0e18baccee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.449456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97990a90-a1e2-4d69-8e50-9c0e18baccee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.488984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-97990a90-a1e2-4d69-8e50-9c0e18baccee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:41.490289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-97990a90-a1e2-4d69-8e50-9c0e18baccee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:41.513263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2067/4160] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:41.514370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97990a90-a1e2-4d69-8e50-9c0e18baccee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:41.517264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a81d42-a2ae-42cc-b4d8-484dd3e9b203 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:41.530316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:41.536708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:41.537764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97990a90-a1e2-4d69-8e50-9c0e18baccee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:41.538836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.539216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.540919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.548533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:41.550565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.553862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.554921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.555378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:41.555748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:41.557635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:41.584169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:41.588338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:41.592021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2068/4161] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:41.596990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:41.618931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae0ad-1221-4863-b20e-c3fecfc13171 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:41.631670 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2094/4162] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:41.632743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:41.633468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.633810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.634135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.634852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:41.636990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:41.702343 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2069/4163] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:41 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:41.706110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:41.722838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c2de1-9ea1-49b1-9be4-f83be51f39a4 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:41.881992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:41.881992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:41.881992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:41.881992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:41.882767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:41.882767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:41.939795 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2095/4164] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:41 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:41.945904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:41.980273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8566165-6c73-4a5d-b380-08e0a8ec64b5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.014722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.015405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.017259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.019983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.023008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.029765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.060377 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2070/4165] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:42.064473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.087567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8c082-48e8-4989-b77e-6049cfd2a784 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.102665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcd093-870c-4972-a908-192c705afb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.106649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcd093-870c-4972-a908-192c705afb9d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.108313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcd093-870c-4972-a908-192c705afb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.109846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcd093-870c-4972-a908-192c705afb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.119248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcd093-870c-4972-a908-192c705afb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.122698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcd093-870c-4972-a908-192c705afb9d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.138948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:42.154583 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2096/4166] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:42.156008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.175266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99692c3-9b36-4d69-a352-b6f9d2ab8d7b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.185355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.186382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.186811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.187206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] PATH_INFO: `/v3/projects/0de900dfcb414d1498fd3a9018c1021c/users/d733cb47ffa74fc0868b15f40cdc1a5e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.227678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de900dfcb414d1498fd3a9018c1021c, user_id=d733cb47ffa74fc0868b15f40cdc1a5e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.233948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.279970 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2071/4167] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:42 2022] PUT /identity/v3/projects/0de900dfcb414d1498fd3a9018c1021c/users/d733cb47ffa74fc0868b15f40cdc1a5e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:42.290483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573f138-9fd8-46a7-ae51-423433f063df None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.290843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573f138-9fd8-46a7-ae51-423433f063df None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.291114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573f138-9fd8-46a7-ae51-423433f063df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.357532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f573f138-9fd8-46a7-ae51-423433f063df None None] MFA Rules not processed for user `d733cb47ffa74fc0868b15f40cdc1a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:42.360631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f573f138-9fd8-46a7-ae51-423433f063df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:42.471431 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2097/4168] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:42.476099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.491455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64be50-9d6a-4377-8705-9529ec19da2b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.503903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.504757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.505015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.505294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.505930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.517888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.560971 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2072/4169] 10.222.0.26 () {60 vars in 1299 bytes} [Tue Sep 27 10:11:42 2022] POST /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2 => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:42.562333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573f138-9fd8-46a7-ae51-423433f063df None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.575949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573f138-9fd8-46a7-ae51-423433f063df None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.588487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.589194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.589468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.589740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/7f1c14527b6042d98b43e68161226bf9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.596405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e, credential_id=7f1c14527b6042d98b43e68161226bf9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.598134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.637605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2098/4170] 10.222.0.26 () {58 vars in 1380 bytes} [Tue Sep 27 10:11:42 2022] DELETE /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/7f1c14527b6042d98b43e68161226bf9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:42.642530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.672580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18677bfe-bbd6-48a3-b1a5-02c241a267c0 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.692681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.693285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.693573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.693851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.702958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.709865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.763150 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2073/4171] 10.222.0.26 () {60 vars in 1299 bytes} [Tue Sep 27 10:11:42 2022] POST /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2 => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:42.764320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.784095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d2fe-44bb-4c5f-ab91-173a2d02bc53 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.789447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.790541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.790541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.790759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/fe2acbd21d68400291f2d60f222aa60d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.798062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e, credential_id=fe2acbd21d68400291f2d60f222aa60d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.799253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.837517 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2099/4172] 10.222.0.26 () {58 vars in 1380 bytes} [Tue Sep 27 10:11:42 2022] DELETE /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/fe2acbd21d68400291f2d60f222aa60d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:42.842427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.855629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adaca0-2ba5-4f70-93bf-971bba2df9d7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.867992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec300-65c3-4a8f-9ea7-5b327df64394 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.868769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec300-65c3-4a8f-9ea7-5b327df64394 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.869341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec300-65c3-4a8f-9ea7-5b327df64394 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.869341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec300-65c3-4a8f-9ea7-5b327df64394 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/fe2acbd21d68400291f2d60f222aa60d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-27aec300-65c3-4a8f-9ea7-5b327df64394 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Could not find credential: 93fd97180cdf9096f125306ff936b63d0ab9847eb7bc46379eefdc666860ea6d.: keystone.exception.CredentialNotFound: Could not find credential: 93fd97180cdf9096f125306ff936b63d0ab9847eb7bc46379eefdc666860ea6d. Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Sep 27 10:11:42.906346 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:11:42.907724 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: 93fd97180cdf9096f125306ff936b63d0ab9847eb7bc46379eefdc666860ea6d. Sep 27 10:11:42.909327 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:11:42.916871 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2074/4173] 10.222.0.26 () {58 vars in 1377 bytes} [Tue Sep 27 10:11:42 2022] GET /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/fe2acbd21d68400291f2d60f222aa60d => generated 148 bytes in 76 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:11:42.917775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:42.931643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663f651-c445-4989-80e5-47cfdae82072 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:42.946114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:42.946877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:42.947247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:42.947580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:42.948211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:42.951429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:42.988621 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2100/4174] 10.222.0.26 () {60 vars in 1299 bytes} [Tue Sep 27 10:11:42 2022] POST /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2 => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:42.993346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec300-65c3-4a8f-9ea7-5b327df64394 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.024643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec300-65c3-4a8f-9ea7-5b327df64394 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.056831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.056831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.056831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.057689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.057689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.057689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.088201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.089335 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2075/4175] 10.222.0.26 () {60 vars in 1299 bytes} [Tue Sep 27 10:11:42 2022] POST /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2 => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:43.099661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0f966-24c0-446a-9d7d-d81238f2751c tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.117955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.118636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.120908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.121207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.121822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=d733cb47ffa74fc0868b15f40cdc1a5e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.124155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.148336 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2101/4176] 10.222.0.26 () {58 vars in 1278 bytes} [Tue Sep 27 10:11:43 2022] GET /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2 => generated 865 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:11:43.152701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.166841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e253-00b9-4357-8a96-0ce7103232a2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.186092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.186775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.187049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.187326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/20d9775c69384885991f4c23196fbb90` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.195047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e, credential_id=20d9775c69384885991f4c23196fbb90)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.196395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.243862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.244358 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2076/4177] 10.222.0.26 () {58 vars in 1380 bytes} [Tue Sep 27 10:11:43 2022] DELETE /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/20d9775c69384885991f4c23196fbb90 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:43.266269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66465e-7fb2-4e30-ad7e-1a26120142b2 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.274266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.274955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.275231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.275499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/26a624ec24aa4d5980eb8a6c53e910bd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.282363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e, credential_id=26a624ec24aa4d5980eb8a6c53e910bd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.283814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.362643 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2102/4178] 10.222.0.26 () {58 vars in 1380 bytes} [Tue Sep 27 10:11:43 2022] DELETE /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/26a624ec24aa4d5980eb8a6c53e910bd => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:43.369843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.388887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a98756-052e-4486-a9ea-ca8ed47ac952 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.405066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.405738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.406018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.406383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.407209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.408517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.470587 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2077/4179] 10.222.0.26 () {60 vars in 1299 bytes} [Tue Sep 27 10:11:43 2022] POST /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2 => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:43.474878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.489243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed769-7048-4d0f-beef-0b2ca74291b7 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.507563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.510252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.512991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.513377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/d7758627721948dda8a12cc22c646690` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.518622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e, credential_id=d7758627721948dda8a12cc22c646690)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.520726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.534010 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2103/4180] 10.222.0.26 () {58 vars in 1377 bytes} [Tue Sep 27 10:11:43 2022] GET /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/d7758627721948dda8a12cc22c646690 => generated 367 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:43.538295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.555497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1db13d-4ef6-4f3b-aec1-9e2606a009b6 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.571384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43abaa-da6e-4f01-a63d-3fc850a5d675 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd733cb47ffa74fc0868b15f40cdc1a5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0de900dfcb414d1498fd3a9018c1021c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.572305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43abaa-da6e-4f01-a63d-3fc850a5d675 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.572678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43abaa-da6e-4f01-a63d-3fc850a5d675 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.573230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43abaa-da6e-4f01-a63d-3fc850a5d675 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/d7758627721948dda8a12cc22c646690` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.577972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43abaa-da6e-4f01-a63d-3fc850a5d675 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=d733cb47ffa74fc0868b15f40cdc1a5e, credential_id=d7758627721948dda8a12cc22c646690)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.581232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43abaa-da6e-4f01-a63d-3fc850a5d675 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.646562 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2078/4181] 10.222.0.26 () {58 vars in 1380 bytes} [Tue Sep 27 10:11:43 2022] DELETE /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e/credentials/OS-EC2/d7758627721948dda8a12cc22c646690 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:43.652222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:43.761281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1222317-f232-46ac-b5b4-ee5198d37e39 tempest-EC2CredentialsTest-40988653 tempest-EC2CredentialsTest-40988653-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:43.767491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:43.769325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:43.769802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:43.770300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] PATH_INFO: `/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:43.772976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d733cb47ffa74fc0868b15f40cdc1a5e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:43.776002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:43.865907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:43.947608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd733cb47ffa74fc0868b15f40cdc1a5e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:43.948914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd733cb47ffa74fc0868b15f40cdc1a5e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:43.974008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd733cb47ffa74fc0868b15f40cdc1a5e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:43.984975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd733cb47ffa74fc0868b15f40cdc1a5e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:43.985774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd733cb47ffa74fc0868b15f40cdc1a5e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:44.027375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2104/4182] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:43 2022] DELETE /identity/v3/users/d733cb47ffa74fc0868b15f40cdc1a5e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:44.031594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:44.032165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:44.032676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:44.058508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:44.060398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:44.170709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2079/4183] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:44.200797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:44.202696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:44.301262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe1cd6-2b79-4f00-8b91-78c5e5ed2ecf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:44.309125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff64e-bb6b-466c-b93f-6dd9a10e8e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:44.309794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff64e-bb6b-466c-b93f-6dd9a10e8e13 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:44.310462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff64e-bb6b-466c-b93f-6dd9a10e8e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:44.310952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff64e-bb6b-466c-b93f-6dd9a10e8e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:44.329523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff64e-bb6b-466c-b93f-6dd9a10e8e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:44.332168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff64e-bb6b-466c-b93f-6dd9a10e8e13 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:44.349102 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2105/4184] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:44.444967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:44.448991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:44.467011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:44.481562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60930573-fa1d-44b7-baa3-1d582d0d6f0d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:44.493906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:44.496419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:44.497095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:44.500898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] PATH_INFO: `/v3/projects/53a09593ff74430fa95aace000e41fcf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:44.503063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a09593ff74430fa95aace000e41fcf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:44.506772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:44.635177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a09593ff74430fa95aace000e41fcf', 'request_id': 'req-82606829-1f5c-4cb9-98bb-7ade35fb73b4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:44.644415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a09593ff74430fa95aace000e41fcf', 'request_id': 'req-82606829-1f5c-4cb9-98bb-7ade35fb73b4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:44.665173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a09593ff74430fa95aace000e41fcf', 'request_id': 'req-82606829-1f5c-4cb9-98bb-7ade35fb73b4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:44.682282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] The token cache is being invalidate because project 53a09593ff74430fa95aace000e41fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:44.682662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:44.684647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-82606829-1f5c-4cb9-98bb-7ade35fb73b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:44.689572 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2080/4185] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:44 2022] DELETE /identity/v3/projects/53a09593ff74430fa95aace000e41fcf => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:44.696227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9419-07ad-418f-b416-a35af15a7a29 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:44.696773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9419-07ad-418f-b416-a35af15a7a29 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:44.696773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9419-07ad-418f-b416-a35af15a7a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:44.745142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-32ad9419-07ad-418f-b416-a35af15a7a29 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:44.746705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-32ad9419-07ad-418f-b416-a35af15a7a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:44.882973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:44.883735 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2106/4186] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:44.884423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:44.884750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:44.918605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:44.920609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:44.949820 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2081/4187] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:44.954295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9419-07ad-418f-b416-a35af15a7a29 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:44.983237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9419-07ad-418f-b416-a35af15a7a29 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:44.989731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:44.992218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:45.007519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d9bf-fe8c-4cc0-b569-30a536c2bdab None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.008903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.010126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.010547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.011008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.017454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.018573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.019011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.019453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] PATH_INFO: `/v3/projects/0de900dfcb414d1498fd3a9018c1021c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.021186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de900dfcb414d1498fd3a9018c1021c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.033950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:45.037570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.039194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.044113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.067705 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2107/4188] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:11:45.072576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:45.091454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc2536-0782-4891-9309-c6bb762aa1a0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.105499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.106682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.108385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.109038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.111406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.120642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.177593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de900dfcb414d1498fd3a9018c1021c', 'request_id': 'req-5b5c05aa-17d9-4c85-b475-5764f673d073'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:45.191222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de900dfcb414d1498fd3a9018c1021c', 'request_id': 'req-5b5c05aa-17d9-4c85-b475-5764f673d073'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:45.204838 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2108/4189] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:45 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:11:45.206911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:45.213482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de900dfcb414d1498fd3a9018c1021c', 'request_id': 'req-5b5c05aa-17d9-4c85-b475-5764f673d073'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:45.219708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:45.235494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] The token cache is being invalidate because project 0de900dfcb414d1498fd3a9018c1021c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:45.235906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:45.236575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b5c05aa-17d9-4c85-b475-5764f673d073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:45.238421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2082/4190] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:44 2022] DELETE /identity/v3/projects/0de900dfcb414d1498fd3a9018c1021c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:45.248096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f911eb1-0550-45f5-940d-151ad8c0dde2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.248371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f911eb1-0550-45f5-940d-151ad8c0dde2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.248699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f911eb1-0550-45f5-940d-151ad8c0dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.286051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8f911eb1-0550-45f5-940d-151ad8c0dde2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:45.289684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f911eb1-0550-45f5-940d-151ad8c0dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:45.393007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1296c-422a-4a4e-9067-0859cd3c3589 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.431627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.434254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.434541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.434821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.435486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.441946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.494336 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2083/4191] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:45.505695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeb9ab-c688-45bb-9e70-2568af7a41f6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.506168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeb9ab-c688-45bb-9e70-2568af7a41f6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.506718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeb9ab-c688-45bb-9e70-2568af7a41f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.516158 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2109/4192] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:45 2022] POST /identity/v3/users => generated 296 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:45.516763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:45.529696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:45.531469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:45.555269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876e55-e860-49fa-a11a-8f4b87195180 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.560597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.561373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.561648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.561785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.562420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.564066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.597085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-20aeb9ab-c688-45bb-9e70-2568af7a41f6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:45.602699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-20aeb9ab-c688-45bb-9e70-2568af7a41f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:45.616361 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2110/4193] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:11:45.617633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:45.649379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2084/4194] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:45.650447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aeb9ab-c688-45bb-9e70-2568af7a41f6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:45.659445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a367be3-d5e5-4b3a-b174-23b17e6e516c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.674745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.675458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.675736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.676009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] PATH_INFO: `/v3/projects/d72ef96e8706461f86a69d57c1f6264c/users/101c9a03329a4102b53b15428e08635d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.679129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aeb9ab-c688-45bb-9e70-2568af7a41f6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.708887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.715212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.716069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.719048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.728348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:45.729586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.733101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.751472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72ef96e8706461f86a69d57c1f6264c, user_id=101c9a03329a4102b53b15428e08635d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.775513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.812923 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2085/4195] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:45.822214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:45.839249 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2111/4196] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:45 2022] PUT /identity/v3/projects/d72ef96e8706461f86a69d57c1f6264c/users/101c9a03329a4102b53b15428e08635d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:45.844614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:45.860227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3ebf5-25e3-4468-a1fd-a55facdfe903 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.955851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105205dd-fdc2-47d5-a740-1c0492d9f092 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:45.964713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:45.965483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:45.965900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:45.966213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:45.967482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:45.969640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:45.988324 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2112/4197] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:45.989376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.002559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0db38-8277-498a-adbc-de7386c47026 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.012078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf360-cf49-4a30-a300-89ff363ea343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.012800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf360-cf49-4a30-a300-89ff363ea343 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.013205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf360-cf49-4a30-a300-89ff363ea343 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.013486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf360-cf49-4a30-a300-89ff363ea343 admin admin] PATH_INFO: `/v3/projects/d72ef96e8706461f86a69d57c1f6264c/users/101c9a03329a4102b53b15428e08635d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.043929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf360-cf49-4a30-a300-89ff363ea343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72ef96e8706461f86a69d57c1f6264c, user_id=101c9a03329a4102b53b15428e08635d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.055938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf360-cf49-4a30-a300-89ff363ea343 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.071904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.072572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.072859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.073130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.073776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.076067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.118266 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2113/4198] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:45 2022] PUT /identity/v3/projects/d72ef96e8706461f86a69d57c1f6264c/users/101c9a03329a4102b53b15428e08635d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:46.122860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5174f-6322-4726-8e9d-cc3ba22c3a17 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.123177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5174f-6322-4726-8e9d-cc3ba22c3a17 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.123443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5174f-6322-4726-8e9d-cc3ba22c3a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.154893 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2086/4199] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:45 2022] POST /identity/v3/projects => generated 383 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:46.158547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.213079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-56d5174f-6322-4726-8e9d-cc3ba22c3a17 None None] MFA Rules not processed for user `101c9a03329a4102b53b15428e08635d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:46.216086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-56d5174f-6322-4726-8e9d-cc3ba22c3a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:46.330300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:46.387163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05888fd7-e73d-4b1f-8252-0acb9199e5ff admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.400392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.401243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.401687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.402138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.402937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.409697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.468108 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2087/4200] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:46 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:46.474074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.499438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b978742-f369-449e-9b6c-d6be074edf17 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.510169 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2114/4201] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:46.520309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.522353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5174f-6322-4726-8e9d-cc3ba22c3a17 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.523730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.523933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.524259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.525183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.528389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.546863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5174f-6322-4726-8e9d-cc3ba22c3a17 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.558076 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2088/4202] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:46.559053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.559053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.559053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.559053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.559884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.561184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.571977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.583763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5cf2e-6fa1-4538-8fa4-beaeb05b53ea admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.597493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.598402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.598594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.598904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.599677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.602117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.640814 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2089/4203] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:46.641086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2115/4204] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:46 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:46.643698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.643875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.669392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed654c-65ec-4818-8081-7342cc5606d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.716067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:46.733584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cedfd-34aa-4ac6-abc4-385b466b1b1e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.739821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.740729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.741050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.741357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.741997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.743967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.746884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f77e0-d873-49c7-ab7d-09a319a41878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.747939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f77e0-d873-49c7-ab7d-09a319a41878 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.748378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f77e0-d873-49c7-ab7d-09a319a41878 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.748962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f77e0-d873-49c7-ab7d-09a319a41878 admin admin] PATH_INFO: `/v3/projects/2d08eecebc044d64a440f68a1740644f/users/19ff37a1115b478b9ffe5eb234e9085d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.781353 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2116/4205] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:46 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:46.785547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.799546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d5fc0-dcd4-45e4-a9dd-91a9847330ac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.802764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f77e0-d873-49c7-ab7d-09a319a41878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d08eecebc044d64a440f68a1740644f, user_id=19ff37a1115b478b9ffe5eb234e9085d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.807552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f77e0-d873-49c7-ab7d-09a319a41878 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.810074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:46.810711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.810993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.811265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.811920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:46.816447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:46.844772 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2090/4206] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:46 2022] PUT /identity/v3/projects/2d08eecebc044d64a440f68a1740644f/users/19ff37a1115b478b9ffe5eb234e9085d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:46.849065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0053e-96c3-483d-9112-547d392529c6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:46.849353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0053e-96c3-483d-9112-547d392529c6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:46.849624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0053e-96c3-483d-9112-547d392529c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:46.860952 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2117/4207] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:11:46.862373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:46.881939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2b801-54f6-43c9-8c3e-86296a549b56 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:46.907794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8be0053e-96c3-483d-9112-547d392529c6 None None] MFA Rules not processed for user `19ff37a1115b478b9ffe5eb234e9085d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:46.909061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8be0053e-96c3-483d-9112-547d392529c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:47.014063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.014723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.015009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.015281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.015928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.017570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.033155 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2091/4208] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:47.037488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0053e-96c3-483d-9112-547d392529c6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.043703 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2118/4209] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:47.045081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.054122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0053e-96c3-483d-9112-547d392529c6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.062500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.064535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.064852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.065401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.066144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.066967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b669db-0509-46b6-a6fe-4db5ba6621b9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.069077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.079711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a66a9-e7f1-415d-897c-a21d1a563874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.080400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a66a9-e7f1-415d-897c-a21d1a563874 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.080744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a66a9-e7f1-415d-897c-a21d1a563874 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.081184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a66a9-e7f1-415d-897c-a21d1a563874 admin admin] PATH_INFO: `/v3/projects/e915a542e0084d9e903a5deadd9ab143/users/77259380daa74e78ab126dfc1fb097ec/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.107605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a66a9-e7f1-415d-897c-a21d1a563874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e915a542e0084d9e903a5deadd9ab143, user_id=77259380daa74e78ab126dfc1fb097ec, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.110591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a66a9-e7f1-415d-897c-a21d1a563874 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.114471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:47.115245 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2092/4210] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:47 2022] POST /identity/v3/projects => generated 379 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:47.120949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.136937 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2119/4211] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:47 2022] PUT /identity/v3/projects/e915a542e0084d9e903a5deadd9ab143/users/77259380daa74e78ab126dfc1fb097ec/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:47.147476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.147763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.148038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.183669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] MFA Rules not processed for user `77259380daa74e78ab126dfc1fb097ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:47.185289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:47.243359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bc19f-bcc4-4f07-af9c-427a990a1b89 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.285335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:47.347470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.350506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.351101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.351517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.366984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.376741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.381353 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2120/4212] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:47.391874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.393566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:47.441377 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2093/4213] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:47 2022] POST /identity/v3/users => generated 296 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:47.445564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.465944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973a78c-a2d3-4fb1-a6c5-769444153dbc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.481904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.482720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.482999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.483363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.484020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.491154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.506542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2094/4214] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:47.511788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.549762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ef4fa-92a1-4c35-a7c2-7b72747a45cd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.558562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.559640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.560009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.560378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] PATH_INFO: `/v3/projects/b1d3133d731746468bea0151cf066205/users/8d6d481970c341b2a887415ade03f029/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.596245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bf00b-7fe5-4fe2-af3f-20aa952d8fe4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.597994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d3133d731746468bea0151cf066205, user_id=8d6d481970c341b2a887415ade03f029, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.601277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.601971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.602243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.602510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.602882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.632182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2095/4215] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:47 2022] PUT /identity/v3/projects/b1d3133d731746468bea0151cf066205/users/8d6d481970c341b2a887415ade03f029/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:47.636156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.762975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.765032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097c35-6de8-4214-8021-32f5f8b07e1e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.768680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.776435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c08d3-ef52-4581-9723-26ce83435c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.777243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c08d3-ef52-4581-9723-26ce83435c0e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.777608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c08d3-ef52-4581-9723-26ce83435c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.778072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c08d3-ef52-4581-9723-26ce83435c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.778856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c08d3-ef52-4581-9723-26ce83435c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.780567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c08d3-ef52-4581-9723-26ce83435c0e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.808968 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2121/4216] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:47.821461 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2096/4217] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:11:47.825718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:47.847764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8228a-ccbe-40e7-b893-5f785457d9d5 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:47.860446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:47.861812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.862462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.865091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] PATH_INFO: `/v3/projects/b1d3133d731746468bea0151cf066205/users/8d6d481970c341b2a887415ade03f029/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:47.868694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d3133d731746468bea0151cf066205, user_id=8d6d481970c341b2a887415ade03f029, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:47.873712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:47.913966 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2122/4218] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:47 2022] PUT /identity/v3/projects/b1d3133d731746468bea0151cf066205/users/8d6d481970c341b2a887415ade03f029/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:47.924853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb0de6-4af3-4090-8032-40014a189359 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:47.925556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb0de6-4af3-4090-8032-40014a189359 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:47.926159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb0de6-4af3-4090-8032-40014a189359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:48.009660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-aeeb0de6-4af3-4090-8032-40014a189359 None None] MFA Rules not processed for user `8d6d481970c341b2a887415ade03f029`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:48.016411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aeeb0de6-4af3-4090-8032-40014a189359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:48.279133 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2097/4219] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:48.314306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:48.318081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:48.357219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:48.484510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab376b-42f9-46f7-9d43-07f581e41527 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:48.496569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e813-de02-40d7-be29-f8221ce89f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:48.496569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e813-de02-40d7-be29-f8221ce89f07 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:48.496569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e813-de02-40d7-be29-f8221ce89f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:48.496569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e813-de02-40d7-be29-f8221ce89f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:48.650373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074e813-de02-40d7-be29-f8221ce89f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:48.651928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074e813-de02-40d7-be29-f8221ce89f07 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:48.679492 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2123/4220] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:49.454744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb0de6-4af3-4090-8032-40014a189359 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:49.482741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:49.649909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb0de6-4af3-4090-8032-40014a189359 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:49.670625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:49.672140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:49.672440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:49.673892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] PATH_INFO: `/v3/users/101c9a03329a4102b53b15428e08635d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:49.676806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101c9a03329a4102b53b15428e08635d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:49.682005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:49.785561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101c9a03329a4102b53b15428e08635d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:49.786303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101c9a03329a4102b53b15428e08635d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:49.799154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101c9a03329a4102b53b15428e08635d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:49.815193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101c9a03329a4102b53b15428e08635d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:49.840520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101c9a03329a4102b53b15428e08635d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:49.885310 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2098/4221] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:49 2022] DELETE /identity/v3/users/101c9a03329a4102b53b15428e08635d => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:49.887436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:49.888777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:49.889237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:49.932324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:49.932779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:50.155016 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2124/4222] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:50.218150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:50.219771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:50.317258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:50.318663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:50.372331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c44eb-15dd-42e8-b9df-7e407f6cbbf9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:50.385126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:50.385923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:50.386181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:50.386522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:50.413440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:50.416222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:50.448401 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2099/4223] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:50.485472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324bb00-9659-400f-8be1-130c442b8aed None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:50.492384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:50.493903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:50.494371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:50.494971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:50.497140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:50.545764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:50.653723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:50.655503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:50.679203 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2125/4224] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:11:52.802228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:52.804558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:52.876142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:52.879050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:52.905874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3ba36-5b68-4ef1-9b88-a5297bb6b302 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:52.912377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:52.924778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:52.926890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:52.927374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:52.929811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] PATH_INFO: `/v3/projects/d72ef96e8706461f86a69d57c1f6264c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:52.936924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72ef96e8706461f86a69d57c1f6264c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:52.941373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:53.082139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99f82-7e24-4fa9-b1e2-b0e37aaa0250 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:53.101796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72ef96e8706461f86a69d57c1f6264c', 'request_id': 'req-936e295d-82f5-49b6-b218-cff259f41ea2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.122524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72ef96e8706461f86a69d57c1f6264c', 'request_id': 'req-936e295d-82f5-49b6-b218-cff259f41ea2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.135955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72ef96e8706461f86a69d57c1f6264c', 'request_id': 'req-936e295d-82f5-49b6-b218-cff259f41ea2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.163366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] The token cache is being invalidate because project d72ef96e8706461f86a69d57c1f6264c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:53.163818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.165267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.168231 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2126/4225] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:52 2022] DELETE /identity/v3/projects/d72ef96e8706461f86a69d57c1f6264c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:53.172626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:53.175857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:53.248829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831f592-50a0-4dd0-a299-1583a7f24829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:53.249995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831f592-50a0-4dd0-a299-1583a7f24829 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:53.250502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831f592-50a0-4dd0-a299-1583a7f24829 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:53.250993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831f592-50a0-4dd0-a299-1583a7f24829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:53.253728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b831f592-50a0-4dd0-a299-1583a7f24829 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:53.464562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e295d-82f5-49b6-b218-cff259f41ea2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:53.484158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:53.484844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:53.485120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:53.489298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] PATH_INFO: `/v3/users/77259380daa74e78ab126dfc1fb097ec` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:53.494275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77259380daa74e78ab126dfc1fb097ec)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:53.498816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:53.518093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831f592-50a0-4dd0-a299-1583a7f24829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:53.523910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831f592-50a0-4dd0-a299-1583a7f24829 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:53.597415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2100/4226] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 796 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:53.622547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:53.641988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77259380daa74e78ab126dfc1fb097ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.644468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77259380daa74e78ab126dfc1fb097ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.672054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77259380daa74e78ab126dfc1fb097ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.691066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77259380daa74e78ab126dfc1fb097ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.695043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77259380daa74e78ab126dfc1fb097ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:53.747317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2127/4227] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:53 2022] DELETE /identity/v3/users/77259380daa74e78ab126dfc1fb097ec => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:54.590844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e36ece-dc65-460a-8732-e82a5625f1d8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:54.591812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e36ece-dc65-460a-8732-e82a5625f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:54.592326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e36ece-dc65-460a-8732-e82a5625f1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:54.658281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-09e36ece-dc65-460a-8732-e82a5625f1d8 None None] MFA Rules not processed for user `015d442ea03c434bb63aa9ba9b537ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:54.662857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-09e36ece-dc65-460a-8732-e82a5625f1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:54.732255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:54.799811 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2101/4228] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:54.815573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:54.817326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:54.938569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d18c-8b16-4b4d-83fe-d8797f40c497 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:54.953613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:54.955265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:54.955745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:54.956180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:54.971964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:54.976841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:54.991855 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2128/4229] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:55.760914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c08d06-1cc9-48aa-8a22-fb2b97430f90 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:55.761496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c08d06-1cc9-48aa-8a22-fb2b97430f90 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:55.761803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c08d06-1cc9-48aa-8a22-fb2b97430f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:55.795754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-50c08d06-1cc9-48aa-8a22-fb2b97430f90 None None] MFA Rules not processed for user `b8d6f6c63fe144959139278ca287e2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:55.797335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50c08d06-1cc9-48aa-8a22-fb2b97430f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:55.898114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:55.899543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:55.976247 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2102/4230] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:56.063336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cb5fe-19f4-476e-8f8f-dc724764b3ba service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:56.070461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:56.071391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:56.071874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:56.073070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] PATH_INFO: `/v3/projects/e915a542e0084d9e903a5deadd9ab143` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:56.075190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e915a542e0084d9e903a5deadd9ab143)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:56.079357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:56.248427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:56.254408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e915a542e0084d9e903a5deadd9ab143', 'request_id': 'req-23a9c7db-6589-4040-864d-52c9aa042e90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:56.277029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e915a542e0084d9e903a5deadd9ab143', 'request_id': 'req-23a9c7db-6589-4040-864d-52c9aa042e90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:56.295322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e915a542e0084d9e903a5deadd9ab143', 'request_id': 'req-23a9c7db-6589-4040-864d-52c9aa042e90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:56.312958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] The token cache is being invalidate because project e915a542e0084d9e903a5deadd9ab143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:11:56.313259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:56.314873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23a9c7db-6589-4040-864d-52c9aa042e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:56.321542 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2129/4231] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:11:55 2022] DELETE /identity/v3/projects/e915a542e0084d9e903a5deadd9ab143 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:11:56.322947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c520-298c-4a7b-8ebd-748bf20115be None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:56.323232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c520-298c-4a7b-8ebd-748bf20115be None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:56.324807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c520-298c-4a7b-8ebd-748bf20115be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:56.354981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-55f7c520-298c-4a7b-8ebd-748bf20115be None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:56.356504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55f7c520-298c-4a7b-8ebd-748bf20115be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:56.501144 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2103/4232] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:56.510278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855e56e-e172-4c20-8c7a-40a8a878b675 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:56.510555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855e56e-e172-4c20-8c7a-40a8a878b675 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:56.510856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855e56e-e172-4c20-8c7a-40a8a878b675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:56.538931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c855e56e-e172-4c20-8c7a-40a8a878b675 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:56.540509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c855e56e-e172-4c20-8c7a-40a8a878b675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:56.552004 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2130/4233] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:56.556575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c520-298c-4a7b-8ebd-748bf20115be None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:56.578568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c520-298c-4a7b-8ebd-748bf20115be None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:56.594644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:56.595595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:56.596826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:56.597284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:56.599672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:11:56.600766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:56.604456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:56.639814 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2104/4234] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:11:56.644333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855e56e-e172-4c20-8c7a-40a8a878b675 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:56.672466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855e56e-e172-4c20-8c7a-40a8a878b675 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:56.690850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:56.693558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:56.694789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:56.696146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:56.697195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:56.714626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:56.824035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:56.826511 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2131/4235] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:11:56 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:56.851921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b4e30-57d1-484c-9c42-7668842b1965 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:57.060985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:57.103331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:57.103963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:57.104497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:57.104900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:57.105642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:57.107627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:57.210098 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2105/4236] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:11:56 2022] POST /identity/v3/users => generated 283 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:11:57.211256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:57.238103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94de4a-f94e-40aa-8099-5dc03dd810e1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:57.247488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:57.250058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:57.251486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:57.252904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:57.257336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:57.260272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:57.281720 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2132/4237] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:57.283624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:57.301322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dd53e-a748-46a8-ba13-9218ef888685 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:57.306893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dfc02-7361-4241-97ca-efb76b57e5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:57.307608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dfc02-7361-4241-97ca-efb76b57e5ed admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:57.308002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dfc02-7361-4241-97ca-efb76b57e5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:57.308360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dfc02-7361-4241-97ca-efb76b57e5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:57.309188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dfc02-7361-4241-97ca-efb76b57e5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:57.311501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dfc02-7361-4241-97ca-efb76b57e5ed admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:57.330404 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2106/4238] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:11:57.335477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:57.350634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f088-1cda-44d0-b8a0-3c4e39c0e03d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:57.365034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:57.369478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:57.370317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:57.370797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] PATH_INFO: `/v3/projects/08e527cf454c4b22b968faa3c149403d/users/0fb4307dfa9845df9b4ddb95c43a48b5/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:57.416555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e527cf454c4b22b968faa3c149403d, user_id=0fb4307dfa9845df9b4ddb95c43a48b5, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:57.422315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:57.473420 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2133/4239] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:11:57 2022] PUT /identity/v3/projects/08e527cf454c4b22b968faa3c149403d/users/0fb4307dfa9845df9b4ddb95c43a48b5/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:57.489786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:57.490150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:57.490793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:57.590583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] MFA Rules not processed for user `0fb4307dfa9845df9b4ddb95c43a48b5`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:57.591978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:57.763590 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2107/4240] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:11:57.777396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:57.781487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:57.939484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4863f-2c99-410d-bf02-6baddead82ab admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:57.951592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:57.953092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:57.953590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:57.953862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:57.993461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:57.994925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:58.057346 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2134/4241] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:58.215934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:58.217596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:58.452604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1093f9d-4a8f-4084-90d1-2227b6e93311 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:58.479001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329db4d-17cf-45af-ac5c-2406996710cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:58.480272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329db4d-17cf-45af-ac5c-2406996710cd service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:58.480937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329db4d-17cf-45af-ac5c-2406996710cd service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:58.481371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329db4d-17cf-45af-ac5c-2406996710cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:58.484744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9329db4d-17cf-45af-ac5c-2406996710cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:58.525527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329db4d-17cf-45af-ac5c-2406996710cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:58.548273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329db4d-17cf-45af-ac5c-2406996710cd service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:58.601099 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2108/4242] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:11:58.920505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:59.164702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f730877-64c7-4599-b313-aa4090c11e28 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:59.179324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:59.180038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:59.180386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:59.180514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] PATH_INFO: `/v3/users/0fb4307dfa9845df9b4ddb95c43a48b5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:59.185647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb4307dfa9845df9b4ddb95c43a48b5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:59.192764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:59.236450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:59.325770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb4307dfa9845df9b4ddb95c43a48b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:59.327121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb4307dfa9845df9b4ddb95c43a48b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:59.350355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb4307dfa9845df9b4ddb95c43a48b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:59.362296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb4307dfa9845df9b4ddb95c43a48b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:59.366734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb4307dfa9845df9b4ddb95c43a48b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:11:59.409486 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2135/4243] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:11:58 2022] DELETE /identity/v3/users/0fb4307dfa9845df9b4ddb95c43a48b5 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:11:59.421704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:59.422269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:59.424066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:59.456373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:11:59.458826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:59.625469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:59.691814 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2109/4244] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:11:59.739950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:11:59.745143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:11:59.934919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5787-2cc6-4177-988d-d8a7e2d319fa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:11:59.940014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1461d-9cb8-4c1e-b095-24bdcbb25b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:11:59.941636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1461d-9cb8-4c1e-b095-24bdcbb25b84 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:11:59.942252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1461d-9cb8-4c1e-b095-24bdcbb25b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:11:59.943595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1461d-9cb8-4c1e-b095-24bdcbb25b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:11:59.959479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1461d-9cb8-4c1e-b095-24bdcbb25b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:11:59.962344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1461d-9cb8-4c1e-b095-24bdcbb25b84 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:11:59.980225 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2136/4245] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:00.502237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:00.505955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:00.542941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2a03f-b450-4c17-86fc-ba4023f07dee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:00.561758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:00.565307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:00.566095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:00.566679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] PATH_INFO: `/v3/users/96e48f22b8e7478986f912e64df670b0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:00.574698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e48f22b8e7478986f912e64df670b0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:00.579665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:00.720836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e48f22b8e7478986f912e64df670b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:00.733864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e48f22b8e7478986f912e64df670b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:00.755002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e48f22b8e7478986f912e64df670b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:00.768755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e48f22b8e7478986f912e64df670b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:00.811738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e48f22b8e7478986f912e64df670b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:00.832397 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2110/4246] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:00 2022] DELETE /identity/v3/users/96e48f22b8e7478986f912e64df670b0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:00.951793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073cad4-b131-476f-8535-e01903124c75 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:00.952890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073cad4-b131-476f-8535-e01903124c75 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:00.953454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073cad4-b131-476f-8535-e01903124c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:00.994273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9073cad4-b131-476f-8535-e01903124c75 None None] MFA Rules not processed for user `015d442ea03c434bb63aa9ba9b537ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:00.996125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9073cad4-b131-476f-8535-e01903124c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:01.091967 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2137/4247] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:01.145201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:01.146826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:01.254070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bf101-5225-4dc5-a083-969ab56a806f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:01.259793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:01.260453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.260783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.261215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] PATH_INFO: `/v3/projects/08e527cf454c4b22b968faa3c149403d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.262390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e527cf454c4b22b968faa3c149403d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:01.264977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:01.350708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e527cf454c4b22b968faa3c149403d', 'request_id': 'req-3a467b49-5528-4f63-854a-e1877cf2ca9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.361339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e527cf454c4b22b968faa3c149403d', 'request_id': 'req-3a467b49-5528-4f63-854a-e1877cf2ca9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.374472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e527cf454c4b22b968faa3c149403d', 'request_id': 'req-3a467b49-5528-4f63-854a-e1877cf2ca9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.377393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:01.393666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] The token cache is being invalidate because project 08e527cf454c4b22b968faa3c149403d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:01.393997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.394745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.398796 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2111/4248] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:01 2022] DELETE /identity/v3/projects/08e527cf454c4b22b968faa3c149403d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:01.405555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17fa35-14a4-4a25-b7bf-e6d6953302ec None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.406347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17fa35-14a4-4a25-b7bf-e6d6953302ec None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.407220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17fa35-14a4-4a25-b7bf-e6d6953302ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.443104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0a17fa35-14a4-4a25-b7bf-e6d6953302ec None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:01.447668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0a17fa35-14a4-4a25-b7bf-e6d6953302ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:01.475651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:01.477569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:01.566893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:01.622274 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2138/4249] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:01.626087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c95d-c9d8-43c5-b991-0f00193bbebd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.626388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c95d-c9d8-43c5-b991-0f00193bbebd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.626725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c95d-c9d8-43c5-b991-0f00193bbebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.679910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5b64c95d-c9d8-43c5-b991-0f00193bbebd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:01.684621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5b64c95d-c9d8-43c5-b991-0f00193bbebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:01.700521 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2139/4250] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:01.706195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64c95d-c9d8-43c5-b991-0f00193bbebd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:01.719957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64c95d-c9d8-43c5-b991-0f00193bbebd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:01.727129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:01.730948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.731646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.732072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.733083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:01.735251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:01.737313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:01.738049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a467b49-5528-4f63-854a-e1877cf2ca9f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:01.749466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:01.750872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.751337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.752033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] PATH_INFO: `/v3/projects/be55c1fb4e6f4870b25e986907d41101` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.753621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be55c1fb4e6f4870b25e986907d41101)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:01.756302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:01.765512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2140/4251] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:01.766484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:01.780451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458a44-5f66-437b-8853-844738366616 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:01.788328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:01.789320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.789767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.790201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.791071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:01.793272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:01.834049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be55c1fb4e6f4870b25e986907d41101', 'request_id': 'req-a8da44ae-d99c-4aa6-820a-5b32c93c4357'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.845932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be55c1fb4e6f4870b25e986907d41101', 'request_id': 'req-a8da44ae-d99c-4aa6-820a-5b32c93c4357'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.846175 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2141/4252] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:01 2022] POST /identity/v3/projects => generated 363 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:01.848041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:01.860337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be55c1fb4e6f4870b25e986907d41101', 'request_id': 'req-a8da44ae-d99c-4aa6-820a-5b32c93c4357'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.881963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] The token cache is being invalidate because project be55c1fb4e6f4870b25e986907d41101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:01.883065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.884629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a8da44ae-d99c-4aa6-820a-5b32c93c4357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:01.891336 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2112/4253] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:01 2022] DELETE /identity/v3/projects/be55c1fb4e6f4870b25e986907d41101 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:12:01.892581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc892-710d-412b-8ec0-0985442e18ad None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.894765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc892-710d-412b-8ec0-0985442e18ad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.896934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc892-710d-412b-8ec0-0985442e18ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.942977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5f9dc892-710d-412b-8ec0-0985442e18ad None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:01.944771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5f9dc892-710d-412b-8ec0-0985442e18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:01.951823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f783d-ce72-4e80-8fc2-638d662e2720 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:01.957014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:01.957644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.958013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.959015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:01.960571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:01.962747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:01.987507 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2113/4254] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:01.991977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd9fd7-9786-4604-980d-8f79f8a11dfc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:01.992306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd9fd7-9786-4604-980d-8f79f8a11dfc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:01.992626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd9fd7-9786-4604-980d-8f79f8a11dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.009974 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2142/4255] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:01 2022] POST /identity/v3/users => generated 282 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:02.014075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.019049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:02.030537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e9bd9fd7-9786-4604-980d-8f79f8a11dfc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:02.033194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e9bd9fd7-9786-4604-980d-8f79f8a11dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:02.044378 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2114/4256] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:02.049027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd9fd7-9786-4604-980d-8f79f8a11dfc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.060604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742713b-142d-49fc-95fb-6d3ab62e0cd3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.062221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd9fd7-9786-4604-980d-8f79f8a11dfc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.069383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.071273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.072408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.073030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.074368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:02.074853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.075430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.076861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.077252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.077611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.077784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.078620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.085035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.098381 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2115/4257] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:02.100876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.112558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2143/4258] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:02.116349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.129099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc64759-9965-40f9-afa4-e8cafa53db88 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.134000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.134653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d28538-80c2-4db6-a18e-4064121eb47e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.134796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.135068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.135664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.136353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.138330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.145734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.147443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.147976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.148470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.149368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.151106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.168067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2144/4259] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:02.169453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.185055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec46ef0-7869-41d7-a395-62214569f1c8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.185256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2116/4260] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:02 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:02.186515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.246439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:02.262335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26b627-82c2-4b99-9f4f-f9a13a03a229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.263519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26b627-82c2-4b99-9f4f-f9a13a03a229 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.264028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26b627-82c2-4b99-9f4f-f9a13a03a229 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.264504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26b627-82c2-4b99-9f4f-f9a13a03a229 admin admin] PATH_INFO: `/v3/projects/da3cbb97415342ddb66f9f4a62a00cd6/users/2b8fea1877d44f8c810fcf1b90575cd3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.292288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb8218-e9c4-4614-98dd-ae296d5e8603 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.311020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.312556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.312893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.313174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.315050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.318110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:02.325139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.328505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26b627-82c2-4b99-9f4f-f9a13a03a229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3cbb97415342ddb66f9f4a62a00cd6, user_id=2b8fea1877d44f8c810fcf1b90575cd3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.336132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26b627-82c2-4b99-9f4f-f9a13a03a229 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.364341 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2145/4261] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:02 2022] PUT /identity/v3/projects/da3cbb97415342ddb66f9f4a62a00cd6/users/2b8fea1877d44f8c810fcf1b90575cd3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:02.373956 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2117/4262] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:02 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:02.375876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4770a8-17a8-48b8-8999-743d3acff712 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.376218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4770a8-17a8-48b8-8999-743d3acff712 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.376501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4770a8-17a8-48b8-8999-743d3acff712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.378293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.399571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d70003-6890-409d-a7a1-cd3da75deec9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.483296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8e4770a8-17a8-48b8-8999-743d3acff712 None None] MFA Rules not processed for user `2b8fea1877d44f8c810fcf1b90575cd3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:02.485016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8e4770a8-17a8-48b8-8999-743d3acff712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:02.545419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.546736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.547018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.547291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.548103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.550292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.564951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2118/4263] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:02.572927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.586397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73974c08-ee9f-48dd-a6e8-518c6401fa8a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.605509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.606158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.606436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.606710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.607363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.609015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.628404 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2146/4264] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:02.630687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4770a8-17a8-48b8-8999-743d3acff712 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.641421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2119/4265] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:12:02.642536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.646441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4770a8-17a8-48b8-8999-743d3acff712 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.658687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.659366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.659740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.660631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.661269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ce013-d5eb-4877-8241-ef54955cab75 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.661813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.663930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.667478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92fa41-a23e-4fd7-9178-0f7c2a16bd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.667874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92fa41-a23e-4fd7-9178-0f7c2a16bd78 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.668264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92fa41-a23e-4fd7-9178-0f7c2a16bd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.668602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92fa41-a23e-4fd7-9178-0f7c2a16bd78 admin admin] PATH_INFO: `/v3/projects/b5087b7fd4b3482082e5dd90fdcef18f/users/976e7f61517e4e3f9f2417078225ca62/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.692906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92fa41-a23e-4fd7-9178-0f7c2a16bd78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5087b7fd4b3482082e5dd90fdcef18f, user_id=976e7f61517e4e3f9f2417078225ca62, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.697235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92fa41-a23e-4fd7-9178-0f7c2a16bd78 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.720593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2147/4266] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:02 2022] POST /identity/v3/projects => generated 363 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:02.721798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.725210 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2120/4267] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:02 2022] PUT /identity/v3/projects/b5087b7fd4b3482082e5dd90fdcef18f/users/976e7f61517e4e3f9f2417078225ca62/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:12:02.735346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9f7c-5b5f-4521-b07f-240813295070 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.735680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9f7c-5b5f-4521-b07f-240813295070 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.736052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9f7c-5b5f-4521-b07f-240813295070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.778325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7a5b9f7c-5b5f-4521-b07f-240813295070 None None] MFA Rules not processed for user `976e7f61517e4e3f9f2417078225ca62`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:02.779744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7a5b9f7c-5b5f-4521-b07f-240813295070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:02.902524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e81df-27be-4e02-a5aa-20777e0499e0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.908488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.909164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.909459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.910665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.911338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.914661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.955377 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2121/4268] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:02.955377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2e75c-353e-43d8-900e-a98e25c82e78 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.955377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2e75c-353e-43d8-900e-a98e25c82e78 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.955377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2e75c-353e-43d8-900e-a98e25c82e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.956311 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2148/4269] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:02 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:02.959798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:02.977796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0b211-28fc-44e1-8d39-e3a8fff8dfb1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:02.983542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829036f-56a1-49fe-b983-58359ec2319e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:02.986149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2ba2e75c-353e-43d8-900e-a98e25c82e78 None None] MFA Rules not processed for user `e4772d35d57f478d929eea70fd11e147`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:02.987066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829036f-56a1-49fe-b983-58359ec2319e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:02.987350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829036f-56a1-49fe-b983-58359ec2319e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:02.987738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829036f-56a1-49fe-b983-58359ec2319e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:02.988488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829036f-56a1-49fe-b983-58359ec2319e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:02.990137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829036f-56a1-49fe-b983-58359ec2319e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:02.990357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2ba2e75c-353e-43d8-900e-a98e25c82e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:03.005132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:03.017307 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2149/4270] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:03.020203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82996-75f0-4b8d-9d7a-a44ddb87ce18 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:03.020492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82996-75f0-4b8d-9d7a-a44ddb87ce18 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:03.020789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82996-75f0-4b8d-9d7a-a44ddb87ce18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:03.044911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-23b82996-75f0-4b8d-9d7a-a44ddb87ce18 None None] MFA Rules not processed for user `976e7f61517e4e3f9f2417078225ca62`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:03.047187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23b82996-75f0-4b8d-9d7a-a44ddb87ce18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:03.059685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2150/4271] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:03.059685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b82996-75f0-4b8d-9d7a-a44ddb87ce18 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:03.078775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b82996-75f0-4b8d-9d7a-a44ddb87ce18 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:03.089582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:03.090269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:03.090544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:03.090813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] PATH_INFO: `/v3/projects/5134b0730f58499b8635d27780002b17/users/77662e6a9fbc473f9e9c8e60683517d6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:03.143188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5134b0730f58499b8635d27780002b17, user_id=77662e6a9fbc473f9e9c8e60683517d6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:03.146097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:03.149922 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2122/4272] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:03.153010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76c0a-74d1-494b-b422-41075898732a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:03.160869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76c0a-74d1-494b-b422-41075898732a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:03.161699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76c0a-74d1-494b-b422-41075898732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:03.171454 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2151/4273] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:03 2022] PUT /identity/v3/projects/5134b0730f58499b8635d27780002b17/users/77662e6a9fbc473f9e9c8e60683517d6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:03.174516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:03.178324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:03.191744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:03.211411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4fc76c0a-74d1-494b-b422-41075898732a None None] MFA Rules not processed for user `e4772d35d57f478d929eea70fd11e147`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:03.212946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4fc76c0a-74d1-494b-b422-41075898732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:03.299121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2123/4274] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:03.300697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc6d2-a983-479f-aff7-3dbedff0031a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:03.300847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76c0a-74d1-494b-b422-41075898732a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:03.310152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:03.310794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:03.311209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:03.311343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:03.448955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:03.451236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:03.475392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2152/4275] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:03.493904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76c0a-74d1-494b-b422-41075898732a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:03.508490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d371a43-991a-4942-aed0-f4d32c8c365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:03.509160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d371a43-991a-4942-aed0-f4d32c8c365e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:03.509431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d371a43-991a-4942-aed0-f4d32c8c365e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:03.509730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d371a43-991a-4942-aed0-f4d32c8c365e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:03.510596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d371a43-991a-4942-aed0-f4d32c8c365e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:03.512215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d371a43-991a-4942-aed0-f4d32c8c365e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:03.526195 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2124/4276] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:03.530454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:03.550498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fdeea-4261-41d8-a67e-fff68d0c3a49 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:03.574615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:03.575497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:03.575775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:03.576172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] PATH_INFO: `/v3/projects/5134b0730f58499b8635d27780002b17/users/77662e6a9fbc473f9e9c8e60683517d6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:03.579793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5134b0730f58499b8635d27780002b17, user_id=77662e6a9fbc473f9e9c8e60683517d6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:03.595560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:03.643549 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2153/4277] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:03 2022] PUT /identity/v3/projects/5134b0730f58499b8635d27780002b17/users/77662e6a9fbc473f9e9c8e60683517d6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:03.655163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606621a-1df2-4461-b983-106151eead8a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:03.655163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606621a-1df2-4461-b983-106151eead8a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:03.655448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606621a-1df2-4461-b983-106151eead8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:03.747022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6606621a-1df2-4461-b983-106151eead8a None None] MFA Rules not processed for user `77662e6a9fbc473f9e9c8e60683517d6`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:03.750099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6606621a-1df2-4461-b983-106151eead8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:03.880752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:03.922412 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2125/4278] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:03.973624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:03.981803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:04.243423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb9fd7-97aa-4300-a4ed-2c989fe0587d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:04.258043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a8752-01ed-4d44-b109-7e0cc29c9403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:04.265488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a8752-01ed-4d44-b109-7e0cc29c9403 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:04.265880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a8752-01ed-4d44-b109-7e0cc29c9403 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:04.266246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a8752-01ed-4d44-b109-7e0cc29c9403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:04.284894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a8752-01ed-4d44-b109-7e0cc29c9403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:04.286359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a8752-01ed-4d44-b109-7e0cc29c9403 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:04.312829 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2154/4279] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:04.870903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606621a-1df2-4461-b983-106151eead8a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:04.872837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6606621a-1df2-4461-b983-106151eead8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:04.895199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606621a-1df2-4461-b983-106151eead8a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:04.922864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:04.924136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:04.924846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:04.925440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:04.931502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:04.933422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11a90d-d988-4de1-8754-7c081cae8ec4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:04.934159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11a90d-d988-4de1-8754-7c081cae8ec4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:04.934159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11a90d-d988-4de1-8754-7c081cae8ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:04.963097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:04.965006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:04.983344 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2126/4280] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:04.993814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6d11a90d-d988-4de1-8754-7c081cae8ec4 None None] MFA Rules not processed for user `976e7f61517e4e3f9f2417078225ca62`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:04.995718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6d11a90d-d988-4de1-8754-7c081cae8ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:05.034772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:05.073211 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2155/4281] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:08.296520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:08.310827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9a713-51c3-40b8-820a-052b8db104b3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:08.319471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:08.321985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:08.321985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:08.321985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:08.476082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:08.485370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:08.503839 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2127/4282] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:10.025474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11a90d-d988-4de1-8754-7c081cae8ec4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:10.048232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11a90d-d988-4de1-8754-7c081cae8ec4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:10.061983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2e81d-13ac-4cc4-87d4-aa83b187e22a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:10.062713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2e81d-13ac-4cc4-87d4-aa83b187e22a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:10.062994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2e81d-13ac-4cc4-87d4-aa83b187e22a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:10.063265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2e81d-13ac-4cc4-87d4-aa83b187e22a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:10.071735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44f2e81d-13ac-4cc4-87d4-aa83b187e22a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:10.141668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2e81d-13ac-4cc4-87d4-aa83b187e22a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:10.155234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2e81d-13ac-4cc4-87d4-aa83b187e22a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:10.190332 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2156/4283] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:11.138045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:11.393142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e364c-7e26-4b96-9bcf-549842fc4bf9 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:11.417635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:11.418414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:11.420888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:11.421514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] PATH_INFO: `/v3/users/2b8fea1877d44f8c810fcf1b90575cd3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:11.422895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8fea1877d44f8c810fcf1b90575cd3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:11.424997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:11.519324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8fea1877d44f8c810fcf1b90575cd3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:11.520605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8fea1877d44f8c810fcf1b90575cd3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:11.530654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8fea1877d44f8c810fcf1b90575cd3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:11.565083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8fea1877d44f8c810fcf1b90575cd3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:11.574171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:11.591676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8fea1877d44f8c810fcf1b90575cd3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:11.619090 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2128/4284] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:11 2022] DELETE /identity/v3/users/2b8fea1877d44f8c810fcf1b90575cd3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:11.624205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:11.624744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:11.625401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:11.677345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:11.679161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:11.818081 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2157/4285] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:11.829374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:11.835875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:11.968471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40241ee1-690b-4ae6-9fc4-b8fc763fb624 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:11.978491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f963-d453-4d1a-a140-d5ceb2211b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:11.980930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f963-d453-4d1a-a140-d5ceb2211b0a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:11.981496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f963-d453-4d1a-a140-d5ceb2211b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:11.981975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f963-d453-4d1a-a140-d5ceb2211b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:12.020180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1f963-d453-4d1a-a140-d5ceb2211b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:12.026681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1f963-d453-4d1a-a140-d5ceb2211b0a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:12.047728 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2129/4286] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:12.219397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:12.221790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:12.249094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1dea7e-dba6-42e7-bbd1-9d3a236aa2b5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:12.254976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:12.256376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:12.256986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:12.257469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] PATH_INFO: `/v3/users/19ff37a1115b478b9ffe5eb234e9085d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:12.258920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ff37a1115b478b9ffe5eb234e9085d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:12.261341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:12.385643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ff37a1115b478b9ffe5eb234e9085d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:12.386986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ff37a1115b478b9ffe5eb234e9085d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:12.409833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ff37a1115b478b9ffe5eb234e9085d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:12.455174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ff37a1115b478b9ffe5eb234e9085d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:12.458492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ff37a1115b478b9ffe5eb234e9085d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:12.520152 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2158/4287] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:12 2022] DELETE /identity/v3/users/19ff37a1115b478b9ffe5eb234e9085d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:12.527492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:12.528061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:12.528885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:12.565231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:12.568820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:12.719828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:12.721372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:12.759997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2130/4288] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:12.774169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:12.777265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:12.843554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:12.886446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c11c8c-8f19-42df-a2f3-349ff1144c68 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:12.897452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:12.899327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:12.900106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:12.900891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] PATH_INFO: `/v3/projects/da3cbb97415342ddb66f9f4a62a00cd6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:12.903803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3cbb97415342ddb66f9f4a62a00cd6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:12.910294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:13.002315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ab24b-1792-425f-ae34-e6ef42c06cea None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:13.003703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3cbb97415342ddb66f9f4a62a00cd6', 'request_id': 'req-b71254c9-cb2c-4da9-becd-c00b53bf85bd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.032275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3cbb97415342ddb66f9f4a62a00cd6', 'request_id': 'req-b71254c9-cb2c-4da9-becd-c00b53bf85bd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.040492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3cbb97415342ddb66f9f4a62a00cd6', 'request_id': 'req-b71254c9-cb2c-4da9-becd-c00b53bf85bd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.071102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] The token cache is being invalidate because project da3cbb97415342ddb66f9f4a62a00cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:13.071678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.073966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.076090 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2159/4289] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:12 2022] DELETE /identity/v3/projects/da3cbb97415342ddb66f9f4a62a00cd6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:13.077234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:13.080022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.101564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:13.170305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:13.171077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:13.171368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:13.171647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:13.174689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.185102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71254c9-cb2c-4da9-becd-c00b53bf85bd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:13.189487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:13.190230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:13.190608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:13.190995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:13.192729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:13.195653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:13.201094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.209747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2131/4290] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:13.210358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:13.212287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.297826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:13.299287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:13.318450 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2160/4291] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:13.320748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:13.323219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.348535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa662b-382b-486a-91d4-c25bfe741ec5 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:13.361204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:13.362148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:13.362435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:13.362735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:13.364909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.370840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff50f8-ea57-4e4d-b665-f5a66c55b84d service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:13.385406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:13.391435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:13.392176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:13.392309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:13.392628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:13.392962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] PATH_INFO: `/v3/users/77662e6a9fbc473f9e9c8e60683517d6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:13.398718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] RBAC: Authorizing `identity:delete_user(user_id=77662e6a9fbc473f9e9c8e60683517d6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:13.401132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:13.436914 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2161/4292] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:13.444496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:13.449093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.489817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:13.506169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb718a0-f003-447d-8bdb-105661ca9b59 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:13.515526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77662e6a9fbc473f9e9c8e60683517d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.516131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77662e6a9fbc473f9e9c8e60683517d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.535127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:13.535919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:13.537723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:13.538071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:13.539780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77662e6a9fbc473f9e9c8e60683517d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.546024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.558719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77662e6a9fbc473f9e9c8e60683517d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.579900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77662e6a9fbc473f9e9c8e60683517d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:13.586873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:13.594500 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2132/4293] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:13 2022] DELETE /identity/v3/users/77662e6a9fbc473f9e9c8e60683517d6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:13.597260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:13.633633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2162/4294] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:13.635046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:13.640751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.706296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:13.747038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a024b9e-4ce9-413b-a821-9e9dd545c67f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:13.753239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:13.754169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:13.754690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:13.755113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:13.767128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:13.768916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:13.784199 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2133/4295] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:12:13.940943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:13.942480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:13.970213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7dbef-6e91-4953-9e5c-b20e3ff9384d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:14.051763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:14.052737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:14.053262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:14.054122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] PATH_INFO: `/v3/projects/2d08eecebc044d64a440f68a1740644f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:14.056092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d08eecebc044d64a440f68a1740644f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:14.058513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:14.185549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d08eecebc044d64a440f68a1740644f', 'request_id': 'req-117e97bd-87ad-4ee7-a832-53cd9cc337ab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.212855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d08eecebc044d64a440f68a1740644f', 'request_id': 'req-117e97bd-87ad-4ee7-a832-53cd9cc337ab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.224714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d08eecebc044d64a440f68a1740644f', 'request_id': 'req-117e97bd-87ad-4ee7-a832-53cd9cc337ab'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.240797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] The token cache is being invalidate because project 2d08eecebc044d64a440f68a1740644f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:14.241615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.242140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.244236 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2163/4296] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:13 2022] DELETE /identity/v3/projects/2d08eecebc044d64a440f68a1740644f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:14.248791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:14.266200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29d8c-ecfd-4770-955a-cdbdfb5d3344 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:14.372243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:14.373178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:14.373555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:14.373926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] PATH_INFO: `/v3/users/8d6d481970c341b2a887415ade03f029` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:14.375796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6d481970c341b2a887415ade03f029)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:14.378013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:14.448393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6d481970c341b2a887415ade03f029'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.449125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6d481970c341b2a887415ade03f029'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.457868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6d481970c341b2a887415ade03f029'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.471731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6d481970c341b2a887415ade03f029'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.499423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a129cef0-71b4-49b9-85aa-385af63e24b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6d481970c341b2a887415ade03f029'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.519781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2134/4297] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:14 2022] DELETE /identity/v3/users/8d6d481970c341b2a887415ade03f029 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:14.559064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:14.561330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:14.581144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e97bd-87ad-4ee7-a832-53cd9cc337ab admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:14.680430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:14.681766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:14.682050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:14.682327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] PATH_INFO: `/v3/projects/5134b0730f58499b8635d27780002b17` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:14.683865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5134b0730f58499b8635d27780002b17)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:14.686594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:14.720712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:14.779146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5134b0730f58499b8635d27780002b17', 'request_id': 'req-1bb07e3a-b99d-42d3-8154-465656bd30b0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.797117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5134b0730f58499b8635d27780002b17', 'request_id': 'req-1bb07e3a-b99d-42d3-8154-465656bd30b0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.814819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5134b0730f58499b8635d27780002b17', 'request_id': 'req-1bb07e3a-b99d-42d3-8154-465656bd30b0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.833851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] The token cache is being invalidate because project 5134b0730f58499b8635d27780002b17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:14.834310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.835439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:14.840606 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2164/4298] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:14 2022] DELETE /identity/v3/projects/5134b0730f58499b8635d27780002b17 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:14.844991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636af6c4-f395-4307-a2ad-373b8574fdf9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:14.845906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636af6c4-f395-4307-a2ad-373b8574fdf9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:14.846344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636af6c4-f395-4307-a2ad-373b8574fdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:14.875186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-636af6c4-f395-4307-a2ad-373b8574fdf9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:14.877446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-636af6c4-f395-4307-a2ad-373b8574fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:14.955035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:14.963760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:14.965502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:14.981449 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2135/4299] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:14.990655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416a3b9-6ed8-42a6-aa09-fe1a05943f6f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:14.990955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416a3b9-6ed8-42a6-aa09-fe1a05943f6f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:14.992343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416a3b9-6ed8-42a6-aa09-fe1a05943f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:14.996913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb07e3a-b99d-42d3-8154-465656bd30b0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.005865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.006963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.007321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.007709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] PATH_INFO: `/v3/projects/b1d3133d731746468bea0151cf066205` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.008973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d3133d731746468bea0151cf066205)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.011074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.019523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6416a3b9-6ed8-42a6-aa09-fe1a05943f6f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:15.020778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6416a3b9-6ed8-42a6-aa09-fe1a05943f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:15.032799 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2136/4300] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:15.036113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416a3b9-6ed8-42a6-aa09-fe1a05943f6f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.047042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416a3b9-6ed8-42a6-aa09-fe1a05943f6f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.053001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.053802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.054557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.054557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.055024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:15.055353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.056977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.071060 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2137/4301] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:15.076389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.088519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d3133d731746468bea0151cf066205', 'request_id': 'req-0d326f70-a911-49c1-82c2-84935e90fd70'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:15.091311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9eb942-bed6-456b-bbcf-0570fd5990e3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.099690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d3133d731746468bea0151cf066205', 'request_id': 'req-0d326f70-a911-49c1-82c2-84935e90fd70'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:15.108174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d3133d731746468bea0151cf066205', 'request_id': 'req-0d326f70-a911-49c1-82c2-84935e90fd70'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:15.124718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] The token cache is being invalidate because project b1d3133d731746468bea0151cf066205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:15.125010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:15.125695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d326f70-a911-49c1-82c2-84935e90fd70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:15.128012 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2165/4302] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:14 2022] DELETE /identity/v3/projects/b1d3133d731746468bea0151cf066205 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:15.134905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909a5d1-3e1c-4b2e-aabd-032bd883769e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.135435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909a5d1-3e1c-4b2e-aabd-032bd883769e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.135912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909a5d1-3e1c-4b2e-aabd-032bd883769e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.174415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0909a5d1-3e1c-4b2e-aabd-032bd883769e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:15.175824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0909a5d1-3e1c-4b2e-aabd-032bd883769e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:15.204552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.208534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.209213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.211019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.212346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.218915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.277241 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2138/4303] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:15 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:15.282876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.285282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:15.287739 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2166/4304] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:15.297825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48ab7-6225-4357-a989-e8a1e0609d47 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.298121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48ab7-6225-4357-a989-e8a1e0609d47 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.299144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48ab7-6225-4357-a989-e8a1e0609d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.333811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-46e48ab7-6225-4357-a989-e8a1e0609d47 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:15.336402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-46e48ab7-6225-4357-a989-e8a1e0609d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:15.379453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:15.413429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8965c8-ba73-4d61-8d5e-5f3bc9223c82 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.420025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.421787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.422077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.422563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.423312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.425579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.462166 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2167/4305] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:15.466925 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2139/4306] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:15 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:15.469352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e48ab7-6225-4357-a989-e8a1e0609d47 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.469611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.489942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e48ab7-6225-4357-a989-e8a1e0609d47 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.496938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.497661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.498077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.498532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.499191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:15.499618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.501534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.523645 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2168/4307] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:15.526256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7adecd-5d2f-48ba-ab1a-a80c6d1a7a60 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.527114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.535517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.536391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.537096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.537601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.539670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.542871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.544852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da195b8f-9796-47a9-ae29-2fe49e663ad4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.551760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.552611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.553781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.554157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.555145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.558656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.564480 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2140/4308] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:15.568370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.594637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335119-e1d1-48e8-8057-a4e581702d6f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.615805 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2169/4309] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:15 2022] POST /identity/v3/projects => generated 411 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:15.619818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.620643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.621073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.621444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.622233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.624756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.635332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.640740 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2141/4310] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:15.645654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.658837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b2f5-2105-4af3-9d4d-c076705d75fe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.702506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:15.749296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2002004-3a54-4d86-8968-b1134898efd8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.757136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2a101-b04b-4f2f-9ce9-e10506884f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.757993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a101-b04b-4f2f-9ce9-e10506884f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.758331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a101-b04b-4f2f-9ce9-e10506884f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.758717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a101-b04b-4f2f-9ce9-e10506884f7a admin admin] PATH_INFO: `/v3/projects/ac343cd9cb8645c192ba283bed0c5a6d/users/cee144a4d6524e4898108a52b1cd0fba/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.763147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:15.763780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.764119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.764497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.765789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.768417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.777689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2a101-b04b-4f2f-9ce9-e10506884f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac343cd9cb8645c192ba283bed0c5a6d, user_id=cee144a4d6524e4898108a52b1cd0fba, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:15.782200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2a101-b04b-4f2f-9ce9-e10506884f7a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:15.796777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2170/4311] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:15 2022] POST /identity/v3/users => generated 306 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:12:15.802607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:15.826774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85298862-43c6-4ea3-a6b3-a5996b4418c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:15.840325 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2142/4312] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:15 2022] PUT /identity/v3/projects/ac343cd9cb8645c192ba283bed0c5a6d/users/cee144a4d6524e4898108a52b1cd0fba/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:15.842606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:15.842894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:15.843302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:15.907297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] MFA Rules not processed for user `cee144a4d6524e4898108a52b1cd0fba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:15.908933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:15.996464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.000186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.000578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.001170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.001761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.003656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:16.036156 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2171/4313] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:12:16.037589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:16.062551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308deda-d2d0-47f0-99e2-01d44a91b4b0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:16.078927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.080434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.081124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.081791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.082758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.085132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:16.121309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2172/4314] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:16.126323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:16.144164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2143/4315] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:16.165021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a25be0-558c-4cdd-842e-fbf8fb958c1d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:16.172644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1dd8e-7521-4df5-9aec-4b6ac9afc553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.173453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1dd8e-7521-4df5-9aec-4b6ac9afc553 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.173930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1dd8e-7521-4df5-9aec-4b6ac9afc553 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.174316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1dd8e-7521-4df5-9aec-4b6ac9afc553 admin admin] PATH_INFO: `/v3/projects/95815c8de11d495f80754ddf93a89691/users/3340268723664b09a9a665a6e830e518/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.187689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:16.189964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:16.198478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:16.206566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1dd8e-7521-4df5-9aec-4b6ac9afc553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95815c8de11d495f80754ddf93a89691, user_id=3340268723664b09a9a665a6e830e518, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.213029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1dd8e-7521-4df5-9aec-4b6ac9afc553 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:16.262786 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2173/4316] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:16 2022] PUT /identity/v3/projects/95815c8de11d495f80754ddf93a89691/users/3340268723664b09a9a665a6e830e518/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:16.274547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923cf8a-f205-4038-89df-6e39b204a2ac None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.274935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923cf8a-f205-4038-89df-6e39b204a2ac None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.275367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923cf8a-f205-4038-89df-6e39b204a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.318335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5923cf8a-f205-4038-89df-6e39b204a2ac None None] MFA Rules not processed for user `3340268723664b09a9a665a6e830e518`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:16.318677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f59e36-4205-4abc-a922-50e77705d37a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:16.320287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5923cf8a-f205-4038-89df-6e39b204a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:16.390302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.391140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.391588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.394125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.410591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:16.429246 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2174/4317] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:16 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:16.433685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923cf8a-f205-4038-89df-6e39b204a2ac None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:16.521246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.524044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:16.542405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923cf8a-f205-4038-89df-6e39b204a2ac None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:16.543980 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2144/4318] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:12:16.565483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.566247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.566619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.566983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.567715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.571144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:16.643179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:16.643480 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2175/4319] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:16 2022] POST /identity/v3/projects => generated 411 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:12:16.675824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97aa1ba-d541-4159-bfdb-1416f152ab2d service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:16.769430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.770283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.770735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.771215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.772026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.775009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:16.815617 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2145/4320] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:16 2022] POST /identity/v3/users => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:12:16.820630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:16.841920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb771dee-e171-453c-a203-3d1d00b352b4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:16.848599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.849610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.850064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.851847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.852733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.856400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:16.902351 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2176/4321] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:16.907684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:16.936108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a1473-65d8-4d0b-a02e-e5bd8c31b809 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:16.954271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:16.955600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:16.955901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:16.956171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] PATH_INFO: `/v3/projects/ce16a1a797ba4522bbde11589817c82a/users/6ee9b067b0ab4f3b8964989d675158a7/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:16.986777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce16a1a797ba4522bbde11589817c82a, user_id=6ee9b067b0ab4f3b8964989d675158a7, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:16.991768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:17.019023 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2146/4322] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:16 2022] PUT /identity/v3/projects/ce16a1a797ba4522bbde11589817c82a/users/6ee9b067b0ab4f3b8964989d675158a7/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:17.023458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:17.056539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d7838-0479-4e33-8b0e-f2abcee9c950 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:17.230868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:17.254764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068c29d-f9a6-4e2c-81de-d298cc22c913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:17.261999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068c29d-f9a6-4e2c-81de-d298cc22c913 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:17.262377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068c29d-f9a6-4e2c-81de-d298cc22c913 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:17.262780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068c29d-f9a6-4e2c-81de-d298cc22c913 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:17.263671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068c29d-f9a6-4e2c-81de-d298cc22c913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:17.265699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068c29d-f9a6-4e2c-81de-d298cc22c913 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:17.300215 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2177/4323] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:12:17.305380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:17.323190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d95b2f-2270-4fe0-9ca2-80eba8772868 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:17.335851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:17.337483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:17.337968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:17.338440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] PATH_INFO: `/v3/projects/ce16a1a797ba4522bbde11589817c82a/users/6ee9b067b0ab4f3b8964989d675158a7/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:17.342669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce16a1a797ba4522bbde11589817c82a, user_id=6ee9b067b0ab4f3b8964989d675158a7, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:17.350062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:17.415208 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2147/4324] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:17 2022] PUT /identity/v3/projects/ce16a1a797ba4522bbde11589817c82a/users/6ee9b067b0ab4f3b8964989d675158a7/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:17.417048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e5cfa-5fa2-49d8-b1c3-f1981c475c29 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:17.418899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e5cfa-5fa2-49d8-b1c3-f1981c475c29 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:17.419358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e5cfa-5fa2-49d8-b1c3-f1981c475c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:17.485534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-462e5cfa-5fa2-49d8-b1c3-f1981c475c29 None None] MFA Rules not processed for user `6ee9b067b0ab4f3b8964989d675158a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:17.487773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-462e5cfa-5fa2-49d8-b1c3-f1981c475c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:17.687236 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2178/4325] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:17 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:17.716758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:17.824921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8051e-559c-41ac-8d25-8a6a6aa886bb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:17.829486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e4cc7-e01f-44cb-a289-5e90617a6617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:17.830231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e4cc7-e01f-44cb-a289-5e90617a6617 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:17.831858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e4cc7-e01f-44cb-a289-5e90617a6617 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:17.832228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e4cc7-e01f-44cb-a289-5e90617a6617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:17.859313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e4cc7-e01f-44cb-a289-5e90617a6617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:17.860799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e4cc7-e01f-44cb-a289-5e90617a6617 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:17.882715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2148/4326] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:17 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:22.501897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e5cfa-5fa2-49d8-b1c3-f1981c475c29 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:22.619996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e5cfa-5fa2-49d8-b1c3-f1981c475c29 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:22.626994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:22.626994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:22.626994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:22.627403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] PATH_INFO: `/v3/users/3340268723664b09a9a665a6e830e518` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:22.628616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3340268723664b09a9a665a6e830e518)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:22.631036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:22.712159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3340268723664b09a9a665a6e830e518'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:22.713140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3340268723664b09a9a665a6e830e518'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:22.723329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3340268723664b09a9a665a6e830e518'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:22.732259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3340268723664b09a9a665a6e830e518'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:22.733060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3340268723664b09a9a665a6e830e518'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:22.745602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:22.753259 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2179/4327] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:22 2022] DELETE /identity/v3/users/3340268723664b09a9a665a6e830e518 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:22.757801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:22.758533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:22.758533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:22.779893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:22.789859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:22.795682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:22.906655 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2149/4328] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:22.932465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:22.935143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:23.011829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4c521-f253-478d-9ada-125781aec604 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:23.015608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:23.016222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:23.016495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:23.016877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:23.029631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:23.031091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:23.043711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2180/4329] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:25.115282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:25.118490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:25.132180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8856d-135d-41d2-9b19-157063e149df None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:25.141736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:25.142807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:25.143315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:25.143840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] PATH_INFO: `/v3/projects/95815c8de11d495f80754ddf93a89691` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:25.145267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95815c8de11d495f80754ddf93a89691)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:25.147822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:25.289964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95815c8de11d495f80754ddf93a89691', 'request_id': 'req-30f38ac1-50d2-409b-a246-2fa34e977767'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.305107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95815c8de11d495f80754ddf93a89691', 'request_id': 'req-30f38ac1-50d2-409b-a246-2fa34e977767'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.331749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95815c8de11d495f80754ddf93a89691', 'request_id': 'req-30f38ac1-50d2-409b-a246-2fa34e977767'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.351664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] The token cache is being invalidate because project 95815c8de11d495f80754ddf93a89691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:25.352235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.355410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.365761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2150/4330] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:25 2022] DELETE /identity/v3/projects/95815c8de11d495f80754ddf93a89691 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:25.367392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:25.387224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ef5c-8b43-4a11-bde1-8c142afa1da6 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:25.516002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:25.517249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:25.517754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:25.518247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] PATH_INFO: `/v3/users/6ee9b067b0ab4f3b8964989d675158a7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:25.522850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ee9b067b0ab4f3b8964989d675158a7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:25.525895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:25.588978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ee9b067b0ab4f3b8964989d675158a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.590121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ee9b067b0ab4f3b8964989d675158a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.638528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ee9b067b0ab4f3b8964989d675158a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.650033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ee9b067b0ab4f3b8964989d675158a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.650956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ee9b067b0ab4f3b8964989d675158a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:25.700750 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2181/4331] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:25 2022] DELETE /identity/v3/users/6ee9b067b0ab4f3b8964989d675158a7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:26.282944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:26.284701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:26.302438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:26.304153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:26.312890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38ac1-50d2-409b-a246-2fa34e977767 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:26.351359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:26.444321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:26.451253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:26.452637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:26.453479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] PATH_INFO: `/v3/projects/ce16a1a797ba4522bbde11589817c82a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:26.456073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce16a1a797ba4522bbde11589817c82a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:26.465269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:26.465901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4bddd-c198-4087-98d5-3f0c56834ff8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:26.481469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:26.482232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:26.482527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:26.482958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] PATH_INFO: `/v3/users/b11313762b324174a9a5f362d1cf4bdd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:26.484649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11313762b324174a9a5f362d1cf4bdd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:26.488364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:26.542521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:26.573833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce16a1a797ba4522bbde11589817c82a', 'request_id': 'req-fc225107-5559-4e34-a60e-52789d439c1c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.585963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce16a1a797ba4522bbde11589817c82a', 'request_id': 'req-fc225107-5559-4e34-a60e-52789d439c1c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.594464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11313762b324174a9a5f362d1cf4bdd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.595261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11313762b324174a9a5f362d1cf4bdd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.604461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce16a1a797ba4522bbde11589817c82a', 'request_id': 'req-fc225107-5559-4e34-a60e-52789d439c1c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.617192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11313762b324174a9a5f362d1cf4bdd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.627204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11313762b324174a9a5f362d1cf4bdd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.628783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e64b8bc5-efb0-42ae-991d-e499744e8825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11313762b324174a9a5f362d1cf4bdd'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.648941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] The token cache is being invalidate because project ce16a1a797ba4522bbde11589817c82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:26.649391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.653041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc225107-5559-4e34-a60e-52789d439c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:26.658153 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2182/4332] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:26 2022] DELETE /identity/v3/users/b11313762b324174a9a5f362d1cf4bdd => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:26.661867 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2151/4333] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:26 2022] DELETE /identity/v3/projects/ce16a1a797ba4522bbde11589817c82a => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:26.669144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33147c5-5aa2-4d5a-9fe1-dbb7293dbea9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:26.670077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33147c5-5aa2-4d5a-9fe1-dbb7293dbea9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:26.670626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33147c5-5aa2-4d5a-9fe1-dbb7293dbea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:26.699754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f33147c5-5aa2-4d5a-9fe1-dbb7293dbea9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:26.701674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f33147c5-5aa2-4d5a-9fe1-dbb7293dbea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:26.843073 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2183/4334] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:26.846216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f440a1-c6e3-468b-aa6f-56c00dab27ac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:26.846833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f440a1-c6e3-468b-aa6f-56c00dab27ac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:26.846956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f440a1-c6e3-468b-aa6f-56c00dab27ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:26.873872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-59f440a1-c6e3-468b-aa6f-56c00dab27ac None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:26.875288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-59f440a1-c6e3-468b-aa6f-56c00dab27ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:26.892108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33147c5-5aa2-4d5a-9fe1-dbb7293dbea9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:26.905430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2152/4335] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:26.908438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33147c5-5aa2-4d5a-9fe1-dbb7293dbea9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:26.920813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:26.921450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:26.921722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:26.921993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:26.923402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:26.923670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:26.925688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:26.944844 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2184/4336] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:26.949815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f440a1-c6e3-468b-aa6f-56c00dab27ac None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:26.961734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f440a1-c6e3-468b-aa6f-56c00dab27ac None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:26.967847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:26.968632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:26.969004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:26.969395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:26.970043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:26.972279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:27.027728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.038437 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2153/4337] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:26 2022] POST /identity/v3/projects => generated 369 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:27.041375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719619f-4ccb-4ff8-9506-eef71d64ea6a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:27.077657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:27.149035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:27.149688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.150286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.150657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.151959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:27.154692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:27.192968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.194656 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2185/4338] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:27 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:27.209012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce23284-b6e4-47b8-82fe-0488c0aab97c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:27.218371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:27.219336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.220814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.220814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.220814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:27.222803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:27.243949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.272871 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2154/4339] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:12:27.283486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b2f1d-0d5b-450a-89ff-40fede8f83f6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:27.302440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:27.308517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.309145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.310363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.311377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:27.321088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:27.343347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.345713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:27.365479 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2186/4340] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:27.372814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.388476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6540e-64f3-417b-9ab2-44ebc977e33c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:27.407400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946685b-a660-4fb6-94cb-c5ab14aa5113 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:27.409653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:27.410647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.410996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.411522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] PATH_INFO: `/v3/projects/c24fd5e915af4b6c872d69b99e7375ce` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.413981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24fd5e915af4b6c872d69b99e7375ce)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:27.416172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:27.418553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2746a-23f7-42c1-a880-6a7c644c6f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:27.420050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2746a-23f7-42c1-a880-6a7c644c6f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.420737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2746a-23f7-42c1-a880-6a7c644c6f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.422676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2746a-23f7-42c1-a880-6a7c644c6f1c admin admin] PATH_INFO: `/v3/projects/a924daeef64b4721a640fadf821f8511/users/11bee4b027a84a979f527dc91586bb63/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.467039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2746a-23f7-42c1-a880-6a7c644c6f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a924daeef64b4721a640fadf821f8511, user_id=11bee4b027a84a979f527dc91586bb63, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:27.475255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2746a-23f7-42c1-a880-6a7c644c6f1c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:27.506108 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2187/4341] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:27 2022] PUT /identity/v3/projects/a924daeef64b4721a640fadf821f8511/users/11bee4b027a84a979f527dc91586bb63/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:27.521790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.523796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.524263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.579344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24fd5e915af4b6c872d69b99e7375ce', 'request_id': 'req-91b0750f-879f-46a3-93b8-4b296c0c9814'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:27.594712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] MFA Rules not processed for user `11bee4b027a84a979f527dc91586bb63`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:27.596127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:27.618291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24fd5e915af4b6c872d69b99e7375ce', 'request_id': 'req-91b0750f-879f-46a3-93b8-4b296c0c9814'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:27.636887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24fd5e915af4b6c872d69b99e7375ce', 'request_id': 'req-91b0750f-879f-46a3-93b8-4b296c0c9814'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:27.658823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] The token cache is being invalidate because project c24fd5e915af4b6c872d69b99e7375ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:27.662182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:27.664873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-91b0750f-879f-46a3-93b8-4b296c0c9814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:27.667770 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2155/4342] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:27 2022] DELETE /identity/v3/projects/c24fd5e915af4b6c872d69b99e7375ce => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:27.674940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6f57-558f-4593-bbb7-6fa7e9fcea37 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.676414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6f57-558f-4593-bbb7-6fa7e9fcea37 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.676960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6f57-558f-4593-bbb7-6fa7e9fcea37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.707857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0f1b6f57-558f-4593-bbb7-6fa7e9fcea37 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:27.710263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f1b6f57-558f-4593-bbb7-6fa7e9fcea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:27.722981 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2188/4343] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:27.729194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.731815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:27.835690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f5929-ca51-4b89-bc63-980da52a0b88 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:27.837888 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2156/4344] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:27.839646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31a37-a2f3-449a-9196-a160ea88cffc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.842325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31a37-a2f3-449a-9196-a160ea88cffc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.842778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a31a37-a2f3-449a-9196-a160ea88cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.843829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:27.844784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:27.845385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:27.845385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:27.846485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:27.849100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:27.870099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:27.902650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:27.904222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-61a31a37-a2f3-449a-9196-a160ea88cffc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:27.907558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-61a31a37-a2f3-449a-9196-a160ea88cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:27.935339 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2157/4345] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:27.936977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a31a37-a2f3-449a-9196-a160ea88cffc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.940139 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2189/4346] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:27 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:12:27.945638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:27.959260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a31a37-a2f3-449a-9196-a160ea88cffc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.075897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65a06-e61f-465a-afed-bc7519c8cf47 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.083743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.085259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.085733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.086258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.087436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.101178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.143972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.145050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.145345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.145838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.146619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:28.147213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.149866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.155840 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2190/4347] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:27 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:28.160058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.170932 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2158/4348] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:12:28.173276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.189047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d3c0-6305-4cc7-bacb-f3c3beb10b77 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.193470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c833cd-235b-4d34-8f6e-9119ac34e7d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.202129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.202979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.203274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.203546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.204401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.205149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.205947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.207195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.207687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.208171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.208844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.212316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.230069 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2191/4349] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:28.235643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.251259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbacb4-ff10-4bad-8dc1-5a8333cfbd8a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.256558 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2159/4350] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:28 2022] POST /identity/v3/projects => generated 395 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:28.260295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.264326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.265319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.266446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.267213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] PATH_INFO: `/v3/projects/14123409b8ea43f7a884288a9145b6bf/users/304144ea918141c18e1d33ac8196c821/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.293892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14123409b8ea43f7a884288a9145b6bf, user_id=304144ea918141c18e1d33ac8196c821, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.301644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.326263 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2192/4351] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:28 2022] PUT /identity/v3/projects/14123409b8ea43f7a884288a9145b6bf/users/304144ea918141c18e1d33ac8196c821/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:12:28.332469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.424066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6bad3-bacb-485f-9535-a9bfe684bc61 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.538564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928b5cb-4439-442e-8766-904226a20f95 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.559729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.561603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.562168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.562826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.565680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.568888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.590238 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2193/4352] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:28.597729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.604866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.604866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.604866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.604866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.605475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.605475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.629046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f6996-0ebc-4b2b-8042-eb8de8247893 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.646771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35101b72-ae66-491a-bb52-bcafd1e82f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.647781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35101b72-ae66-491a-bb52-bcafd1e82f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.648549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35101b72-ae66-491a-bb52-bcafd1e82f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.648966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35101b72-ae66-491a-bb52-bcafd1e82f0a admin admin] PATH_INFO: `/v3/projects/14123409b8ea43f7a884288a9145b6bf/users/304144ea918141c18e1d33ac8196c821/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.652644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35101b72-ae66-491a-bb52-bcafd1e82f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14123409b8ea43f7a884288a9145b6bf, user_id=304144ea918141c18e1d33ac8196c821, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.656205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35101b72-ae66-491a-bb52-bcafd1e82f0a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.670907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2160/4353] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:28 2022] POST /identity/v3/users => generated 298 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:28.674812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.701216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ed46-526c-4ae4-a9c6-86267af1a494 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.708047 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2194/4354] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:28 2022] PUT /identity/v3/projects/14123409b8ea43f7a884288a9145b6bf/users/304144ea918141c18e1d33ac8196c821/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:28.708912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:28.710447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e60c11-5564-4402-ad1e-f9c9196d8f12 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.710932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e60c11-5564-4402-ad1e-f9c9196d8f12 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.711479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e60c11-5564-4402-ad1e-f9c9196d8f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.761412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-54e60c11-5564-4402-ad1e-f9c9196d8f12 None None] MFA Rules not processed for user `304144ea918141c18e1d33ac8196c821`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:28.763140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-54e60c11-5564-4402-ad1e-f9c9196d8f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:28.797210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:28.845917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.846567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.846832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.847419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.848079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.864728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.868783 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2161/4355] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:12:28.870952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.881696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f980b0-4358-46c4-b761-e62e1218c80c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.887147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.887798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.888050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.888329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.889037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:28.895261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:28.897227 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2195/4356] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:28.904828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2162/4357] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:12:28.909630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e60c11-5564-4402-ad1e-f9c9196d8f12 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.912682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:28.914635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:28.936559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e60c11-5564-4402-ad1e-f9c9196d8f12 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:28.953402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130e5d9-c566-4240-bb03-1c45b8857f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:28.954197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e5d9-c566-4240-bb03-1c45b8857f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:28.954362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e5d9-c566-4240-bb03-1c45b8857f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:28.955165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e5d9-c566-4240-bb03-1c45b8857f02 admin admin] PATH_INFO: `/v3/projects/f23e44d8a5094104b35d0288d33f0ab6/users/241251b91c43444b900922f942681cc5/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:28.987606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130e5d9-c566-4240-bb03-1c45b8857f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23e44d8a5094104b35d0288d33f0ab6, user_id=241251b91c43444b900922f942681cc5, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:29.004699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130e5d9-c566-4240-bb03-1c45b8857f02 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:29.029434 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2196/4358] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:28 2022] PUT /identity/v3/projects/f23e44d8a5094104b35d0288d33f0ab6/users/241251b91c43444b900922f942681cc5/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:29.035281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b9d54-cf9f-4b7f-91ab-79f2d4ad40f7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:29.036978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b9d54-cf9f-4b7f-91ab-79f2d4ad40f7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:29.038184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b9d54-cf9f-4b7f-91ab-79f2d4ad40f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:29.045938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acb226-3d32-43f0-b65f-3887e08faf65 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:29.072480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2b8b9d54-cf9f-4b7f-91ab-79f2d4ad40f7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:29.086481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2b8b9d54-cf9f-4b7f-91ab-79f2d4ad40f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:29.144609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:29.145708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:29.145970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:29.146234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:29.192610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2197/4359] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:29.250155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b9d54-cf9f-4b7f-91ab-79f2d4ad40f7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:29.255457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:29.258641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:29.263745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:29.272288 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2163/4360] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:29.273478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b9d54-cf9f-4b7f-91ab-79f2d4ad40f7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:29.277920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acab9bd-71b7-48c8-9956-568080d174c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:29.278645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab9bd-71b7-48c8-9956-568080d174c2 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:29.279314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab9bd-71b7-48c8-9956-568080d174c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:29.279314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab9bd-71b7-48c8-9956-568080d174c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:29.291240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acab9bd-71b7-48c8-9956-568080d174c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:29.292880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acab9bd-71b7-48c8-9956-568080d174c2 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:29.304810 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2198/4361] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:12:29.549223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:29.552286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:29.573113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa341f-1c17-42b0-a1a1-374fb5e800a4 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:29.578077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:29.578888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:29.579256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:29.579612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] PATH_INFO: `/v3/users/cee144a4d6524e4898108a52b1cd0fba` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:29.581152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee144a4d6524e4898108a52b1cd0fba)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:29.583539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:29.649044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee144a4d6524e4898108a52b1cd0fba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:29.650172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee144a4d6524e4898108a52b1cd0fba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:29.659048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee144a4d6524e4898108a52b1cd0fba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:29.670490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee144a4d6524e4898108a52b1cd0fba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:29.689944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee144a4d6524e4898108a52b1cd0fba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:29.700432 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2164/4362] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:29 2022] DELETE /identity/v3/users/cee144a4d6524e4898108a52b1cd0fba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:29.709071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:29.709610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:29.709883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:29.734526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:29.736165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:29.863652 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2199/4363] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:29.885112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:29.885112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:30.039864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78203193-e6a3-46e7-ba9f-0f7553749ed0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:30.045782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:30.046639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:30.046917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:30.047269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:30.070780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:30.073924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:30.095012 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2165/4364] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:30.810176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:30.811952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:30.837637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1abacd-2be3-4131-b8d0-78d26b9d22b0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:30.857552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:30.857552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:30.857552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:30.857552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] PATH_INFO: `/v3/projects/ac343cd9cb8645c192ba283bed0c5a6d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:30.858064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac343cd9cb8645c192ba283bed0c5a6d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:30.860524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:30.880421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:30.882782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:30.903900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:30.982149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac343cd9cb8645c192ba283bed0c5a6d', 'request_id': 'req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:31.007184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac343cd9cb8645c192ba283bed0c5a6d', 'request_id': 'req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:31.015034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac343cd9cb8645c192ba283bed0c5a6d', 'request_id': 'req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:31.032811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358fe7-c451-4866-a007-94e82ad877d2 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:31.042523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] The token cache is being invalidate because project ac343cd9cb8645c192ba283bed0c5a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:31.043049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:31.043735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37af1cbf-f2a4-4fb6-9f35-d3d792712fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:31.046093 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2200/4365] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:30 2022] DELETE /identity/v3/projects/ac343cd9cb8645c192ba283bed0c5a6d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:31.051101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14508a0-76c2-47b2-8726-c60cd123c491 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.051753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14508a0-76c2-47b2-8726-c60cd123c491 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.052221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14508a0-76c2-47b2-8726-c60cd123c491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.059178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:31.102468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d14508a0-76c2-47b2-8726-c60cd123c491 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:31.104280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d14508a0-76c2-47b2-8726-c60cd123c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:31.215736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:31.216724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.217403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.217848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.222673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:31.247500 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2201/4366] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:31.250688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:31.254942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:31.257443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec7d97-2dd8-4ae2-ad16-64f6865feb6d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.262858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec7d97-2dd8-4ae2-ad16-64f6865feb6d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.262858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec7d97-2dd8-4ae2-ad16-64f6865feb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.274477 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2166/4367] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:31.294663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e9ec7d97-2dd8-4ae2-ad16-64f6865feb6d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:31.296293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e9ec7d97-2dd8-4ae2-ad16-64f6865feb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:31.309745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2202/4368] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:31.314067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:31.334915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17136-fb8e-4527-9a34-834e6618ee5b service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:31.347348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:31.348184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.348466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.350953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.351826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:31.352144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:31.355714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:31.382144 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2167/4369] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:31.383016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec7d97-2dd8-4ae2-ad16-64f6865feb6d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:31.403886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec7d97-2dd8-4ae2-ad16-64f6865feb6d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:31.417901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:31.418539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.418814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.419085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.419868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:31.422069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:31.500684 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2203/4370] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:31 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:31.502957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:31.517600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6424-180d-42ec-b61d-25812ef086f0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:31.646354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:31.650358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.652931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.653326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.654569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:31.657340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:31.702749 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2168/4371] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:31 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:31.707248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:31.732623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54747f84-33ce-4f62-8ea2-50fab001190a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:31.745991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:31.746646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.746924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.747290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.747959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:31.750805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:31.778351 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2204/4372] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:31.782620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:31.804886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fadd4-5b64-4c1e-b330-04e7effe6b60 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:31.821944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106373f-c853-4e1d-93d9-5a3fe35b792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:31.823281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106373f-c853-4e1d-93d9-5a3fe35b792e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.823569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106373f-c853-4e1d-93d9-5a3fe35b792e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.823934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106373f-c853-4e1d-93d9-5a3fe35b792e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.824979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106373f-c853-4e1d-93d9-5a3fe35b792e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:31.826643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106373f-c853-4e1d-93d9-5a3fe35b792e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:31.857092 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2169/4373] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:31.859021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:31.874729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c7dbf-bbfb-4da3-8d30-9012acbd15ae admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:31.908471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:31.908471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.908471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.908970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] PATH_INFO: `/v3/projects/b412ae6345c5431db774b5786b1a24a3/users/fb59b75f396441a89e153d61d6726cdf/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:31.916525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b412ae6345c5431db774b5786b1a24a3, user_id=fb59b75f396441a89e153d61d6726cdf, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:31.920714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:31.956647 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2205/4374] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:31 2022] PUT /identity/v3/projects/b412ae6345c5431db774b5786b1a24a3/users/fb59b75f396441a89e153d61d6726cdf/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:31.968310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc61e8f-6498-41bb-b1af-f13cfbb76cfb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:31.968975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc61e8f-6498-41bb-b1af-f13cfbb76cfb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:31.969228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc61e8f-6498-41bb-b1af-f13cfbb76cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:32.020003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3bc61e8f-6498-41bb-b1af-f13cfbb76cfb None None] MFA Rules not processed for user `fb59b75f396441a89e153d61d6726cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:32.021824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3bc61e8f-6498-41bb-b1af-f13cfbb76cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:32.102221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:32.131556 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2170/4375] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:32.159346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:32.165920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:32.250734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:32.276833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6559c-130b-4b33-9582-0235e56227df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:32.287198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563ad74-d122-44bd-bf36-5dd562fbfdb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:32.288454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ad74-d122-44bd-bf36-5dd562fbfdb0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:32.288872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ad74-d122-44bd-bf36-5dd562fbfdb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:32.289261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ad74-d122-44bd-bf36-5dd562fbfdb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:32.307101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563ad74-d122-44bd-bf36-5dd562fbfdb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:32.309994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563ad74-d122-44bd-bf36-5dd562fbfdb0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:32.336360 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2206/4376] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:32.399810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc61e8f-6498-41bb-b1af-f13cfbb76cfb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:32.499525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc61e8f-6498-41bb-b1af-f13cfbb76cfb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:32.504597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:32.505471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:32.505793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:32.506156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] PATH_INFO: `/v3/users/fb59b75f396441a89e153d61d6726cdf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:32.507493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb59b75f396441a89e153d61d6726cdf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:32.509704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:32.582091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb59b75f396441a89e153d61d6726cdf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:32.582895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb59b75f396441a89e153d61d6726cdf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:32.593856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb59b75f396441a89e153d61d6726cdf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:32.617645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb59b75f396441a89e153d61d6726cdf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:32.640336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb59b75f396441a89e153d61d6726cdf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:32.649217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2171/4377] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:12:32 2022] DELETE /identity/v3/users/fb59b75f396441a89e153d61d6726cdf => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:32.654463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:32.654977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:32.655509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:32.682852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:32.684166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:32.766691 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2207/4378] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:32.816085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:32.822449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:32.999123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482915d-d4ec-4a5c-87f7-d21dfdcef335 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:33.006033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e93a9-ba92-4d94-b81a-2e20eca34dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:33.007527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e93a9-ba92-4d94-b81a-2e20eca34dcf service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:33.008383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e93a9-ba92-4d94-b81a-2e20eca34dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:33.009745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e93a9-ba92-4d94-b81a-2e20eca34dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:33.027545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e93a9-ba92-4d94-b81a-2e20eca34dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:33.029192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e93a9-ba92-4d94-b81a-2e20eca34dcf service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:33.047456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2172/4379] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:34.226834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:34.231290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:34.245048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac765-842f-42d9-87ee-44e4e9818169 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:34.252480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:34.253452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:34.253951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:34.254487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] PATH_INFO: `/v3/projects/b412ae6345c5431db774b5786b1a24a3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:34.256010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b412ae6345c5431db774b5786b1a24a3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:34.259202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:34.355311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b412ae6345c5431db774b5786b1a24a3', 'request_id': 'req-3f552cf2-fb3e-4901-bef8-461bf8fd1784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:34.371606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b412ae6345c5431db774b5786b1a24a3', 'request_id': 'req-3f552cf2-fb3e-4901-bef8-461bf8fd1784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:34.381266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b412ae6345c5431db774b5786b1a24a3', 'request_id': 'req-3f552cf2-fb3e-4901-bef8-461bf8fd1784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:34.403319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] The token cache is being invalidate because project b412ae6345c5431db774b5786b1a24a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:34.404285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:34.404985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f552cf2-fb3e-4901-bef8-461bf8fd1784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:34.410392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2208/4380] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:34 2022] DELETE /identity/v3/projects/b412ae6345c5431db774b5786b1a24a3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:34.411860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a011ab-3de5-479c-94eb-09c35e1e5df8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:34.412380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a011ab-3de5-479c-94eb-09c35e1e5df8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:34.412496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a011ab-3de5-479c-94eb-09c35e1e5df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:34.442286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e7a011ab-3de5-479c-94eb-09c35e1e5df8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:34.443877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e7a011ab-3de5-479c-94eb-09c35e1e5df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:34.471978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:34.562472 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2173/4381] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:12:34.572694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81972281-8231-4de5-8e82-8c9f6180ed32 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:34.573202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81972281-8231-4de5-8e82-8c9f6180ed32 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:34.573549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81972281-8231-4de5-8e82-8c9f6180ed32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:34.604029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-81972281-8231-4de5-8e82-8c9f6180ed32 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:34.605974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-81972281-8231-4de5-8e82-8c9f6180ed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:34.620942 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2209/4382] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:34.622367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a011ab-3de5-479c-94eb-09c35e1e5df8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:34.635366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a011ab-3de5-479c-94eb-09c35e1e5df8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:34.647402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:34.648441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:34.649165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:34.649620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:34.650443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:34.650918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:34.657579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:34.683270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81972281-8231-4de5-8e82-8c9f6180ed32 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:34.691540 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2174/4383] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:12:34.698916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81972281-8231-4de5-8e82-8c9f6180ed32 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:34.705588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:34.706775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:34.707163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:34.708299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:34.709405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:34.711882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:34.774520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2210/4384] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:34 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:34.780699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:34.800590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ff2aa-18f3-411d-85d3-a8824cd193ac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:34.901576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:34.902216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:34.902526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:34.903746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:34.903883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:34.905930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:34.938731 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2175/4385] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:34 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:34.939709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:34.957635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab639c0-0029-4975-aef9-9fc260bd3873 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:34.965635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:34.966374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:34.966647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:34.967491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:34.968430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:34.970307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:35.001412 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2211/4386] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:12:35.004137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:35.019584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a8a3-a6f3-49c5-ad39-50413a2c9092 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:35.026147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d6dbe-4814-4a84-bf3e-e94c6c1c2244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:35.026770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d6dbe-4814-4a84-bf3e-e94c6c1c2244 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:35.027030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d6dbe-4814-4a84-bf3e-e94c6c1c2244 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:35.027303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d6dbe-4814-4a84-bf3e-e94c6c1c2244 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:35.028353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d6dbe-4814-4a84-bf3e-e94c6c1c2244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:35.030166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d6dbe-4814-4a84-bf3e-e94c6c1c2244 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:35.044448 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2176/4387] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:35.049605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:35.064930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189cbed-54a8-4b29-9843-9c421c590db2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:35.070826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:35.072237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:35.072237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:35.072237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] PATH_INFO: `/v3/projects/8734d2a922bc49999e789453be69a0e3/users/4ec825af35b24524baee61763cbd431a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:35.092757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8734d2a922bc49999e789453be69a0e3, user_id=4ec825af35b24524baee61763cbd431a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:35.095806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:35.127757 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2212/4388] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:35 2022] PUT /identity/v3/projects/8734d2a922bc49999e789453be69a0e3/users/4ec825af35b24524baee61763cbd431a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:35.141443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d11fa-d829-4999-8bc5-f24b916e4121 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:35.141847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d11fa-d829-4999-8bc5-f24b916e4121 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:35.142210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d11fa-d829-4999-8bc5-f24b916e4121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:35.196727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-899d11fa-d829-4999-8bc5-f24b916e4121 None None] MFA Rules not processed for user `4ec825af35b24524baee61763cbd431a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:35.198316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-899d11fa-d829-4999-8bc5-f24b916e4121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:35.343682 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2177/4389] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:35.350315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:35.356080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:35.447451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07c5c-b92c-4e55-9b5a-df8b3511f38b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:35.454375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:35.455303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:35.455811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:35.456133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:35.456816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:35.459582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:35.531238 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2213/4390] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:35 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:35.534893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d11fa-d829-4999-8bc5-f24b916e4121 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:35.555140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d11fa-d829-4999-8bc5-f24b916e4121 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:35.569348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:35.714938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:35.717505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:35.717910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:35.718372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:35.719666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:35.722243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:35.767374 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2178/4391] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:12:35 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:12:35.771521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:35.795350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329549e-0dc9-4347-a7fd-10541db8949f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:35.801183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:35.801822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:35.802149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:35.802407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:35.803113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:35.805550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:35.839416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2214/4392] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:35.843607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:35.860015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515f852-6b9c-4576-aaaa-4e1b90b89e4f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:35.866264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:35.867543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:35.868001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:35.868879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] PATH_INFO: `/v3/projects/56e0e5c372c94fb9a6b7f37fc76505b3/users/2923c807de654b8482b881d611f961b5/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:35.888172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e0e5c372c94fb9a6b7f37fc76505b3, user_id=2923c807de654b8482b881d611f961b5, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:35.891297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:35.917772 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2179/4393] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:35 2022] PUT /identity/v3/projects/56e0e5c372c94fb9a6b7f37fc76505b3/users/2923c807de654b8482b881d611f961b5/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:35.922589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:35.968143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f5e9-b225-4a0b-9e40-18715d9672bc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:36.070551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b685cce-1d4a-47a4-be7e-2429c033c1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:36.071056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b685cce-1d4a-47a4-be7e-2429c033c1ae admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:36.071349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b685cce-1d4a-47a4-be7e-2429c033c1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:36.071673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b685cce-1d4a-47a4-be7e-2429c033c1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:36.072336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b685cce-1d4a-47a4-be7e-2429c033c1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:36.074644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b685cce-1d4a-47a4-be7e-2429c033c1ae admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:36.092213 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2215/4394] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:12:36.096235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:36.114317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7cdfb-b623-426b-9b56-f97b155f09bb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:36.136536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:36.137587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:36.137799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:36.138079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] PATH_INFO: `/v3/projects/56e0e5c372c94fb9a6b7f37fc76505b3/users/2923c807de654b8482b881d611f961b5/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:36.143347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e0e5c372c94fb9a6b7f37fc76505b3, user_id=2923c807de654b8482b881d611f961b5, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:36.146422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:36.171547 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2180/4395] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:12:36 2022] PUT /identity/v3/projects/56e0e5c372c94fb9a6b7f37fc76505b3/users/2923c807de654b8482b881d611f961b5/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:36.181907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19aca-b958-4602-a3e3-63bf9f22f9c4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:36.182533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19aca-b958-4602-a3e3-63bf9f22f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:36.182931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19aca-b958-4602-a3e3-63bf9f22f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:36.232922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-baf19aca-b958-4602-a3e3-63bf9f22f9c4 None None] MFA Rules not processed for user `2923c807de654b8482b881d611f961b5`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:36.234347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-baf19aca-b958-4602-a3e3-63bf9f22f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:36.449655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:36.465692 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2216/4396] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:36.476076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:36.478706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:36.629028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6344c-e220-46e8-8f09-fa6bb4ff7f61 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:36.640593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:36.641225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:36.642107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:36.642529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:36.668800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:36.674160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:36.701943 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2181/4397] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:36.886190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19aca-b958-4602-a3e3-63bf9f22f9c4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:36.902558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19aca-b958-4602-a3e3-63bf9f22f9c4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:36.911962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:36.912620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:36.913128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:36.915636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:37.072365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:37.074763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:37.103059 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2217/4398] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:40.986438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:40.988023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:41.182999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fb4c4-bc27-431f-b75e-1f4a54ef2373 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:41.191725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '304144ea918141c18e1d33ac8196c821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14123409b8ea43f7a884288a9145b6bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:41.192356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:41.193796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:41.194300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:41.196257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:41.198233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:41.200264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:41.215209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:41.247855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:41.249363 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2182/4399] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:41.261992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab597b0-489e-4d3b-bb9f-ef96519ced20 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:41.268621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '304144ea918141c18e1d33ac8196c821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14123409b8ea43f7a884288a9145b6bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:41.269544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:41.270011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:41.270500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:41.271298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:41.273774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:41.344809 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2218/4400] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:12:41 2022] POST /identity/v3/projects => generated 347 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:12:41.349215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:41.364494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99bce6-6e93-4f12-ad1e-32c853b08d61 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:41.488018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '304144ea918141c18e1d33ac8196c821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14123409b8ea43f7a884288a9145b6bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:41.489021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:41.491247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:41.491247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:41.491247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:12:41.491247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:41.492985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:41.513436 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2183/4401] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:12:42.068069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:42.080043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397600-4e85-456e-9935-e3006254eb4a tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:42.085935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '304144ea918141c18e1d33ac8196c821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14123409b8ea43f7a884288a9145b6bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:42.086985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:42.087246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:42.087989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] PATH_INFO: `/v3/projects/18339d24a96042a8a735b87f4ed70b63` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:42.089766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorizing `identity:delete_project(project_id=18339d24a96042a8a735b87f4ed70b63)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:42.091924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:42.152047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18339d24a96042a8a735b87f4ed70b63', 'request_id': 'req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:42.163996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18339d24a96042a8a735b87f4ed70b63', 'request_id': 'req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:42.171076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18339d24a96042a8a735b87f4ed70b63', 'request_id': 'req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:42.185848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] The token cache is being invalidate because project 18339d24a96042a8a735b87f4ed70b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:12:42.186235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:42.187107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1387117e-e4a4-4a5e-a4f6-b2d4f80d4725 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:12:42.189752 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2219/4402] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:12:42 2022] DELETE /identity/v3/projects/18339d24a96042a8a735b87f4ed70b63 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:12:42.202775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:42.204938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:42.291667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1bd6d-e822-4d60-8150-ecc3dcdf47e0 tempest-RoutersAdminTest-1551194410 tempest-RoutersAdminTest-1551194410-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:42.368836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7037abd-e962-4188-8545-a83ed4ca163a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:42.370063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7037abd-e962-4188-8545-a83ed4ca163a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:42.370448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7037abd-e962-4188-8545-a83ed4ca163a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:42.371173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7037abd-e962-4188-8545-a83ed4ca163a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:42.373577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e7037abd-e962-4188-8545-a83ed4ca163a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:42.513499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7037abd-e962-4188-8545-a83ed4ca163a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:42.515515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7037abd-e962-4188-8545-a83ed4ca163a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:42.527113 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2184/4403] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:43.941407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:43.941931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:43.941931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:43.998253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] MFA Rules not processed for user `241251b91c43444b900922f942681cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:44.002627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:44.124273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:44.170237 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2220/4404] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:44.177426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:44.177990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:44.177990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:44.190559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:44.207414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] MFA Rules not processed for user `241251b91c43444b900922f942681cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:12:44.209012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:44.224460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2185/4405] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:12:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:12:44.232635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:12:44.235071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:12:44.398968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358a7d8-c9a9-4c1b-be5b-1f826a357cf9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:12:44.419709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef771d5-9949-46e4-963c-9e3a1cff340e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:12:44.420291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef771d5-9949-46e4-963c-9e3a1cff340e service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:12:44.420696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef771d5-9949-46e4-963c-9e3a1cff340e service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:12:44.421302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef771d5-9949-46e4-963c-9e3a1cff340e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:12:44.438170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef771d5-9949-46e4-963c-9e3a1cff340e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:12:44.440774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef771d5-9949-46e4-963c-9e3a1cff340e service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:12:44.464503 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2221/4406] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:12:44 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:12:54.201018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:54.456429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=40 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:00.319017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:00.319017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:00.419261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b62f0e-d156-407a-a6a4-f276852cc26f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:00.424955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:00.426467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:00.427268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:00.427815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] PATH_INFO: `/v3/users/976e7f61517e4e3f9f2417078225ca62` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:00.430399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=976e7f61517e4e3f9f2417078225ca62)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:00.432951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:00.519214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '976e7f61517e4e3f9f2417078225ca62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:00.519950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '976e7f61517e4e3f9f2417078225ca62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:00.547835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '976e7f61517e4e3f9f2417078225ca62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:00.560693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '976e7f61517e4e3f9f2417078225ca62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:00.621347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '976e7f61517e4e3f9f2417078225ca62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:00.632353 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2186/4407] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:00 2022] DELETE /identity/v3/users/976e7f61517e4e3f9f2417078225ca62 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:00.637247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:00.637833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:00.637833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:00.660764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:00.662312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:00.751839 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2222/4408] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:00.762448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:00.764354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:00.849814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49557ed5-ea21-4c02-80ce-971ad93b6a81 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:00.854499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:00.856335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:00.856834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:00.857825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:00.872471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:00.874991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:00.904777 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2187/4409] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:01.283535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:01.285476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:01.310360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e24b3-7e06-4875-b559-79c1df0031f6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:01.317331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:01.318214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:01.318541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:01.318820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] PATH_INFO: `/v3/users/b8d6f6c63fe144959139278ca287e2ec` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:01.320330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d6f6c63fe144959139278ca287e2ec)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:01.322783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:01.411484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d6f6c63fe144959139278ca287e2ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:01.412179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d6f6c63fe144959139278ca287e2ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:01.425177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d6f6c63fe144959139278ca287e2ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:01.441113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d6f6c63fe144959139278ca287e2ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:01.441931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-192992b9-5db6-4785-9b38-7c4d85b3c187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d6f6c63fe144959139278ca287e2ec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:01.472677 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2223/4410] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:01 2022] DELETE /identity/v3/users/b8d6f6c63fe144959139278ca287e2ec => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:01.785333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:01.786895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:01.810243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2b8c8-8af8-4f9a-a302-04b936f9342c service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:01.838024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:01.905080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:01.906167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:01.906547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:01.906819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] PATH_INFO: `/v3/projects/b5087b7fd4b3482082e5dd90fdcef18f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:01.908136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5087b7fd4b3482082e5dd90fdcef18f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:01.910524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:01.984362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5087b7fd4b3482082e5dd90fdcef18f', 'request_id': 'req-fb952019-76c3-4b32-b899-5737f0d0da10'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:01.998071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5087b7fd4b3482082e5dd90fdcef18f', 'request_id': 'req-fb952019-76c3-4b32-b899-5737f0d0da10'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.012501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5087b7fd4b3482082e5dd90fdcef18f', 'request_id': 'req-fb952019-76c3-4b32-b899-5737f0d0da10'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.033911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] The token cache is being invalidate because project b5087b7fd4b3482082e5dd90fdcef18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:02.034504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.035354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.038351 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2188/4411] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:01 2022] DELETE /identity/v3/projects/b5087b7fd4b3482082e5dd90fdcef18f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:02.046672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a321-2472-4b13-a8db-d0d3b2123790 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.047412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a321-2472-4b13-a8db-d0d3b2123790 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.047539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a321-2472-4b13-a8db-d0d3b2123790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.094607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-01b0a321-2472-4b13-a8db-d0d3b2123790 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:02.095986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-01b0a321-2472-4b13-a8db-d0d3b2123790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:02.145479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.147685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:02.257821 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2224/4412] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:02.266083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4d4d-c0c3-4a11-ac70-9246108af344 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.266625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4d4d-c0c3-4a11-ac70-9246108af344 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.267191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4d4d-c0c3-4a11-ac70-9246108af344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.284538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb952019-76c3-4b32-b899-5737f0d0da10 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.291014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.291686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.291968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.292246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] PATH_INFO: `/v3/projects/f847bee14ff74dfbafe7c6a199f1b99f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.293693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f847bee14ff74dfbafe7c6a199f1b99f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:02.295894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:02.305931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ac7a4d4d-c0c3-4a11-ac70-9246108af344 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:02.308560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ac7a4d4d-c0c3-4a11-ac70-9246108af344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:02.324537 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2225/4413] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:02.329726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a4d4d-c0c3-4a11-ac70-9246108af344 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.344916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a4d4d-c0c3-4a11-ac70-9246108af344 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.350864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:02.370123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f847bee14ff74dfbafe7c6a199f1b99f', 'request_id': 'req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.385406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f847bee14ff74dfbafe7c6a199f1b99f', 'request_id': 'req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.391824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:02.398068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f847bee14ff74dfbafe7c6a199f1b99f', 'request_id': 'req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.414966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] The token cache is being invalidate because project f847bee14ff74dfbafe7c6a199f1b99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:02.416717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.417638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.420617 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2189/4414] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:02 2022] DELETE /identity/v3/projects/f847bee14ff74dfbafe7c6a199f1b99f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:02.424991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.427339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:02.453852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.454564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.454906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.456124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.456892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:02.457615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:02.459495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:02.479507 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2226/4415] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:02.488053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.490443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:02.506958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd570e-1d0b-42d6-b957-9b8b173407e5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.516293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.517030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.517315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.517589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.518299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:02.520733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:02.525272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb07fd5-068e-4e93-8ef3-eb34f17a00c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.531661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.532640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.533188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.533639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] PATH_INFO: `/v3/users/31e653b281b64ade86e15660158ff07e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.535867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31e653b281b64ade86e15660158ff07e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:02.538857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:02.556148 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2227/4416] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:02 2022] POST /identity/v3/projects => generated 399 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:02.561417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.602305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31e653b281b64ade86e15660158ff07e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.603557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31e653b281b64ade86e15660158ff07e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.614231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31e653b281b64ade86e15660158ff07e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.629238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31e653b281b64ade86e15660158ff07e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.647469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf5ebb-3032-4dc9-bb99-8c4f5de6fdf4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.648620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31e653b281b64ade86e15660158ff07e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:02.656002 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2190/4417] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:02 2022] DELETE /identity/v3/users/31e653b281b64ade86e15660158ff07e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:02.657518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.658394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.659084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.659540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.660843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:02.663679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:02.700928 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2228/4418] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:02 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:02.702668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.704588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:02.820478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bdb8d-807b-4b99-9c75-4cf93358b720 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.828101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.829000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.829404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.830156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.831466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:02.833871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:02.838839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:02.873197 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2191/4419] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:02.874625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.892027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1aa81-888e-4738-8f37-9e468e052914 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.904245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.905596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.905925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.906192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:02.906852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:02.910423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:02.927841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2229/4420] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:02.932993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:02.957096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5394c0b-085c-4e19-b872-12156be3bdd3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:02.967783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ab4fc-0c8c-4de6-bca7-2d7b56c68cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:02.968858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ab4fc-0c8c-4de6-bca7-2d7b56c68cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:02.969142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ab4fc-0c8c-4de6-bca7-2d7b56c68cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:02.969419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ab4fc-0c8c-4de6-bca7-2d7b56c68cc0 admin admin] PATH_INFO: `/v3/projects/de843eb4d6c441b6a84838eba570d99c/users/c988f6e6bccd456395888922739432b0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.043328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ab4fc-0c8c-4de6-bca7-2d7b56c68cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de843eb4d6c441b6a84838eba570d99c, user_id=c988f6e6bccd456395888922739432b0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:03.046775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ab4fc-0c8c-4de6-bca7-2d7b56c68cc0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:03.072431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:03.074510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:03.082856 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2192/4421] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:02 2022] PUT /identity/v3/projects/de843eb4d6c441b6a84838eba570d99c/users/c988f6e6bccd456395888922739432b0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:03.095572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eade7-6a3b-4b61-a17e-38dee3c47102 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.095572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eade7-6a3b-4b61-a17e-38dee3c47102 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.095572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eade7-6a3b-4b61-a17e-38dee3c47102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.111656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef8a6e-c00c-4b26-9d0e-02807052ac89 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:03.152432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-747eade7-6a3b-4b61-a17e-38dee3c47102 None None] MFA Rules not processed for user `c988f6e6bccd456395888922739432b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:03.155170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-747eade7-6a3b-4b61-a17e-38dee3c47102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:03.197812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:03.205094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:03.206033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.206795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.207237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] PATH_INFO: `/v3/projects/688812897dc54f40ad95e885b40a23f5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.209587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688812897dc54f40ad95e885b40a23f5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:03.212906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:03.279880 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2193/4422] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:03.292236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688812897dc54f40ad95e885b40a23f5', 'request_id': 'req-da69c4ba-990c-4084-b717-f967e46ca3e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.297398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747eade7-6a3b-4b61-a17e-38dee3c47102 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:03.310093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747eade7-6a3b-4b61-a17e-38dee3c47102 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:03.310684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688812897dc54f40ad95e885b40a23f5', 'request_id': 'req-da69c4ba-990c-4084-b717-f967e46ca3e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.328243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688812897dc54f40ad95e885b40a23f5', 'request_id': 'req-da69c4ba-990c-4084-b717-f967e46ca3e3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.357560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] The token cache is being invalidate because project 688812897dc54f40ad95e885b40a23f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:03.358167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.359173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-da69c4ba-990c-4084-b717-f967e46ca3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.365850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2230/4423] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:03 2022] DELETE /identity/v3/projects/688812897dc54f40ad95e885b40a23f5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:03.368413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882b25-6924-4d51-bffe-c51d5ac2377c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.368831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882b25-6924-4d51-bffe-c51d5ac2377c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.369190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882b25-6924-4d51-bffe-c51d5ac2377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.396917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b6882b25-6924-4d51-bffe-c51d5ac2377c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:03.399899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b6882b25-6924-4d51-bffe-c51d5ac2377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:03.417445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:03.418249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.418533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.418807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] PATH_INFO: `/v3/users/c988f6e6bccd456395888922739432b0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.420119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c988f6e6bccd456395888922739432b0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:03.422120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:03.504109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c988f6e6bccd456395888922739432b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.512579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c988f6e6bccd456395888922739432b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.533154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c988f6e6bccd456395888922739432b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.540710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2231/4424] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:03.546213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b5d6-9b2f-4a55-903c-32ef273ee268 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.546986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b5d6-9b2f-4a55-903c-32ef273ee268 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.547448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b5d6-9b2f-4a55-903c-32ef273ee268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.553605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c988f6e6bccd456395888922739432b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.557459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:03.575847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26182b56-7a2e-4159-b594-a21cacd4e8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c988f6e6bccd456395888922739432b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:03.610144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cc93b5d6-9b2f-4a55-903c-32ef273ee268 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:03.611469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cc93b5d6-9b2f-4a55-903c-32ef273ee268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:03.615220 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2194/4425] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:03 2022] DELETE /identity/v3/users/c988f6e6bccd456395888922739432b0 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:03.620198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.620536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.620851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.649581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:03.651515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:03.702235 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2232/4426] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:03.703692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93b5d6-9b2f-4a55-903c-32ef273ee268 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:03.719059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93b5d6-9b2f-4a55-903c-32ef273ee268 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:03.729491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:03.730590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.731510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.731890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.732723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:03.733159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:03.736175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:03.751734 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2195/4427] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:03.764248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:03.765163 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2233/4428] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:03.766210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:03.769745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:03.789587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a7ff1-6d94-4e21-b810-a42f75cff989 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:03.797147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:03.797957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.797957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.797957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.798811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:03.801303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:03.827963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:03.857533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f184aa-e1b2-4bd3-91a1-78d393f5c7c9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:03.858816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2234/4429] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:03 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:03.860735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:03.935504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:03.936304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:03.936771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:03.937138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:03.992567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6639d2-f27c-4718-b479-98bb2d209813 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.001039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.002512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.003198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.003657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.004499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.007114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.037620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:04.038947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2235/4430] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:03 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:04.043772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:04.061333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.064133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.064457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acfabb-749e-4315-adf1-26bc90ac9f05 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.071291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.072051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.072412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.073517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.074165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.076047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.091901 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2236/4431] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:13:04.093768 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2196/4432] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:04.093927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:04.113221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b45fc-2728-4b09-9a2f-4fcd48540c1f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.120432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d157cfa-52dc-446d-9d8a-f783216ff717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.120942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157cfa-52dc-446d-9d8a-f783216ff717 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.121228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157cfa-52dc-446d-9d8a-f783216ff717 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.121566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157cfa-52dc-446d-9d8a-f783216ff717 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.122789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d157cfa-52dc-446d-9d8a-f783216ff717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.124965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d157cfa-52dc-446d-9d8a-f783216ff717 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.141683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2237/4433] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:04.142982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:04.156044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c921078-2973-40a7-b7ab-90eed8e4036d service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.161828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.162509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.165815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.166202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] PATH_INFO: `/v3/projects/0c09a67832414ec0ab88df3e0ba92d92/users/28d6b4f6ac624ad6984bad527eb7555d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.198358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c09a67832414ec0ab88df3e0ba92d92, user_id=28d6b4f6ac624ad6984bad527eb7555d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.202665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.223552 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2197/4434] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:04 2022] PUT /identity/v3/projects/0c09a67832414ec0ab88df3e0ba92d92/users/28d6b4f6ac624ad6984bad527eb7555d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:04.228834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.229453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.230185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.253345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:04.254759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:04.387453 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2238/4435] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:04.427705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:04.431157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:04.478103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:04.479667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:04.506127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55fb1c-c8ec-47cd-9121-2879a9c16a68 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.516049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14deb8-c074-41a4-b762-07b635ac8744 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.527533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68283ca-4f61-45e2-8171-01f329ed942f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.528205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68283ca-4f61-45e2-8171-01f329ed942f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.528483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68283ca-4f61-45e2-8171-01f329ed942f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.528981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68283ca-4f61-45e2-8171-01f329ed942f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.531475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.532237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.533388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.533388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] PATH_INFO: `/v3/projects/de843eb4d6c441b6a84838eba570d99c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.534355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de843eb4d6c441b6a84838eba570d99c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.538203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.560430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68283ca-4f61-45e2-8171-01f329ed942f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.562802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68283ca-4f61-45e2-8171-01f329ed942f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.587941 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2198/4436] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:04.661545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de843eb4d6c441b6a84838eba570d99c', 'request_id': 'req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:04.684570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de843eb4d6c441b6a84838eba570d99c', 'request_id': 'req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:04.686603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=5,keystone:SELECT=90,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:04.701074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de843eb4d6c441b6a84838eba570d99c', 'request_id': 'req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:04.720427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] The token cache is being invalidate because project de843eb4d6c441b6a84838eba570d99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:04.720813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:04.722160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df9f00cb-2aaf-46c4-b4e4-eb91f5c13665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:04.729280 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2239/4437] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:04 2022] DELETE /identity/v3/projects/de843eb4d6c441b6a84838eba570d99c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:04.731088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152b2fa-b07a-46d5-9af3-5c0891fd45df None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.731763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152b2fa-b07a-46d5-9af3-5c0891fd45df None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.731763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152b2fa-b07a-46d5-9af3-5c0891fd45df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.762498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4152b2fa-b07a-46d5-9af3-5c0891fd45df None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:04.764018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4152b2fa-b07a-46d5-9af3-5c0891fd45df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:04.854795 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2199/4438] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:04.865214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84822c-e9b2-4eff-b3c3-0f9f6b79528e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.865214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84822c-e9b2-4eff-b3c3-0f9f6b79528e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.865214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84822c-e9b2-4eff-b3c3-0f9f6b79528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.887335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fd84822c-e9b2-4eff-b3c3-0f9f6b79528e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:04.888482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fd84822c-e9b2-4eff-b3c3-0f9f6b79528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:04.898375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2240/4439] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:04.902580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152b2fa-b07a-46d5-9af3-5c0891fd45df None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:04.914280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152b2fa-b07a-46d5-9af3-5c0891fd45df None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.925419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.926168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.926719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.927143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.927879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:04.928184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.931137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:04.952512 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2200/4440] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:04.953893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84822c-e9b2-4eff-b3c3-0f9f6b79528e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:04.977594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84822c-e9b2-4eff-b3c3-0f9f6b79528e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:04.983760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:04.984782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:04.985067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:04.985394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:04.987227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:04.989642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:05.040098 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2241/4441] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:04 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:05.044716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:05.058510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac669328-ae32-48e7-a603-a7a1e77b4be0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:05.164508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:05.185698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:05.186502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:05.186907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:05.187279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:05.188584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:05.191418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:05.229396 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2201/4442] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:05 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:05.234006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:05.255712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061ec13-7212-4b4c-ac61-8af27a9b02af admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:05.262665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:05.263401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:05.263777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:05.264293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:05.265037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:05.268072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:05.284412 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2242/4443] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:05.290652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:05.302501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e410ac4-c316-4cd3-9880-cc79472db8b4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:05.310558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0131c-1041-4ed0-8187-2e3a60593a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:05.311386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0131c-1041-4ed0-8187-2e3a60593a34 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:05.311701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0131c-1041-4ed0-8187-2e3a60593a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:05.312226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0131c-1041-4ed0-8187-2e3a60593a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:05.313011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0131c-1041-4ed0-8187-2e3a60593a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:05.315155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0131c-1041-4ed0-8187-2e3a60593a34 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:05.335820 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2202/4444] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:05.339105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:05.351798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98609569-b584-4ffe-a0ef-29e51d0da13f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:05.358466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:05.359121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:05.359400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:05.359662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] PATH_INFO: `/v3/projects/e2e00c90216b4d1b80ac7892ee23550f/users/279d3662f60c4a979c22ee3e8fa92865/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:05.402528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e00c90216b4d1b80ac7892ee23550f, user_id=279d3662f60c4a979c22ee3e8fa92865, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:05.407058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:05.429985 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2243/4445] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:05 2022] PUT /identity/v3/projects/e2e00c90216b4d1b80ac7892ee23550f/users/279d3662f60c4a979c22ee3e8fa92865/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:05.435498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a581bf-8568-462b-bd81-afc074e4639e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:05.436062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a581bf-8568-462b-bd81-afc074e4639e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:05.436415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a581bf-8568-462b-bd81-afc074e4639e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:05.458454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f0a581bf-8568-462b-bd81-afc074e4639e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:05.460439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f0a581bf-8568-462b-bd81-afc074e4639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:05.557542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2203/4446] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:05.584508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:05.588358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:05.738183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af1872-8c57-45d1-9908-dbcab86e4783 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:05.743050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45948-fa06-4e59-9267-52afb9948408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:05.743761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45948-fa06-4e59-9267-52afb9948408 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:05.744129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45948-fa06-4e59-9267-52afb9948408 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:05.744452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45948-fa06-4e59-9267-52afb9948408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:05.763740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f45948-fa06-4e59-9267-52afb9948408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:05.765774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f45948-fa06-4e59-9267-52afb9948408 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:05.782039 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2244/4447] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:11.869670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2efca-defd-4538-abcf-889011975af7 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:11.870258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2efca-defd-4538-abcf-889011975af7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:11.870258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2efca-defd-4538-abcf-889011975af7 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:11.871667 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2204/4448] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:13:11.883084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d53cf-a954-487d-80f0-ab8afb97445e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:11.883807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d53cf-a954-487d-80f0-ab8afb97445e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:11.884222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d53cf-a954-487d-80f0-ab8afb97445e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:12.002639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-561d53cf-a954-487d-80f0-ab8afb97445e None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:12.004196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-561d53cf-a954-487d-80f0-ab8afb97445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:12.019526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2245/4449] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:13:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:12.046606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2efca-defd-4538-abcf-889011975af7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:12.064334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2efca-defd-4538-abcf-889011975af7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:12.076688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:12.077478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:12.077866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:12.078236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:12.097488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:12.099522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:12.114218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2205/4450] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:17.412737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b76447-2302-4f8b-a62f-56e07ddfcb4e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:17.413247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b76447-2302-4f8b-a62f-56e07ddfcb4e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:17.413358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b76447-2302-4f8b-a62f-56e07ddfcb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:17.486408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-18b76447-2302-4f8b-a62f-56e07ddfcb4e None None] MFA Rules not processed for user `28d6b4f6ac624ad6984bad527eb7555d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:17.488263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18b76447-2302-4f8b-a62f-56e07ddfcb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:17.629378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:17.670910 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2246/4451] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:17.682444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:17.688761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:17.864362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6261ff-cdde-43a1-9ec7-ad75e66e2e37 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:17.898667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:17.902509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:17.904873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:17.905167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:17.939956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:17.942965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:17.960680 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2206/4452] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:20.865994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:20.866854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:20.867324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:20.950841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] MFA Rules not processed for user `279d3662f60c4a979c22ee3e8fa92865`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:20.952193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:21.176256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:21.178284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:21.187254 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2247/4453] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:21.200450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee5cfb-04f5-4fa6-bed9-6e5b23e997ee service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:21.217733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ffb49-7c76-4a13-ae4f-f980f7159d2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:21.218508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ffb49-7c76-4a13-ae4f-f980f7159d2e service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:21.218790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ffb49-7c76-4a13-ae4f-f980f7159d2e service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:21.219060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ffb49-7c76-4a13-ae4f-f980f7159d2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:21.238523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ffb49-7c76-4a13-ae4f-f980f7159d2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:21.240225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ffb49-7c76-4a13-ae4f-f980f7159d2e service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:21.270251 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2207/4454] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:13:23.273032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:23.275045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:23.304268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195063b-6d3e-46d4-bf4a-972d49c3f286 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:23.311834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:23.312718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:23.313140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:23.313561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] PATH_INFO: `/v3/users/4ec825af35b24524baee61763cbd431a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:23.315834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec825af35b24524baee61763cbd431a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:23.322186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:23.432103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec825af35b24524baee61763cbd431a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:23.432944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec825af35b24524baee61763cbd431a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:23.458927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec825af35b24524baee61763cbd431a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:23.469212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec825af35b24524baee61763cbd431a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:23.474789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec825af35b24524baee61763cbd431a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:23.527774 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2248/4455] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:23 2022] DELETE /identity/v3/users/4ec825af35b24524baee61763cbd431a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:23.532701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:23.533243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:23.533339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:23.575576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:23.577168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:23.664060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:23.731859 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2208/4456] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:23.787035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:23.789640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:23.903218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:23.913575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61fed-a80e-4524-8748-739eb36b5d44 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:23.921384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:23.922373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:23.922916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:23.923698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:23.941350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:23.943130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:23.957785 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2249/4457] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:25.548443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:25.551690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:25.578997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1167db-8c99-4c23-bb4b-6468cda190a2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:25.592275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:25.592275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:25.593365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:25.595219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] PATH_INFO: `/v3/projects/8734d2a922bc49999e789453be69a0e3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:25.598563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8734d2a922bc49999e789453be69a0e3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:25.601005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:25.744337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8734d2a922bc49999e789453be69a0e3', 'request_id': 'req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:25.752798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8734d2a922bc49999e789453be69a0e3', 'request_id': 'req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:25.766731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8734d2a922bc49999e789453be69a0e3', 'request_id': 'req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:25.795007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] The token cache is being invalidate because project 8734d2a922bc49999e789453be69a0e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:25.795469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:25.796866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f88c9fa2-d89f-40d9-9612-1ea4d3d4bd23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:25.803575 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2209/4458] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:25 2022] DELETE /identity/v3/projects/8734d2a922bc49999e789453be69a0e3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:25.811913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:25.829398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcc5c2-99af-45ad-ba29-98cdc3daebc5 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:25.941354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:25.942364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:25.942907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:25.943419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] PATH_INFO: `/v3/users/2923c807de654b8482b881d611f961b5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:25.945058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2923c807de654b8482b881d611f961b5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:25.947905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:26.111070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2923c807de654b8482b881d611f961b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:26.112218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2923c807de654b8482b881d611f961b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:26.140820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2923c807de654b8482b881d611f961b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:26.170079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2923c807de654b8482b881d611f961b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:26.171071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2923c807de654b8482b881d611f961b5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:26.221428 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2250/4459] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:25 2022] DELETE /identity/v3/users/2923c807de654b8482b881d611f961b5 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:26.702833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35619c-c3f1-4148-95de-d8851fa08403 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:26.703332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35619c-c3f1-4148-95de-d8851fa08403 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:26.703550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35619c-c3f1-4148-95de-d8851fa08403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:26.743519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ad35619c-c3f1-4148-95de-d8851fa08403 None None] MFA Rules not processed for user `279d3662f60c4a979c22ee3e8fa92865`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:26.746017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ad35619c-c3f1-4148-95de-d8851fa08403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:26.898134 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2210/4460] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:27.216999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:27.219024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:27.432771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf78794-49b8-4088-a4ba-e9b58e2109b7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:27.442574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:27.450313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:27.450605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:27.451396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] PATH_INFO: `/v3/projects/56e0e5c372c94fb9a6b7f37fc76505b3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:27.452740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e0e5c372c94fb9a6b7f37fc76505b3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:27.455259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:27.471959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:27.551184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e0e5c372c94fb9a6b7f37fc76505b3', 'request_id': 'req-8cbc801a-94db-46bc-a335-69bf9e0c28d2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:27.565727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e0e5c372c94fb9a6b7f37fc76505b3', 'request_id': 'req-8cbc801a-94db-46bc-a335-69bf9e0c28d2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:27.577124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e0e5c372c94fb9a6b7f37fc76505b3', 'request_id': 'req-8cbc801a-94db-46bc-a335-69bf9e0c28d2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:27.599393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] The token cache is being invalidate because project 56e0e5c372c94fb9a6b7f37fc76505b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:27.599698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:27.602013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cbc801a-94db-46bc-a335-69bf9e0c28d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:27.607883 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2251/4461] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:27 2022] DELETE /identity/v3/projects/56e0e5c372c94fb9a6b7f37fc76505b3 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:27.609283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8b06a-a998-4bb6-b566-fb0b16a7b9f1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:27.609883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8b06a-a998-4bb6-b566-fb0b16a7b9f1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:27.610459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8b06a-a998-4bb6-b566-fb0b16a7b9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:27.664053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c9a8b06a-a998-4bb6-b566-fb0b16a7b9f1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:27.666095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c9a8b06a-a998-4bb6-b566-fb0b16a7b9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:27.781301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2211/4462] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:27.789708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0796f77-b034-4e5c-858b-ab118a202e82 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:27.790042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0796f77-b034-4e5c-858b-ab118a202e82 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:27.790292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0796f77-b034-4e5c-858b-ab118a202e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:27.818683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a0796f77-b034-4e5c-858b-ab118a202e82 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:27.820540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a0796f77-b034-4e5c-858b-ab118a202e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:27.833220 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2252/4463] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:27.834869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8b06a-a998-4bb6-b566-fb0b16a7b9f1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:27.840438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:27.848007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8b06a-a998-4bb6-b566-fb0b16a7b9f1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:27.855807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:27.856624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:27.856910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:27.857184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:27.857919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:27.858410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:27.862763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:27.884722 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2212/4464] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:27.888990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0796f77-b034-4e5c-858b-ab118a202e82 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:27.900317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0796f77-b034-4e5c-858b-ab118a202e82 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:27.907063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:27.907911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:27.908308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:27.908609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:27.909488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:27.911590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:27.949354 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2253/4465] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:27 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:27.951279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:27.968688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eb812-2c96-4d13-83d2-3b090aa22e92 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.044766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.047083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:28.094567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.095621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.096138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.097000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.098594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.103292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.145323 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2213/4466] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:27 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:28.147346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.169072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846d7b9-681f-441f-8e24-da531d5cbe6a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.176362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.178064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.179036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.179831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.180709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.182622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.185180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef5284-a068-40d1-bffb-c49e05e92b43 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.196071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.197172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.197530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.197972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] PATH_INFO: `/v3/users/015d442ea03c434bb63aa9ba9b537ce8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.199195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=015d442ea03c434bb63aa9ba9b537ce8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.199741 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2214/4467] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:28.203207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.204630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.218876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4b0a-865d-40d0-8891-a496a558382a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.226154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.227407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.228118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.228755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.230884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.233077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.269439 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2215/4468] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:28.270812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.285726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5dafe-75bf-4efe-bdb7-c25c0d11257c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.291429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65286b4-4d23-4cd9-8669-2bd87f2d4e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.292186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65286b4-4d23-4cd9-8669-2bd87f2d4e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.292536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65286b4-4d23-4cd9-8669-2bd87f2d4e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.292912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65286b4-4d23-4cd9-8669-2bd87f2d4e96 admin admin] PATH_INFO: `/v3/projects/ca4783b61c2e414fb7a72e705669e0d8/users/09b7a4956e0346689c30bb87d87ec8ae/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.316080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65286b4-4d23-4cd9-8669-2bd87f2d4e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4783b61c2e414fb7a72e705669e0d8, user_id=09b7a4956e0346689c30bb87d87ec8ae, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.321727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65286b4-4d23-4cd9-8669-2bd87f2d4e96 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.327074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '015d442ea03c434bb63aa9ba9b537ce8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:28.328158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '015d442ea03c434bb63aa9ba9b537ce8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:28.341504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '015d442ea03c434bb63aa9ba9b537ce8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:28.342967 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2216/4469] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:28 2022] PUT /identity/v3/projects/ca4783b61c2e414fb7a72e705669e0d8/users/09b7a4956e0346689c30bb87d87ec8ae/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:28.354773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '015d442ea03c434bb63aa9ba9b537ce8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:28.355388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.355935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.356218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.356869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '015d442ea03c434bb63aa9ba9b537ce8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:28.372232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:28.397896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] MFA Rules not processed for user `09b7a4956e0346689c30bb87d87ec8ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:28.398622 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2254/4470] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:28 2022] DELETE /identity/v3/users/015d442ea03c434bb63aa9ba9b537ce8 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:28.400146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:28.451467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.453242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:28.527542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2217/4471] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:28.530207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.533669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:28.546234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7f71-1d8a-4b06-9143-b97d354c4c02 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.551048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.551682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.551948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.552215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.554321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:28.566940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.568919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.578313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:28.579495 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2255/4472] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:28.639610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e339a8-90ff-4cc7-b8b1-cc541a8bfc26 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.645652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.646293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.646573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.646849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.647555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.649586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.686287 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2218/4473] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:28 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:28.687594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.700699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98262b0c-fdef-46c0-904b-aacab99a7f96 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.770965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.771526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.771787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.772184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.773020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.775475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.805262 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2256/4474] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:28 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:28.807499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.819639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87cffd-c0f5-46ab-b3ca-e38b274b9d29 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.826293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.826929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.827206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.827480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.828138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.829945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.848761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2219/4475] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:28.849801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.861818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918d581-4a63-4b50-9d5c-6d8c1fd6ca0d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:28.867271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:28.867946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:28.868224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:28.868498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] PATH_INFO: `/v3/projects/5baede7af6b541cb8c886f83ab04570c/users/a29366268adf416eb7a6f3377817d1f6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:28.886605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5baede7af6b541cb8c886f83ab04570c, user_id=a29366268adf416eb7a6f3377817d1f6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:28.889345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:28.908806 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2257/4476] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:28 2022] PUT /identity/v3/projects/5baede7af6b541cb8c886f83ab04570c/users/a29366268adf416eb7a6f3377817d1f6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:28.910425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:28.923297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f402d-65f1-47a1-93d4-ddf13ec35858 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:29.000427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474abc7-e685-410f-9e49-59448bbfcd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:29.001242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474abc7-e685-410f-9e49-59448bbfcd3b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.001526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474abc7-e685-410f-9e49-59448bbfcd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.001943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474abc7-e685-410f-9e49-59448bbfcd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.002791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474abc7-e685-410f-9e49-59448bbfcd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:29.005306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474abc7-e685-410f-9e49-59448bbfcd3b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:29.025008 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2220/4477] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:29.026533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:29.039747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383602-967e-422a-8515-cd1944f877f1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:29.058203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:29.058902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.059177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.059450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] PATH_INFO: `/v3/projects/5baede7af6b541cb8c886f83ab04570c/users/a29366268adf416eb7a6f3377817d1f6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.061544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5baede7af6b541cb8c886f83ab04570c, user_id=a29366268adf416eb7a6f3377817d1f6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:29.064498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:29.098659 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2258/4478] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:29 2022] PUT /identity/v3/projects/5baede7af6b541cb8c886f83ab04570c/users/a29366268adf416eb7a6f3377817d1f6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:29.104177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.104509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.104804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.113106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:29.114880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:29.169882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] MFA Rules not processed for user `a29366268adf416eb7a6f3377817d1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:29.171152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:29.243492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:29.287204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb08603-57c4-4046-ad9f-5e582066b408 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:29.306156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:29.309598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.309978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.310382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] PATH_INFO: `/v3/projects/0a1ced5ea0884fc99d7644f37ca3f376` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.312759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a1ced5ea0884fc99d7644f37ca3f376)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:29.316564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:29.372761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2221/4479] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:29.381232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:29.384772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:29.416014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a1ced5ea0884fc99d7644f37ca3f376', 'request_id': 'req-a944640a-e781-4104-b752-5edf1bc36ccb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:29.426773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:29.434205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a1ced5ea0884fc99d7644f37ca3f376', 'request_id': 'req-a944640a-e781-4104-b752-5edf1bc36ccb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:29.461378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a1ced5ea0884fc99d7644f37ca3f376', 'request_id': 'req-a944640a-e781-4104-b752-5edf1bc36ccb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:29.504949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] The token cache is being invalidate because project 0a1ced5ea0884fc99d7644f37ca3f376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:29.505283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:29.505974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a944640a-e781-4104-b752-5edf1bc36ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:29.511573 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2259/4480] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:29 2022] DELETE /identity/v3/projects/0a1ced5ea0884fc99d7644f37ca3f376 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:29.513202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7053d-abf1-475a-ab49-2984d58baac3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.513556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7053d-abf1-475a-ab49-2984d58baac3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.517283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e89a1-7a2c-4fc1-b7bb-a7b5a33c5377 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:29.518681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7053d-abf1-475a-ab49-2984d58baac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.543556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e1b7053d-abf1-475a-ab49-2984d58baac3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:29.544992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e1b7053d-abf1-475a-ab49-2984d58baac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:29.676909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:29.679875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.680161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.680440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.684505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:29.688503 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2260/4481] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:29.696778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3129e27-0622-417b-b4ee-f59a26e448f3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.697900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3129e27-0622-417b-b4ee-f59a26e448f3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.698178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3129e27-0622-417b-b4ee-f59a26e448f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.741891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a3129e27-0622-417b-b4ee-f59a26e448f3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:29.742964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3129e27-0622-417b-b4ee-f59a26e448f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:29.759804 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2261/4482] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:29.765711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3129e27-0622-417b-b4ee-f59a26e448f3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:29.796194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3129e27-0622-417b-b4ee-f59a26e448f3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:29.811828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:29.812747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.814743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.815026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.815706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:29.816009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:29.817708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:29.839448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:29.842746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:29.851549 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2262/4483] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:29.857010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:29.875118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f447b-af8b-4b54-afa8-20e023426b55 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:29.880638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:29.881631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:29.881631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:29.881862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:29.882609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:29.883902 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2222/4484] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:13:29.884505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:29.991116 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2263/4485] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:29 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:29.991400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:30.030805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa64b22-e8bb-47f7-87df-5e2153e8a118 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:30.146655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:30.147579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:30.148039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:30.148487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:30.161387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:30.166199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:30.210956 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2223/4486] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:29 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:30.215157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:30.234101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a97a-4396-4a4f-b182-2f4fbd900978 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:30.242109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:30.242552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:30.242832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:30.243099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:30.243790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:30.245507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:30.258377 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2264/4487] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:30.263175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:30.281134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a63e8-fdb4-48fe-a1ef-25c5b1132e94 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:30.286351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75e65f-f4bb-4b3f-a129-82cc3a01bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:30.288151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75e65f-f4bb-4b3f-a129-82cc3a01bb61 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:30.288987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75e65f-f4bb-4b3f-a129-82cc3a01bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:30.289271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75e65f-f4bb-4b3f-a129-82cc3a01bb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:30.289933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75e65f-f4bb-4b3f-a129-82cc3a01bb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:30.293217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75e65f-f4bb-4b3f-a129-82cc3a01bb61 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:30.307882 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2224/4488] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:30.311818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:30.328636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492c8e2-771a-4ed6-90fd-98ead99dd228 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:30.342465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:30.343285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:30.343633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:30.344110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] PATH_INFO: `/v3/projects/326c35c0bc52463e95a56ba3371a276a/users/a5e050c7389844a78199faef5a0c9da4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:30.394838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=326c35c0bc52463e95a56ba3371a276a, user_id=a5e050c7389844a78199faef5a0c9da4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:30.399269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:30.417621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:30.442233 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2265/4489] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:30 2022] PUT /identity/v3/projects/326c35c0bc52463e95a56ba3371a276a/users/a5e050c7389844a78199faef5a0c9da4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:30.448253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:30.449264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:30.449548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:30.459964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:30.503708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:30.505141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:30.622803 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2225/4490] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:30.638663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:30.641900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:30.757638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033116a-7911-44a9-9dd5-051aa96d6ccc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:30.767911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0a48d-62ae-4821-b485-07c4d623bf7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:30.768821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a48d-62ae-4821-b485-07c4d623bf7a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:30.768821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a48d-62ae-4821-b485-07c4d623bf7a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:30.769105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a48d-62ae-4821-b485-07c4d623bf7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:30.784991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0a48d-62ae-4821-b485-07c4d623bf7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:30.844164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0a48d-62ae-4821-b485-07c4d623bf7a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:30.844164 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2266/4491] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:32.017555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:32.019490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:32.033922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2fcfb-8d6b-4e1a-bb02-759a0fb29a03 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:32.041864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:32.042655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:32.043006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:32.043349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] PATH_INFO: `/v3/users/09b7a4956e0346689c30bb87d87ec8ae` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:32.045175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b7a4956e0346689c30bb87d87ec8ae)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:32.047329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:32.109909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b7a4956e0346689c30bb87d87ec8ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:32.110598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b7a4956e0346689c30bb87d87ec8ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:32.121715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b7a4956e0346689c30bb87d87ec8ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:32.135683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b7a4956e0346689c30bb87d87ec8ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:32.153218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b7a4956e0346689c30bb87d87ec8ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:32.160760 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2226/4492] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:32 2022] DELETE /identity/v3/users/09b7a4956e0346689c30bb87d87ec8ae => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:32.165766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:32.166306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:32.166635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:32.189182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:32.190602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:32.266856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2267/4493] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:32.317355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:32.319994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:32.402595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a98c9-8dfe-4c5a-9437-168eb3f47fe0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:32.422659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:32.422659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:32.422659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:32.422659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:32.457497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:32.459476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:32.486166 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2227/4494] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:33.987934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:33.989984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:34.006290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14086f9-471d-41d7-8748-cd64fb333193 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:34.013441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:34.014126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:34.014448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:34.014721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] PATH_INFO: `/v3/projects/ca4783b61c2e414fb7a72e705669e0d8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:34.015965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4783b61c2e414fb7a72e705669e0d8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:34.018172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:34.109570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4783b61c2e414fb7a72e705669e0d8', 'request_id': 'req-6e549994-eada-49de-941b-3d56a48aae70'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.139376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4783b61c2e414fb7a72e705669e0d8', 'request_id': 'req-6e549994-eada-49de-941b-3d56a48aae70'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.155432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4783b61c2e414fb7a72e705669e0d8', 'request_id': 'req-6e549994-eada-49de-941b-3d56a48aae70'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.171594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] The token cache is being invalidate because project ca4783b61c2e414fb7a72e705669e0d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:34.172295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.173122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.175118 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2268/4495] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:33 2022] DELETE /identity/v3/projects/ca4783b61c2e414fb7a72e705669e0d8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:34.179768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:34.191721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f74bf-eba4-430c-b0c6-02898667d9fb service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:34.211357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:34.357996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:34.358475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:34.358933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:34.359041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] PATH_INFO: `/v3/users/a29366268adf416eb7a6f3377817d1f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:34.361313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29366268adf416eb7a6f3377817d1f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:34.363449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:34.468077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29366268adf416eb7a6f3377817d1f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.468899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29366268adf416eb7a6f3377817d1f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.484270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29366268adf416eb7a6f3377817d1f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.498268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29366268adf416eb7a6f3377817d1f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.522264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50345827-0ca6-4d3c-b3c1-eda491f61715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29366268adf416eb7a6f3377817d1f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:34.536008 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2228/4496] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:34 2022] DELETE /identity/v3/users/a29366268adf416eb7a6f3377817d1f6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:36.680784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:36.682601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:36.696848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e549994-eada-49de-941b-3d56a48aae70 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:36.748481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:36.794943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:36.795596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:36.795875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:36.796432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] PATH_INFO: `/v3/projects/5baede7af6b541cb8c886f83ab04570c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:36.797973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5baede7af6b541cb8c886f83ab04570c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:36.800174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:36.862303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5baede7af6b541cb8c886f83ab04570c', 'request_id': 'req-3d49cb92-05c5-437e-82a6-f19ebb08149b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:36.875619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5baede7af6b541cb8c886f83ab04570c', 'request_id': 'req-3d49cb92-05c5-437e-82a6-f19ebb08149b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:36.885087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5baede7af6b541cb8c886f83ab04570c', 'request_id': 'req-3d49cb92-05c5-437e-82a6-f19ebb08149b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:36.901744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] The token cache is being invalidate because project 5baede7af6b541cb8c886f83ab04570c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:36.902045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:36.902789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d49cb92-05c5-437e-82a6-f19ebb08149b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:36.905472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2269/4497] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:36 2022] DELETE /identity/v3/projects/5baede7af6b541cb8c886f83ab04570c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:36.914507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c99861-09e4-48f2-8ee1-5e654f313945 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:36.914963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c99861-09e4-48f2-8ee1-5e654f313945 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:36.916303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c99861-09e4-48f2-8ee1-5e654f313945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:36.949719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b2c99861-09e4-48f2-8ee1-5e654f313945 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:36.949719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b2c99861-09e4-48f2-8ee1-5e654f313945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:37.051583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2229/4498] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:37.054281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaa839-6d2f-4d9d-9636-eb2d61e98418 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.055468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaa839-6d2f-4d9d-9636-eb2d61e98418 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.056550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaa839-6d2f-4d9d-9636-eb2d61e98418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.094343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a8aaa839-6d2f-4d9d-9636-eb2d61e98418 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:37.096523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a8aaa839-6d2f-4d9d-9636-eb2d61e98418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:37.113837 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2270/4499] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:37.118523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c99861-09e4-48f2-8ee1-5e654f313945 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:37.130665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c99861-09e4-48f2-8ee1-5e654f313945 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:37.135847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:37.136775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.137056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.137327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.137988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:37.138288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:37.139943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:37.160781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aaa839-6d2f-4d9d-9636-eb2d61e98418 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:37.166814 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2230/4500] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:37.171389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aaa839-6d2f-4d9d-9636-eb2d61e98418 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:37.177888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:37.179114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.179114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.180065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.180926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:37.183032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:37.238635 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2271/4501] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:37 2022] POST /identity/v3/projects => generated 393 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:37.240756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:37.256175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81b66-7847-470a-bed8-4ed5d0aaa886 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:37.330411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:37.401301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:37.402069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.402442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.402895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.403655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:37.405943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:37.457604 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2231/4502] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:37 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:37.458475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:37.474806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6880b-3fc1-4dd8-9cb1-ee061a8ec006 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:37.487310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:37.489067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.490004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.490848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.492793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:37.495436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:37.512451 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2272/4503] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:13:37.517151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:37.534759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a871de-ec45-40e2-ad1f-67a5a3590d64 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:37.541713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63ff1-c287-47be-8cba-55c684d1e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:37.542703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63ff1-c287-47be-8cba-55c684d1e5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.543064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63ff1-c287-47be-8cba-55c684d1e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.543470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63ff1-c287-47be-8cba-55c684d1e5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.544294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c63ff1-c287-47be-8cba-55c684d1e5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:37.546600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c63ff1-c287-47be-8cba-55c684d1e5e9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:37.563285 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2232/4504] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:37.568782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:37.590539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d3f58-4e28-41de-85f8-414c5f2e7fc1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:37.597586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:37.598469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.598845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.599526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] PATH_INFO: `/v3/projects/18102a849cbe4b16a0d874e0240630de/users/c258dd49c93c48aca11be52aadf62ae9/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.637971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18102a849cbe4b16a0d874e0240630de, user_id=c258dd49c93c48aca11be52aadf62ae9, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:37.641037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:37.672881 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2273/4505] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:37 2022] PUT /identity/v3/projects/18102a849cbe4b16a0d874e0240630de/users/c258dd49c93c48aca11be52aadf62ae9/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:37.675775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb14e-1062-4a32-98e4-57dc07181a96 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.676066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb14e-1062-4a32-98e4-57dc07181a96 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.676342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb14e-1062-4a32-98e4-57dc07181a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.711908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f05cb14e-1062-4a32-98e4-57dc07181a96 None None] MFA Rules not processed for user `c258dd49c93c48aca11be52aadf62ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:37.713500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f05cb14e-1062-4a32-98e4-57dc07181a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:37.832270 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2233/4506] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:37.835063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:37.953325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:37.979156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ee512-9f5c-4235-b96a-d7105051d63b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:37.993733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:37.995073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:37.995525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:37.995910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:37.997271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:37.999633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:38.067577 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2274/4507] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:37 2022] POST /identity/v3/projects => generated 389 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:38.072043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb14e-1062-4a32-98e4-57dc07181a96 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:38.084200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb14e-1062-4a32-98e4-57dc07181a96 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:38.177188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:38.177910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:38.178458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:38.178892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:38.179569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:38.181716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:38.241946 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2234/4508] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:38 2022] POST /identity/v3/users => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:38.247660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:38.267191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d46c-1384-4465-94b7-817a632372fa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:38.282892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:38.283865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:38.286138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:38.287735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:38.289885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:38.298103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:38.337066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:38.337644 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2275/4509] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:38.354430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae56d7-bfe1-4085-b3e9-b2be81a8089c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:38.372293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:38.373231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:38.373681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:38.374119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] PATH_INFO: `/v3/projects/e9b5dab97d2a4b4dba6d2c7350ad0613/users/e1fe15f7b53a46e7840fd8342266ce2b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:38.417606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b5dab97d2a4b4dba6d2c7350ad0613, user_id=e1fe15f7b53a46e7840fd8342266ce2b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:38.419770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:38.434529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:38.447791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:38.448427 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2235/4510] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:38 2022] PUT /identity/v3/projects/e9b5dab97d2a4b4dba6d2c7350ad0613/users/e1fe15f7b53a46e7840fd8342266ce2b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:38.461012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c192388-4ddb-40fe-9f94-88b8ec68ecb0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:38.635593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1322c-8ed1-4bc4-952f-31e64e1c294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:38.636444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1322c-8ed1-4bc4-952f-31e64e1c294b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:38.637887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1322c-8ed1-4bc4-952f-31e64e1c294b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:38.638275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1322c-8ed1-4bc4-952f-31e64e1c294b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:38.639043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1322c-8ed1-4bc4-952f-31e64e1c294b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:38.642253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1322c-8ed1-4bc4-952f-31e64e1c294b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:38.661525 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2276/4511] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:38.666669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:38.679569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff64f01-8383-4f3f-8a18-0b8921346257 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:38.686115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:38.686861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:38.688588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:38.688991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] PATH_INFO: `/v3/projects/e9b5dab97d2a4b4dba6d2c7350ad0613/users/e1fe15f7b53a46e7840fd8342266ce2b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:38.694900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b5dab97d2a4b4dba6d2c7350ad0613, user_id=e1fe15f7b53a46e7840fd8342266ce2b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:38.706028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:38.734607 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2236/4512] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:38 2022] PUT /identity/v3/projects/e9b5dab97d2a4b4dba6d2c7350ad0613/users/e1fe15f7b53a46e7840fd8342266ce2b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:38.746037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:38.746381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:38.746580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:38.783925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] MFA Rules not processed for user `e1fe15f7b53a46e7840fd8342266ce2b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:38.787022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:38.937385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2277/4513] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:38 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:38.949307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:38.951798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:39.099127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2322c52-743e-4444-9894-789d343af97c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:39.113560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390ee80-73fa-4c72-ade2-fecb5a6d8310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:39.114673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ee80-73fa-4c72-ade2-fecb5a6d8310 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:39.114673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ee80-73fa-4c72-ade2-fecb5a6d8310 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:39.114854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ee80-73fa-4c72-ade2-fecb5a6d8310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:39.126609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390ee80-73fa-4c72-ade2-fecb5a6d8310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:39.128850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390ee80-73fa-4c72-ade2-fecb5a6d8310 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:39.141986 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2237/4514] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:39.248372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:39.249943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:39.269577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7ed45-d0d5-4d59-bc0b-288d4676b263 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:39.274579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:39.275489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:39.276214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:39.276821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:39.397227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:39.432463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:39.434254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:39.455967 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2278/4515] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:43.667126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48403304-de7c-4428-a1c1-1b3a0670cdc9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:43.667618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48403304-de7c-4428-a1c1-1b3a0670cdc9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:43.667707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48403304-de7c-4428-a1c1-1b3a0670cdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:43.768310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-48403304-de7c-4428-a1c1-1b3a0670cdc9 None None] MFA Rules not processed for user `a5e050c7389844a78199faef5a0c9da4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:43.771437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-48403304-de7c-4428-a1c1-1b3a0670cdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:43.928222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:43.957415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2238/4516] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:43.971458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943efef1-da44-446a-ad9e-92381a87526f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:43.971892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943efef1-da44-446a-ad9e-92381a87526f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:43.972526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943efef1-da44-446a-ad9e-92381a87526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:43.997731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-943efef1-da44-446a-ad9e-92381a87526f None None] MFA Rules not processed for user `a5e050c7389844a78199faef5a0c9da4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:44.000199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-943efef1-da44-446a-ad9e-92381a87526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:44.016242 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2239/4517] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:44.048210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95db7c-9464-4c11-89e1-247f1c58bcea service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:44.054057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:44.054998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:44.055547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:44.055957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] PATH_INFO: `/v3/users/c258dd49c93c48aca11be52aadf62ae9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:44.057540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c258dd49c93c48aca11be52aadf62ae9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:44.059617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:44.131026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c258dd49c93c48aca11be52aadf62ae9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:44.131826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c258dd49c93c48aca11be52aadf62ae9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:44.150647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c258dd49c93c48aca11be52aadf62ae9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:44.161211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c258dd49c93c48aca11be52aadf62ae9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:44.163608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3ad8235b-5ada-4ca4-a321-1d41080abcf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c258dd49c93c48aca11be52aadf62ae9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:44.174392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943efef1-da44-446a-ad9e-92381a87526f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:44.175770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-943efef1-da44-446a-ad9e-92381a87526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:44.214779 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2279/4518] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:43 2022] DELETE /identity/v3/users/c258dd49c93c48aca11be52aadf62ae9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:44.220340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:44.220866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:44.222410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:44.264401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:44.266137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:44.374545 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2280/4519] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:44.382214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943efef1-da44-446a-ad9e-92381a87526f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:44.390169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:44.392505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:44.401065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:44.407529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:44.521319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc2c5-85cf-4d88-b456-6a74bbd8f708 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:44.528053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:44.530299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:44.530736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:44.531711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:44.532015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:44.533437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:44.534246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:44.534847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:44.535736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:44.556275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:44.558016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:44.582370 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2281/4520] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:44.652243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:44.654511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:44.668842 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2240/4521] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:13:45.305576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:45.307672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:45.343137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e0611-89e0-4601-923c-e115ca2d33dc service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:45.357944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:45.358940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:45.359430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:45.359830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] PATH_INFO: `/v3/projects/18102a849cbe4b16a0d874e0240630de` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:45.361519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18102a849cbe4b16a0d874e0240630de)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:45.363930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:45.466565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18102a849cbe4b16a0d874e0240630de', 'request_id': 'req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.486723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18102a849cbe4b16a0d874e0240630de', 'request_id': 'req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.505589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18102a849cbe4b16a0d874e0240630de', 'request_id': 'req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.530100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] The token cache is being invalidate because project 18102a849cbe4b16a0d874e0240630de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:45.530693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.531871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.534741 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2282/4522] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:45 2022] DELETE /identity/v3/projects/18102a849cbe4b16a0d874e0240630de => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:45.538923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:45.551777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f704245-22da-4223-b0f6-97ad5f57ed81 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:45.670547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:45.671508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:45.671799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:45.672072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] PATH_INFO: `/v3/users/e1fe15f7b53a46e7840fd8342266ce2b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:45.673824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fe15f7b53a46e7840fd8342266ce2b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:45.679915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:45.789286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fe15f7b53a46e7840fd8342266ce2b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.790143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fe15f7b53a46e7840fd8342266ce2b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.791648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:45.801837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fe15f7b53a46e7840fd8342266ce2b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.817302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fe15f7b53a46e7840fd8342266ce2b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.863847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f84b423f-7150-4180-82c5-d75d445f26b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fe15f7b53a46e7840fd8342266ce2b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:45.882931 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2241/4523] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:45 2022] DELETE /identity/v3/users/e1fe15f7b53a46e7840fd8342266ce2b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:48.106758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:48.108326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:48.133136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d18e8-5f93-4b5f-871b-c43a9cd09ea6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:48.305210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:48.306468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.307209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.307711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] PATH_INFO: `/v3/projects/e9b5dab97d2a4b4dba6d2c7350ad0613` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.309486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b5dab97d2a4b4dba6d2c7350ad0613)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:48.312291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:48.378141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:48.402666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b5dab97d2a4b4dba6d2c7350ad0613', 'request_id': 'req-35673094-2acb-4c6a-8473-efbab47bca2e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:48.417159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b5dab97d2a4b4dba6d2c7350ad0613', 'request_id': 'req-35673094-2acb-4c6a-8473-efbab47bca2e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:48.427367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b5dab97d2a4b4dba6d2c7350ad0613', 'request_id': 'req-35673094-2acb-4c6a-8473-efbab47bca2e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:48.460631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] The token cache is being invalidate because project e9b5dab97d2a4b4dba6d2c7350ad0613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:48.460945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:48.461672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35673094-2acb-4c6a-8473-efbab47bca2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:48.463265 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2283/4524] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:48 2022] DELETE /identity/v3/projects/e9b5dab97d2a4b4dba6d2c7350ad0613 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:48.468303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f32ac-94d7-4597-8a16-18094d0b159f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.469488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f32ac-94d7-4597-8a16-18094d0b159f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.470112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f32ac-94d7-4597-8a16-18094d0b159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.500084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-512f32ac-94d7-4597-8a16-18094d0b159f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:48.501937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-512f32ac-94d7-4597-8a16-18094d0b159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:48.590121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2242/4525] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:48.601518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20cbdf-78d0-4a9d-b0e1-ca742bc8b6c6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.602218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20cbdf-78d0-4a9d-b0e1-ca742bc8b6c6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.602500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20cbdf-78d0-4a9d-b0e1-ca742bc8b6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.626713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6b20cbdf-78d0-4a9d-b0e1-ca742bc8b6c6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:48.628101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6b20cbdf-78d0-4a9d-b0e1-ca742bc8b6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:48.661858 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2284/4526] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:48.666579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f32ac-94d7-4597-8a16-18094d0b159f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:48.678544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f32ac-94d7-4597-8a16-18094d0b159f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:48.687071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:48.687722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.688206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.688472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.690672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:48.691034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:48.692607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:48.707413 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2243/4527] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:48.711788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20cbdf-78d0-4a9d-b0e1-ca742bc8b6c6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:48.723182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20cbdf-78d0-4a9d-b0e1-ca742bc8b6c6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:48.728465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:48.729189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.729398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.729661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.730298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:48.732853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:48.773609 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2285/4528] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:48 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:48.777597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:48.793049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a99d3-4f54-463f-b302-eee947b6c40c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:48.878711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:48.879348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.880770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.881059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.882144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:48.884406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:48.924394 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2244/4529] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:48 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:48.925148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:48.937863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762d5dd-73f7-4296-98a4-836de8d175a5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:48.943547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:48.945170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.945613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.945880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.946872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:48.948518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:48.960731 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2286/4530] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:13:48.965643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:48.981430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfdc66-aefa-4e84-b207-1ee1413075d7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:48.993303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf7f30-f3d3-491d-8de7-829404875839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:48.993862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf7f30-f3d3-491d-8de7-829404875839 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:48.994794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf7f30-f3d3-491d-8de7-829404875839 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:48.995081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf7f30-f3d3-491d-8de7-829404875839 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:48.996309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf7f30-f3d3-491d-8de7-829404875839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:48.999476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf7f30-f3d3-491d-8de7-829404875839 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:49.017775 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2245/4531] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:49.018596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:49.030837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fbe3d-990b-4387-adaf-23b213bfebd5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:49.035919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:49.036781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.037067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.037344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] PATH_INFO: `/v3/projects/ef3736d32d44426d921789191fdbe507/users/04c6e14a03984c97b9ef7df71dcf4f1d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.062162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3736d32d44426d921789191fdbe507, user_id=04c6e14a03984c97b9ef7df71dcf4f1d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:49.066707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:49.107851 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2287/4532] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:49 2022] PUT /identity/v3/projects/ef3736d32d44426d921789191fdbe507/users/04c6e14a03984c97b9ef7df71dcf4f1d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:49.118681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e956f-c844-4431-a006-ba3ba4830aa7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.118995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e956f-c844-4431-a006-ba3ba4830aa7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.119237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e956f-c844-4431-a006-ba3ba4830aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.184408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9f3e956f-c844-4431-a006-ba3ba4830aa7 None None] MFA Rules not processed for user `04c6e14a03984c97b9ef7df71dcf4f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:49.186453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9f3e956f-c844-4431-a006-ba3ba4830aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:49.213350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:49.283812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2246/4533] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:49.288598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:49.360851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72c8a7-02f6-4478-97b7-094be893981f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:49.366912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:49.367845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.368267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.368709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.369517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:49.371755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:49.425168 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2288/4534] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:49 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:49.426858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e956f-c844-4431-a006-ba3ba4830aa7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:49.444507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e956f-c844-4431-a006-ba3ba4830aa7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:49.537979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:49.540123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.540555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.540991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.542650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:49.544948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:49.587644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2247/4535] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:49 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:49.590984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:49.607772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d457d6c-b4da-437c-9ae5-be15afd94416 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:49.629653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:49.630199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:49.630314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.630581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.630848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.632567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:49.634435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:49.636126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:49.659502 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2289/4536] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:49.659668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69cf6d-e069-4391-990a-588729c21373 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:49.660632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:49.671452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:49.673317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.673600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.673876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] PATH_INFO: `/v3/users/11bee4b027a84a979f527dc91586bb63` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.678082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11bee4b027a84a979f527dc91586bb63)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:49.681325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:49.697755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858131-9bc5-4a31-97e1-002750dbc654 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:49.711136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:49.715124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.717992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.718452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] PATH_INFO: `/v3/projects/c715a92969ef485e99d9240921e4bf17/users/468194f1f4fa4a0c9c6b60e90f7db9a5/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.748503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c715a92969ef485e99d9240921e4bf17, user_id=468194f1f4fa4a0c9c6b60e90f7db9a5, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:49.756236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:49.796852 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2290/4537] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:49 2022] PUT /identity/v3/projects/c715a92969ef485e99d9240921e4bf17/users/468194f1f4fa4a0c9c6b60e90f7db9a5/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:49.800855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:49.807497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11bee4b027a84a979f527dc91586bb63'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:49.809305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11bee4b027a84a979f527dc91586bb63'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:49.818494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:49.835528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11bee4b027a84a979f527dc91586bb63'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:49.859757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11bee4b027a84a979f527dc91586bb63'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:49.883988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f48dde39-227c-46dd-80eb-a11183483352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11bee4b027a84a979f527dc91586bb63'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:49.915522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2248/4538] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:49 2022] DELETE /identity/v3/users/11bee4b027a84a979f527dc91586bb63 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:49.919631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:49.919914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:49.920186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:49.924306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122af566-7707-419a-b756-26993eeb56fd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:49.960447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:49.961495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:50.007619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:50.095028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:50.095656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:50.095930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:50.096238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:50.096862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:50.098486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:50.113872 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2249/4539] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:50.123266 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2291/4540] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:50.125083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:50.125968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:50.146974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:50.149032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:50.154382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717972c5-5521-49a6-b8d4-6a44403bbde6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:50.179310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144946c-7640-409d-825c-f2a9fe36be83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:50.180050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144946c-7640-409d-825c-f2a9fe36be83 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:50.180156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144946c-7640-409d-825c-f2a9fe36be83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:50.180350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144946c-7640-409d-825c-f2a9fe36be83 admin admin] PATH_INFO: `/v3/projects/c715a92969ef485e99d9240921e4bf17/users/468194f1f4fa4a0c9c6b60e90f7db9a5/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:50.187956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144946c-7640-409d-825c-f2a9fe36be83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c715a92969ef485e99d9240921e4bf17, user_id=468194f1f4fa4a0c9c6b60e90f7db9a5, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:50.190691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144946c-7640-409d-825c-f2a9fe36be83 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:50.211786 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2250/4541] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:50 2022] PUT /identity/v3/projects/c715a92969ef485e99d9240921e4bf17/users/468194f1f4fa4a0c9c6b60e90f7db9a5/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:50.222353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:50.223046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:50.223574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:50.268313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] MFA Rules not processed for user `468194f1f4fa4a0c9c6b60e90f7db9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:50.269006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba813bc-f892-4aa1-b17a-246ed5b1a29b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:50.270619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:50.345197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:50.345832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:50.346298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:50.346644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:50.370735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:50.390633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2251/4542] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:50.398451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:50.400257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:50.434379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:50.437837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:50.455494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2292/4543] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:13:50.486830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ac47b-eeff-4912-a0a7-b7b92ac521fe None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:50.492178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:50.493034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:50.493549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:50.493703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:50.503825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:50.505435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:50.517630 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2252/4544] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:13:50.649392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:50.663021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345be6a-5d4a-4e8d-9a8c-6603296c1806 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:50.669704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:50.671094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:50.671479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:50.672037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:50.674387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:50.794342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:50.800882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:50.828083 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2293/4545] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:50.841615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:50.843200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:50.869505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091823ff-397d-455f-aa78-a3687ee1a00c service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:50.874708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:50.875575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:50.875944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:50.876792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] PATH_INFO: `/v3/projects/a924daeef64b4721a640fadf821f8511` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:50.878701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a924daeef64b4721a640fadf821f8511)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:50.881803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:50.952862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:50.988770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a924daeef64b4721a640fadf821f8511', 'request_id': 'req-768274c8-178b-48cb-afa6-81ae2062b6d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.001308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a924daeef64b4721a640fadf821f8511', 'request_id': 'req-768274c8-178b-48cb-afa6-81ae2062b6d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.034418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a924daeef64b4721a640fadf821f8511', 'request_id': 'req-768274c8-178b-48cb-afa6-81ae2062b6d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.056537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] The token cache is being invalidate because project a924daeef64b4721a640fadf821f8511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:51.057277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.058461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.061190 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2253/4546] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:50 2022] DELETE /identity/v3/projects/a924daeef64b4721a640fadf821f8511 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:51.064975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:51.081191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489b592-9f74-4fb3-bb02-a3ea992f28be service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:51.168838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:51.169741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:51.169819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:51.170089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] PATH_INFO: `/v3/users/304144ea918141c18e1d33ac8196c821` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:51.171434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] RBAC: Authorizing `identity:delete_user(user_id=304144ea918141c18e1d33ac8196c821)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:51.173811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:51.229909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '304144ea918141c18e1d33ac8196c821'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.230624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '304144ea918141c18e1d33ac8196c821'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.253362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '304144ea918141c18e1d33ac8196c821'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.260087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:51.273608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '304144ea918141c18e1d33ac8196c821'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.281766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-391a9dde-c0f9-4250-9db1-d47d1da0112b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '304144ea918141c18e1d33ac8196c821'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.311167 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2294/4547] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:51 2022] DELETE /identity/v3/users/304144ea918141c18e1d33ac8196c821 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:51.548538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:51.551402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:51.576202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768274c8-178b-48cb-afa6-81ae2062b6d6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:51.652523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:51.653453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:51.653737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:51.653999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] PATH_INFO: `/v3/projects/14123409b8ea43f7a884288a9145b6bf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:51.655132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14123409b8ea43f7a884288a9145b6bf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:51.657315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:51.710147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14123409b8ea43f7a884288a9145b6bf', 'request_id': 'req-e462cec9-6a4e-4616-aa25-f325c9c00d24'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.716758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14123409b8ea43f7a884288a9145b6bf', 'request_id': 'req-e462cec9-6a4e-4616-aa25-f325c9c00d24'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.726541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14123409b8ea43f7a884288a9145b6bf', 'request_id': 'req-e462cec9-6a4e-4616-aa25-f325c9c00d24'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.740785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] The token cache is being invalidate because project 14123409b8ea43f7a884288a9145b6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:51.741337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.742242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e462cec9-6a4e-4616-aa25-f325c9c00d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:51.744012 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2254/4548] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:51 2022] DELETE /identity/v3/projects/14123409b8ea43f7a884288a9145b6bf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:51.749511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c911a-167e-421a-8c98-ad434f14e1ed None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:51.749876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c911a-167e-421a-8c98-ad434f14e1ed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:51.750216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c911a-167e-421a-8c98-ad434f14e1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:51.772870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6e4c911a-167e-421a-8c98-ad434f14e1ed None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:51.774131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e4c911a-167e-421a-8c98-ad434f14e1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:51.842074 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2295/4549] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:51.851479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c776b44-1919-4097-abbb-e46e1c1ee6eb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:51.851971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c776b44-1919-4097-abbb-e46e1c1ee6eb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:51.853507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c776b44-1919-4097-abbb-e46e1c1ee6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:51.874646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8c776b44-1919-4097-abbb-e46e1c1ee6eb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:51.875866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8c776b44-1919-4097-abbb-e46e1c1ee6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:51.884131 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2255/4550] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:51.889562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c911a-167e-421a-8c98-ad434f14e1ed None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:51.899962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c911a-167e-421a-8c98-ad434f14e1ed None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:51.905341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:51.906294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:51.906633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:51.906966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:51.907678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:51.908056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:51.909762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:51.925961 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2296/4551] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:13:51.927353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c776b44-1919-4097-abbb-e46e1c1ee6eb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:51.937339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c776b44-1919-4097-abbb-e46e1c1ee6eb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:51.942476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:51.943235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:51.943514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:51.943847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:51.944480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:51.946937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:51.981688 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2256/4552] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:51 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:51.983242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:51.994079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90cb7b-6b17-46de-9561-f26aa1839f4f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:52.058285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:52.058769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:52.059054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:52.060152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:52.060920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:52.062939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:52.088310 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2297/4553] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:51 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:52.092730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:52.105964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6593f-7100-45d4-ae12-369ea577693d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:52.111001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:52.112919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:52.112919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:52.112919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:52.112919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:52.114391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:52.126292 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2257/4554] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:52.130745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:52.154292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0a10d-0d74-484c-9cf5-27156ae2f12c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:52.160773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6474f-1ea0-40e0-9d7d-975d2522e728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:52.161398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6474f-1ea0-40e0-9d7d-975d2522e728 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:52.161861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6474f-1ea0-40e0-9d7d-975d2522e728 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:52.162135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6474f-1ea0-40e0-9d7d-975d2522e728 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:52.162983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6474f-1ea0-40e0-9d7d-975d2522e728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:52.164608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6474f-1ea0-40e0-9d7d-975d2522e728 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:52.187747 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2298/4555] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:52.190716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:52.221841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99051391-748e-4ff9-a90d-2b30dd86fc5e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:52.231395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:52.232242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:52.232596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:52.233029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] PATH_INFO: `/v3/projects/eef1dd3fd4c34136b5c6d3e2df69427b/users/91986814eaa9450d88bd866f8f86dd8d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:52.245068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:52.256553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef1dd3fd4c34136b5c6d3e2df69427b, user_id=91986814eaa9450d88bd866f8f86dd8d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:52.262308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:52.299888 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2258/4556] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:52 2022] PUT /identity/v3/projects/eef1dd3fd4c34136b5c6d3e2df69427b/users/91986814eaa9450d88bd866f8f86dd8d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:52.302342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dccb08-5c23-4b1f-86eb-8de6b4460aea None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:52.303597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dccb08-5c23-4b1f-86eb-8de6b4460aea None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:52.303884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dccb08-5c23-4b1f-86eb-8de6b4460aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:52.346006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-47dccb08-5c23-4b1f-86eb-8de6b4460aea None None] MFA Rules not processed for user `91986814eaa9450d88bd866f8f86dd8d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:52.347543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-47dccb08-5c23-4b1f-86eb-8de6b4460aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:52.439748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:52.527776 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2299/4557] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:52.628265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:52.631315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:52.785833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c03bfb-199d-4f13-8040-18747ccb4616 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:52.800292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece100f5-09ad-4bfe-8326-d0d443a443b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:52.801275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece100f5-09ad-4bfe-8326-d0d443a443b5 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:52.801578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece100f5-09ad-4bfe-8326-d0d443a443b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:52.801855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece100f5-09ad-4bfe-8326-d0d443a443b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:52.820489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece100f5-09ad-4bfe-8326-d0d443a443b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:52.822326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece100f5-09ad-4bfe-8326-d0d443a443b5 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:52.840173 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2259/4558] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:52.971046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dccb08-5c23-4b1f-86eb-8de6b4460aea None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:53.079579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dccb08-5c23-4b1f-86eb-8de6b4460aea None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:53.086326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:53.087067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:53.087381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:53.088228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] PATH_INFO: `/v3/users/91986814eaa9450d88bd866f8f86dd8d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:53.089922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91986814eaa9450d88bd866f8f86dd8d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:53.093470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:53.197683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91986814eaa9450d88bd866f8f86dd8d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:53.198466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91986814eaa9450d88bd866f8f86dd8d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:53.259601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91986814eaa9450d88bd866f8f86dd8d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:53.274262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91986814eaa9450d88bd866f8f86dd8d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:53.276213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91986814eaa9450d88bd866f8f86dd8d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:53.324186 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2300/4559] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:52 2022] DELETE /identity/v3/users/91986814eaa9450d88bd866f8f86dd8d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:53.328581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:53.329129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:53.329213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:53.383753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:53.387237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:53.527069 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2260/4560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:13:53.595534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:53.597807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:53.682814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:53.691430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce1c44-f918-4dfa-98b2-55f3a6fb1b59 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:53.700732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200d912-755a-43b6-86a2-395129caf2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:53.702003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200d912-755a-43b6-86a2-395129caf2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:53.703577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200d912-755a-43b6-86a2-395129caf2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:53.704214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200d912-755a-43b6-86a2-395129caf2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:53.751380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200d912-755a-43b6-86a2-395129caf2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:53.754591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200d912-755a-43b6-86a2-395129caf2c5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:53.774057 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2301/4561] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:53.875510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:53.877054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:53.890254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa005-16dc-442a-b789-16c5ab47807d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:53.901134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:53.902384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:53.902384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:53.902384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] PATH_INFO: `/v3/users/04c6e14a03984c97b9ef7df71dcf4f1d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:53.906704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c6e14a03984c97b9ef7df71dcf4f1d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:53.909017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:53.996167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c6e14a03984c97b9ef7df71dcf4f1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:53.996976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c6e14a03984c97b9ef7df71dcf4f1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:54.008697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c6e14a03984c97b9ef7df71dcf4f1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:54.025413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c6e14a03984c97b9ef7df71dcf4f1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:54.051176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c6e14a03984c97b9ef7df71dcf4f1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:54.059468 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2261/4562] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:53 2022] DELETE /identity/v3/users/04c6e14a03984c97b9ef7df71dcf4f1d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:54.064219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:54.064591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:54.064939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:54.097931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:54.099172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:54.231981 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2302/4563] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:54.242045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:54.245569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:54.259989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:54.343092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa671e-3657-4bc3-855f-38928bc9a3b8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:54.347363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661b9e7-80ac-49b4-9a3c-819aec9ea873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:54.348420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b9e7-80ac-49b4-9a3c-819aec9ea873 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:54.348915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b9e7-80ac-49b4-9a3c-819aec9ea873 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:54.349818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661b9e7-80ac-49b4-9a3c-819aec9ea873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:54.363492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661b9e7-80ac-49b4-9a3c-819aec9ea873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:54.365389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661b9e7-80ac-49b4-9a3c-819aec9ea873 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:54.378711 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2262/4564] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:13:55.057664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:55.060137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:55.080303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ccc22-86dc-4fdf-8439-0214176900ef None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:55.089728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:55.090564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.090889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.091157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] PATH_INFO: `/v3/projects/eef1dd3fd4c34136b5c6d3e2df69427b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.092328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef1dd3fd4c34136b5c6d3e2df69427b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:55.094646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:55.212174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef1dd3fd4c34136b5c6d3e2df69427b', 'request_id': 'req-e6f2e615-eb52-4091-bf47-b456ae21b89f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:55.233035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef1dd3fd4c34136b5c6d3e2df69427b', 'request_id': 'req-e6f2e615-eb52-4091-bf47-b456ae21b89f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:55.258709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef1dd3fd4c34136b5c6d3e2df69427b', 'request_id': 'req-e6f2e615-eb52-4091-bf47-b456ae21b89f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:55.275706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] The token cache is being invalidate because project eef1dd3fd4c34136b5c6d3e2df69427b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:55.276275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:55.277349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e6f2e615-eb52-4091-bf47-b456ae21b89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:55.279392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2303/4565] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:55 2022] DELETE /identity/v3/projects/eef1dd3fd4c34136b5c6d3e2df69427b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:55.284270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136968e9-5c01-46ef-916f-8301c145161f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.284560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136968e9-5c01-46ef-916f-8301c145161f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.286504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136968e9-5c01-46ef-916f-8301c145161f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.315340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-136968e9-5c01-46ef-916f-8301c145161f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:55.317571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-136968e9-5c01-46ef-916f-8301c145161f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:55.431369 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2263/4566] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:55.440674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8816c-5bcc-4d33-a2f6-4440c43cdeec None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.441080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8816c-5bcc-4d33-a2f6-4440c43cdeec None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.441430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8816c-5bcc-4d33-a2f6-4440c43cdeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.490162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-55a8816c-5bcc-4d33-a2f6-4440c43cdeec None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:55.497158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-55a8816c-5bcc-4d33-a2f6-4440c43cdeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:55.514687 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2304/4567] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:55.519308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136968e9-5c01-46ef-916f-8301c145161f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:55.530322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136968e9-5c01-46ef-916f-8301c145161f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:55.541693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:55.542445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.542733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.543009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.544312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:55.544312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:55.546166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:55.567132 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2264/4568] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:55.568120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8816c-5bcc-4d33-a2f6-4440c43cdeec None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:55.585320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8816c-5bcc-4d33-a2f6-4440c43cdeec None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:55.597222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:55.597864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.598142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.598450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.599223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:55.601583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:55.654476 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2305/4569] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:55 2022] POST /identity/v3/projects => generated 393 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:55.655778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:55.667149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804385f7-922c-4a60-a057-6980aeabad38 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:55.838635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:55.840297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.840297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.840297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.840297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:55.842288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:55.900145 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2265/4570] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:55 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:55.904933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:55.935735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554e966-4656-4ae8-91f8-60d46e6e8fc9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:55.941659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:55.942300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.943328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.943328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.943528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:55.945414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:55.959375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2306/4571] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:55.963452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:55.982309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659d31-f063-4fa0-b3d0-da0484522db0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:55.991528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2623b3-b419-4531-aeef-e9cc3c9f2495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:55.991901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2623b3-b419-4531-aeef-e9cc3c9f2495 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:55.992210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2623b3-b419-4531-aeef-e9cc3c9f2495 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:55.992462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2623b3-b419-4531-aeef-e9cc3c9f2495 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:55.993167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2623b3-b419-4531-aeef-e9cc3c9f2495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:55.994837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2623b3-b419-4531-aeef-e9cc3c9f2495 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:56.009782 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2266/4572] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:56.010820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:56.023323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae4591-2e12-4784-9e56-b9d864c96234 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:56.029059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:56.030573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:56.030856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:56.031129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] PATH_INFO: `/v3/projects/59136525ae52452490d15c6c2d623cad/users/1adb648b5b9a4d448391ab2741e56130/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:56.067529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59136525ae52452490d15c6c2d623cad, user_id=1adb648b5b9a4d448391ab2741e56130, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:56.072700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:56.108583 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2307/4573] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:56 2022] PUT /identity/v3/projects/59136525ae52452490d15c6c2d623cad/users/1adb648b5b9a4d448391ab2741e56130/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:56.118773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:56.119060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:56.119340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:56.168119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:56.168464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:56.170463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:56.190666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] MFA Rules not processed for user `1adb648b5b9a4d448391ab2741e56130`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:56.192879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:56.194790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:56.307197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8392e9-6126-4184-bc2d-715172531be7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:56.321515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:56.322525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:56.322809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:56.323086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] PATH_INFO: `/v3/projects/ef3736d32d44426d921789191fdbe507` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:56.324347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3736d32d44426d921789191fdbe507)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:56.326339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:56.335488 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2267/4574] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:56.379978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:56.382222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:56.427029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3736d32d44426d921789191fdbe507', 'request_id': 'req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.450987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3736d32d44426d921789191fdbe507', 'request_id': 'req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.457572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3736d32d44426d921789191fdbe507', 'request_id': 'req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.486668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] The token cache is being invalidate because project ef3736d32d44426d921789191fdbe507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:56.486964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.494093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.498800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2308/4575] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:56 2022] DELETE /identity/v3/projects/ef3736d32d44426d921789191fdbe507 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:56.500385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:56.501563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:56.523742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9441821-2645-4405-b5d6-a84b7aa56780 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:56.627512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f6931-ea49-4912-a9f5-7997d78e2a4e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:56.631689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:56.634273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:56.635111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:56.635224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:56.635626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:56.635736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:56.636069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:56.636286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] PATH_INFO: `/v3/users/468194f1f4fa4a0c9c6b60e90f7db9a5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:56.637609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=468194f1f4fa4a0c9c6b60e90f7db9a5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:56.639035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:56.641352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:56.739388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:56.743757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468194f1f4fa4a0c9c6b60e90f7db9a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.746540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468194f1f4fa4a0c9c6b60e90f7db9a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.763444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468194f1f4fa4a0c9c6b60e90f7db9a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.783330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468194f1f4fa4a0c9c6b60e90f7db9a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.784929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468194f1f4fa4a0c9c6b60e90f7db9a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:56.824139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:56.827692 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2309/4576] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:56 2022] DELETE /identity/v3/users/468194f1f4fa4a0c9c6b60e90f7db9a5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:56.840959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:56.894033 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2268/4577] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:13:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:13:58.205491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:58.211548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:58.395953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3203b6-fd50-44fa-bf17-5b3455f71533 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:58.402097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:58.402978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:58.403246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:58.403661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] PATH_INFO: `/v3/users/28d6b4f6ac624ad6984bad527eb7555d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:58.405513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d6b4f6ac624ad6984bad527eb7555d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:58.407612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:58.485302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d6b4f6ac624ad6984bad527eb7555d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.486426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d6b4f6ac624ad6984bad527eb7555d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.499590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d6b4f6ac624ad6984bad527eb7555d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.503617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:58.527422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620800ba-720d-4a17-ae52-cbbe97c19eec service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:58.538484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d6b4f6ac624ad6984bad527eb7555d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.539263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a6d2738-2717-48fc-91c1-bb040ad5e89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d6b4f6ac624ad6984bad527eb7555d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.568127 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2310/4578] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:13:58 2022] DELETE /identity/v3/users/28d6b4f6ac624ad6984bad527eb7555d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:58.573174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:58.578846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:58.579146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:58.579442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] PATH_INFO: `/v3/projects/c715a92969ef485e99d9240921e4bf17` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:58.586898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c715a92969ef485e99d9240921e4bf17)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:58.591114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:58.599741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:58.685768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c715a92969ef485e99d9240921e4bf17', 'request_id': 'req-a4e036f6-407d-4e76-8989-3f3b552055f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.698696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c715a92969ef485e99d9240921e4bf17', 'request_id': 'req-a4e036f6-407d-4e76-8989-3f3b552055f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.713529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c715a92969ef485e99d9240921e4bf17', 'request_id': 'req-a4e036f6-407d-4e76-8989-3f3b552055f4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.736379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] The token cache is being invalidate because project c715a92969ef485e99d9240921e4bf17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:58.736908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.738130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4e036f6-407d-4e76-8989-3f3b552055f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:58.739971 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2269/4579] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:58 2022] DELETE /identity/v3/projects/c715a92969ef485e99d9240921e4bf17 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:58.745986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056c7bd-40b3-4038-b4f4-1f98bbfa8cf4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:58.746374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056c7bd-40b3-4038-b4f4-1f98bbfa8cf4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:58.746711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056c7bd-40b3-4038-b4f4-1f98bbfa8cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:58.773825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a056c7bd-40b3-4038-b4f4-1f98bbfa8cf4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:58.775790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a056c7bd-40b3-4038-b4f4-1f98bbfa8cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:58.819343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:58.868866 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2311/4580] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:58.879000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9d2ee-0005-4e3a-aa10-e8b4ac6e5a33 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:58.879203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9d2ee-0005-4e3a-aa10-e8b4ac6e5a33 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:58.879577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9d2ee-0005-4e3a-aa10-e8b4ac6e5a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:58.929164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-53b9d2ee-0005-4e3a-aa10-e8b4ac6e5a33 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:58.930792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-53b9d2ee-0005-4e3a-aa10-e8b4ac6e5a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:58.944688 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2270/4581] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:58.949216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056c7bd-40b3-4038-b4f4-1f98bbfa8cf4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:58.960526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056c7bd-40b3-4038-b4f4-1f98bbfa8cf4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:58.966535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:58.967935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:58.968220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:58.968706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:58.969497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:58.969900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:58.973215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:58.987188 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2312/4582] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:58.991513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9d2ee-0005-4e3a-aa10-e8b4ac6e5a33 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.007307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9d2ee-0005-4e3a-aa10-e8b4ac6e5a33 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.014873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.015514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.015790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.016060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.016944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.018844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.057879 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2271/4583] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:58 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:59.061927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.074321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62502851-76c8-4c32-91ad-a86f7c0750a6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.143678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.144451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.144871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.145285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.145918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.148007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.178161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2313/4584] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:59 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:13:59.183407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.192165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.194228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:59.204421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b198f-2d93-4017-9fde-26adc38c70b7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.207660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f716d-5fb8-4ce4-bcaf-df6500977af8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.211549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.212431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.213334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.214501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.215263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.216553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.217315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.217424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.217730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.218109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] PATH_INFO: `/v3/projects/0c09a67832414ec0ab88df3e0ba92d92` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.219454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c09a67832414ec0ab88df3e0ba92d92)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.222705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.244869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2272/4585] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:13:59.248630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.273434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06386614-c2c9-419e-aba9-ea38c8bc9886 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.289791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.291423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.292802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.293234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.296528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.299235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.320716 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2273/4586] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:13:59.326397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c09a67832414ec0ab88df3e0ba92d92', 'request_id': 'req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:59.332106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.346252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c09a67832414ec0ab88df3e0ba92d92', 'request_id': 'req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:59.356834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94de88a-dbeb-4c66-9434-f957c55b6f27 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.358715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c09a67832414ec0ab88df3e0ba92d92', 'request_id': 'req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:59.386277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] The token cache is being invalidate because project 0c09a67832414ec0ab88df3e0ba92d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:13:59.386811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:59.387611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3c26c07-7a1b-4d8c-b508-1eed3147cd9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:13:59.391491 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2314/4587] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:13:59 2022] DELETE /identity/v3/projects/0c09a67832414ec0ab88df3e0ba92d92 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:13:59.400370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174d9b4-ff29-4c5b-8c87-e28c32c76386 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.400896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174d9b4-ff29-4c5b-8c87-e28c32c76386 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.401280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174d9b4-ff29-4c5b-8c87-e28c32c76386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.446078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4174d9b4-ff29-4c5b-8c87-e28c32c76386 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:59.447968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4174d9b4-ff29-4c5b-8c87-e28c32c76386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:59.459728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ac908-1fe0-4ee0-9f15-9b82a00cdb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.461478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac908-1fe0-4ee0-9f15-9b82a00cdb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.463841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac908-1fe0-4ee0-9f15-9b82a00cdb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.464163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac908-1fe0-4ee0-9f15-9b82a00cdb36 admin admin] PATH_INFO: `/v3/projects/f7e2c5717b264d81900ff088a424fc34/users/de3d102d4ea34b2981829813e211a010/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.485007 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2315/4588] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:59.491485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ac908-1fe0-4ee0-9f15-9b82a00cdb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e2c5717b264d81900ff088a424fc34, user_id=de3d102d4ea34b2981829813e211a010, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.504147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ac908-1fe0-4ee0-9f15-9b82a00cdb36 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.565195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c09772-265c-4529-b112-5fe7e0ab94ff None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.565625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c09772-265c-4529-b112-5fe7e0ab94ff None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.566000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c09772-265c-4529-b112-5fe7e0ab94ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.570995 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2274/4589] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:13:59 2022] PUT /identity/v3/projects/f7e2c5717b264d81900ff088a424fc34/users/de3d102d4ea34b2981829813e211a010/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:13:59.590425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.591004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.591362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-80c09772-265c-4529-b112-5fe7e0ab94ff None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:59.591702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.592792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-80c09772-265c-4529-b112-5fe7e0ab94ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:59.647753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:59.670535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] MFA Rules not processed for user `de3d102d4ea34b2981829813e211a010`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:13:59.673856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:59.710199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:59.711992 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2316/4590] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:59.716962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c09772-265c-4529-b112-5fe7e0ab94ff None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.730239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c09772-265c-4529-b112-5fe7e0ab94ff None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.737550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.738437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.739030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.739406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.740288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:13:59.741004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.743345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.760203 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2317/4591] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:13:59.765292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.779164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532777af-515e-4bf6-a54b-f33c66da226d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:13:59.786981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:13:59.788070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:13:59.788070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:13:59.788535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:13:59.789489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:13:59.792452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:13:59.836092 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2275/4592] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:13:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:13:59.839489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.841020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:13:59.855177 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2318/4593] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:59 2022] POST /identity/v3/projects => generated 361 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:13:59.860339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:13:59.868730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c3e9f-b483-4633-98d2-db581319dfc9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.027868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbc6b-0371-4dca-9b92-de8f56d491b4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.029635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.030915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.031707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.033550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.035099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.041484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.042172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.044576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.046571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.047022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.050299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.055582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.128763 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2319/4594] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:13:59 2022] POST /identity/v3/users => generated 281 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:14:00.134593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.139352 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2276/4595] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:13:59 2022] POST /identity/v3/projects => generated 365 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:14:00.158636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.182133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d735-f7e7-4495-a7a3-92960a702cc5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.367294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750564e3-dcf3-4fca-b221-ee48fb389b11 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.374508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.375693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.376108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.376578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.377209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.377563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.377859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.378140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.378459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.379337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.381885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.384956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.409882 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2320/4596] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:00.414080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.434195 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2277/4597] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:00 2022] POST /identity/v3/users => generated 289 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:00.435688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.437090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457ea59-d659-456b-b27a-9969c09f98f0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.448098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.448784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.449125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.449356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.451834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.451834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.488829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b264b5f-ff4a-439b-8b7d-b656f7cf7243 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.508790 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2321/4598] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:14:00.510959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.512048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.513069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.513418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.513697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.514660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.516710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.526284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dd07e-0136-4832-a3c6-5be0ac97620e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.536486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c9b35-f749-4ccc-8e4c-3d51f2bc74bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.539979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c9b35-f749-4ccc-8e4c-3d51f2bc74bf admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.540289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c9b35-f749-4ccc-8e4c-3d51f2bc74bf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.540557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c9b35-f749-4ccc-8e4c-3d51f2bc74bf admin admin] PATH_INFO: `/v3/projects/fdc8c943bd8d4736ac49c6b0885693fe/users/524ef6a20bc14636bf2d6a835ed9616a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.546267 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2278/4599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:00.550732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.582874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7916c-b61d-40c0-b98d-0a86c59bcd1b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.588892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:00.598442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c9b35-f749-4ccc-8e4c-3d51f2bc74bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc8c943bd8d4736ac49c6b0885693fe, user_id=524ef6a20bc14636bf2d6a835ed9616a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.601202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c9b35-f749-4ccc-8e4c-3d51f2bc74bf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.612349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.613482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.614019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.614515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] PATH_INFO: `/v3/projects/aa34bb5b700d44e4b63fb6f95a66c79a/users/589640fa4656481ca566ece6fd0a79f8/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.635253 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2322/4600] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:00 2022] PUT /identity/v3/projects/fdc8c943bd8d4736ac49c6b0885693fe/users/524ef6a20bc14636bf2d6a835ed9616a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:00.648071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.648358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.648634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.677085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa34bb5b700d44e4b63fb6f95a66c79a, user_id=589640fa4656481ca566ece6fd0a79f8, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.685673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:00.691295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:00.731401 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2279/4601] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:00 2022] PUT /identity/v3/projects/aa34bb5b700d44e4b63fb6f95a66c79a/users/589640fa4656481ca566ece6fd0a79f8/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:00.733916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.757532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] MFA Rules not processed for user `524ef6a20bc14636bf2d6a835ed9616a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:00.758773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:00.933435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2323/4602] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:00.943120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f3c1-3d2c-4d64-8dfb-11147cd32012 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:00.948406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:00.950233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:00.963935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:00.964579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:00.965467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:00.965467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:00.965844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:00.973151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:01.007465 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2280/4603] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:01.011889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:01.035672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4dba9-ffbf-470f-9088-74b3eabe8a83 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:01.071375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec086d-24a5-4052-903e-e6def2af1cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:01.072403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec086d-24a5-4052-903e-e6def2af1cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:01.072706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec086d-24a5-4052-903e-e6def2af1cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:01.072987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec086d-24a5-4052-903e-e6def2af1cb2 admin admin] PATH_INFO: `/v3/projects/aa34bb5b700d44e4b63fb6f95a66c79a/users/589640fa4656481ca566ece6fd0a79f8/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:01.076169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec086d-24a5-4052-903e-e6def2af1cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa34bb5b700d44e4b63fb6f95a66c79a, user_id=589640fa4656481ca566ece6fd0a79f8, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:01.078939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec086d-24a5-4052-903e-e6def2af1cb2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:01.112252 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2281/4604] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:01 2022] PUT /identity/v3/projects/aa34bb5b700d44e4b63fb6f95a66c79a/users/589640fa4656481ca566ece6fd0a79f8/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:01.113937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:01.114466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:01.114466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:01.152247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c64472-a6b8-402d-ab92-d1e4bd8f3f03 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:01.156635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] MFA Rules not processed for user `589640fa4656481ca566ece6fd0a79f8`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:01.158200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:01.311187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:01.311526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:01.311732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:01.312006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:01.355625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:01.367512 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2282/4605] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:01.375488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:01.378326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:01.527431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:01.531100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:01.555720 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2324/4606] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:01.562593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101531ac-7b8c-4a16-a20c-dd2dea7c8694 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:01.567418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:01.568046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:01.568323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:01.568590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:01.581710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:01.583166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:01.611149 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2283/4607] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:01.629088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:01.773498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:01.784695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8ae95-9407-4533-b702-34ddf9dd42ea service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:01.793726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4576d-5b0a-4032-85f5-b92e9cc7e36a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:01.795625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4576d-5b0a-4032-85f5-b92e9cc7e36a service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:01.796294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4576d-5b0a-4032-85f5-b92e9cc7e36a service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:01.796872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4576d-5b0a-4032-85f5-b92e9cc7e36a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:01.804596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:01.890080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de82402-d0c9-45ba-bb47-3e5d43c5b722 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:01.898828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:01.901126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:01.902755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:01.903176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] PATH_INFO: `/v3/users/524ef6a20bc14636bf2d6a835ed9616a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:01.904947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] RBAC: Authorizing `identity:delete_user(user_id=524ef6a20bc14636bf2d6a835ed9616a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:01.909259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:01.965101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4576d-5b0a-4032-85f5-b92e9cc7e36a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:01.966951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4576d-5b0a-4032-85f5-b92e9cc7e36a service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:01.983317 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2284/4608] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:02.011799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524ef6a20bc14636bf2d6a835ed9616a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:02.016505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524ef6a20bc14636bf2d6a835ed9616a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:02.041536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524ef6a20bc14636bf2d6a835ed9616a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:02.054208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524ef6a20bc14636bf2d6a835ed9616a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:02.055906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524ef6a20bc14636bf2d6a835ed9616a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:02.101389 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2325/4609] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:01 2022] DELETE /identity/v3/users/524ef6a20bc14636bf2d6a835ed9616a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:14:02.105843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:02.106131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:02.106513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:02.145302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:02.146996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:02.307245 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2285/4610] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:02.360347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:02.362163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:02.455968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:02.535935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b405071-76e3-491a-bc56-b2da3786a04c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:02.542550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea254036-fb07-46a7-9ccd-ed2dfbe6477c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:02.555858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea254036-fb07-46a7-9ccd-ed2dfbe6477c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:02.556965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea254036-fb07-46a7-9ccd-ed2dfbe6477c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:02.557416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea254036-fb07-46a7-9ccd-ed2dfbe6477c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:02.571363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea254036-fb07-46a7-9ccd-ed2dfbe6477c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:02.574092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea254036-fb07-46a7-9ccd-ed2dfbe6477c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:02.591198 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2326/4611] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:03.147371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:03.149594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:03.166958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd816ba-db35-4cfe-ad4c-c753d15cedab None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:03.177094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:03.177766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:03.178070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:03.178446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] PATH_INFO: `/v3/projects/fdc8c943bd8d4736ac49c6b0885693fe` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:03.180072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc8c943bd8d4736ac49c6b0885693fe)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:03.186610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:03.332819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc8c943bd8d4736ac49c6b0885693fe', 'request_id': 'req-64488c03-6b59-402f-a0a0-40626175d2f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:03.350742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc8c943bd8d4736ac49c6b0885693fe', 'request_id': 'req-64488c03-6b59-402f-a0a0-40626175d2f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:03.375713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc8c943bd8d4736ac49c6b0885693fe', 'request_id': 'req-64488c03-6b59-402f-a0a0-40626175d2f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:03.389880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] The token cache is being invalidate because project fdc8c943bd8d4736ac49c6b0885693fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:03.390467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:03.391291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64488c03-6b59-402f-a0a0-40626175d2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:03.393593 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2286/4612] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:03 2022] DELETE /identity/v3/projects/fdc8c943bd8d4736ac49c6b0885693fe => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:03.402437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97815d78-cab3-4ccc-8ef2-a7b9294c7c74 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:03.404682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97815d78-cab3-4ccc-8ef2-a7b9294c7c74 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:03.405117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97815d78-cab3-4ccc-8ef2-a7b9294c7c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:03.454760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-97815d78-cab3-4ccc-8ef2-a7b9294c7c74 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:03.461172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-97815d78-cab3-4ccc-8ef2-a7b9294c7c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:03.667312 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2327/4613] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:03.669228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb2cde-31d6-45ba-9319-0ee1a796492b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:03.669729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb2cde-31d6-45ba-9319-0ee1a796492b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:03.670234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb2cde-31d6-45ba-9319-0ee1a796492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:03.703619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2feb2cde-31d6-45ba-9319-0ee1a796492b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:03.711201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2feb2cde-31d6-45ba-9319-0ee1a796492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:03.769030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2287/4614] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:03.770269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97815d78-cab3-4ccc-8ef2-a7b9294c7c74 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:03.785293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97815d78-cab3-4ccc-8ef2-a7b9294c7c74 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:03.796219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:03.799154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:03.799646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:03.799749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:03.800490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:14:03.802837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:03.804619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:03.844529 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2328/4615] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:14:03.845789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb2cde-31d6-45ba-9319-0ee1a796492b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:03.864914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb2cde-31d6-45ba-9319-0ee1a796492b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:03.880609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:03.883652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:03.883652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:03.883652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:03.883924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:03.889044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:03.948511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:03.966557 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2288/4616] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:03 2022] POST /identity/v3/projects => generated 399 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:03.970972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:03.986168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9a419-7f29-469a-8c97-82568cc4d285 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:04.124605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:04.125864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:04.125864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:04.125864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:04.126465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:04.128347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:04.187102 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2329/4617] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:03 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:04.188336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:04.212448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03daf7-760f-41fc-84b6-f4145df0aab0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:04.225848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:04.230579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:04.237873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:04.238398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:04.239268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:04.244385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:04.284252 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2289/4618] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:04.289533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:04.311135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ee6f-1317-48f1-b51e-d399da9dd8e1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:04.324399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadb6a8-bd92-4c84-bdbb-5dbf2fd69d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:04.325278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadb6a8-bd92-4c84-bdbb-5dbf2fd69d37 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:04.326053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadb6a8-bd92-4c84-bdbb-5dbf2fd69d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:04.326053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadb6a8-bd92-4c84-bdbb-5dbf2fd69d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:04.326464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadb6a8-bd92-4c84-bdbb-5dbf2fd69d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:04.331584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadb6a8-bd92-4c84-bdbb-5dbf2fd69d37 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:04.358013 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2330/4619] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:04.362291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:04.394969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda480d-ba9b-4c03-a0ee-e41878817736 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:04.411096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:04.414995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:04.415375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:04.418786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] PATH_INFO: `/v3/projects/1e7e90f89b4844dbafdc1c5e5f17b407/users/c28c5e12dbb444cab4e376a3015c6bd1/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:04.442891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7e90f89b4844dbafdc1c5e5f17b407, user_id=c28c5e12dbb444cab4e376a3015c6bd1, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:04.445914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:04.470231 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2290/4620] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:04 2022] PUT /identity/v3/projects/1e7e90f89b4844dbafdc1c5e5f17b407/users/c28c5e12dbb444cab4e376a3015c6bd1/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:04.476813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:04.477642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:04.478142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:04.503351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:04.504748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:04.569701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:04.604924 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2331/4621] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:04.617504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:04.619795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:04.704499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5362d5-320c-45f1-bba1-c514f5d73b39 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:04.708284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93da40d-944b-4e33-b822-a4fc041c18da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:04.708929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93da40d-944b-4e33-b822-a4fc041c18da service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:04.709202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93da40d-944b-4e33-b822-a4fc041c18da service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:04.709524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93da40d-944b-4e33-b822-a4fc041c18da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:04.720897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93da40d-944b-4e33-b822-a4fc041c18da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:04.722349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93da40d-944b-4e33-b822-a4fc041c18da service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:04.733736 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2291/4622] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:08.128457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:08.130254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:08.166996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30824b54-0b2f-4210-a1d1-e754faba5ae3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:08.183664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:08.184404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:08.184700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:08.185195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] PATH_INFO: `/v3/users/de3d102d4ea34b2981829813e211a010` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:08.187365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=de3d102d4ea34b2981829813e211a010)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:08.191130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:08.286760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de3d102d4ea34b2981829813e211a010'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:08.288583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de3d102d4ea34b2981829813e211a010'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:08.328040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de3d102d4ea34b2981829813e211a010'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:08.340445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de3d102d4ea34b2981829813e211a010'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:08.341207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e2cf5cc1-5df4-4a55-a265-b7daf509c3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de3d102d4ea34b2981829813e211a010'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:08.379236 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2332/4623] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:08 2022] DELETE /identity/v3/users/de3d102d4ea34b2981829813e211a010 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:08.385299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:08.386269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:08.386731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:08.409245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:08.411224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:08.520636 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2292/4624] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:08.555145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:08.555720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:08.556192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:08.564433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:08.567188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:08.586713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] MFA Rules not processed for user `a5e050c7389844a78199faef5a0c9da4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:08.588852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:08.682592 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2333/4625] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:08.709526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1a729-3e1a-4b46-98e7-3622d7cf8d25 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:08.737172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:08.738074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:08.738360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:08.742462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:08.778622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:08.781922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:08.839070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2293/4626] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:14:10.531784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:10.533553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:10.559774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2271f7-ae9b-44e1-9584-089c916a7887 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:10.576152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:10.577204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:10.577677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:10.578155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] PATH_INFO: `/v3/projects/f7e2c5717b264d81900ff088a424fc34` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:10.579609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e2c5717b264d81900ff088a424fc34)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:10.581975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:10.723164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e2c5717b264d81900ff088a424fc34', 'request_id': 'req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:10.732668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:10.753031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e2c5717b264d81900ff088a424fc34', 'request_id': 'req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:10.761856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e2c5717b264d81900ff088a424fc34', 'request_id': 'req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:10.793671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] The token cache is being invalidate because project f7e2c5717b264d81900ff088a424fc34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:10.794980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:10.797500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:10.800707 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2334/4627] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:10 2022] DELETE /identity/v3/projects/f7e2c5717b264d81900ff088a424fc34 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:14:10.805481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:10.830124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a41449-5e58-4628-a2af-f41f20e60ee7 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:10.863763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:11.011852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:11.017512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:11.017800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:11.018102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] PATH_INFO: `/v3/users/589640fa4656481ca566ece6fd0a79f8` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:11.019977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=589640fa4656481ca566ece6fd0a79f8)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:11.024459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:11.118189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589640fa4656481ca566ece6fd0a79f8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:11.119574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589640fa4656481ca566ece6fd0a79f8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:11.133346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589640fa4656481ca566ece6fd0a79f8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:11.154607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589640fa4656481ca566ece6fd0a79f8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:11.186103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-944fc7e8-3dd9-41c5-a983-f463d1875e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589640fa4656481ca566ece6fd0a79f8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:11.195229 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2294/4628] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:10 2022] DELETE /identity/v3/users/589640fa4656481ca566ece6fd0a79f8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:13.947479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:13.950619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:13.970773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83efb49-182e-4df5-b28a-ec79c6c9ad3d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:14.079306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:14.079968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:14.080245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:14.080528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] PATH_INFO: `/v3/projects/aa34bb5b700d44e4b63fb6f95a66c79a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:14.081759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa34bb5b700d44e4b63fb6f95a66c79a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:14.102841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:14.281672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa34bb5b700d44e4b63fb6f95a66c79a', 'request_id': 'req-e537f7e9-2d79-4b42-a1e2-474302391d2f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:14.306341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa34bb5b700d44e4b63fb6f95a66c79a', 'request_id': 'req-e537f7e9-2d79-4b42-a1e2-474302391d2f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:14.323641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa34bb5b700d44e4b63fb6f95a66c79a', 'request_id': 'req-e537f7e9-2d79-4b42-a1e2-474302391d2f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:14.363834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] The token cache is being invalidate because project aa34bb5b700d44e4b63fb6f95a66c79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:14.365143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:14.367258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e537f7e9-2d79-4b42-a1e2-474302391d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:14.374187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2335/4629] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:13 2022] DELETE /identity/v3/projects/aa34bb5b700d44e4b63fb6f95a66c79a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:14.377262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51470365-ba9d-430c-ab6f-302a457a0a7a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:14.377805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51470365-ba9d-430c-ab6f-302a457a0a7a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:14.377805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51470365-ba9d-430c-ab6f-302a457a0a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:14.447560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-51470365-ba9d-430c-ab6f-302a457a0a7a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:14.448171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-51470365-ba9d-430c-ab6f-302a457a0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:14.623799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8b78f-6270-491e-bc07-6075038c4c5c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:14.624564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8b78f-6270-491e-bc07-6075038c4c5c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:14.625118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8b78f-6270-491e-bc07-6075038c4c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:14.633895 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2295/4630] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:14.663024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e4d8b78f-6270-491e-bc07-6075038c4c5c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:14.665127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e4d8b78f-6270-491e-bc07-6075038c4c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:14.678633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2336/4631] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:14.684219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51470365-ba9d-430c-ab6f-302a457a0a7a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:14.696143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51470365-ba9d-430c-ab6f-302a457a0a7a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:14.702868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:14.704032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:14.704032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:14.704727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:14.717440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:14:14.717958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:14.720040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:14.784271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8b78f-6270-491e-bc07-6075038c4c5c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:14.785295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2296/4632] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:14:14.799031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8b78f-6270-491e-bc07-6075038c4c5c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:14.806550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:14.807342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:14.807675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:14.807883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:14.808811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:14.811149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:14.866192 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2337/4633] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:14 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:14.871118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:14.899342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48da70-f945-4b5f-9f9d-6a2c0f0e4fbc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:14.960070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:15.160643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:15.163805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:15.168902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:15.170098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:15.174091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:15.181926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:15.268871 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2297/4634] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:14 2022] POST /identity/v3/users => generated 292 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:15.270629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:15.284132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8acb45-6e2e-4177-a0fb-268aa1dda37d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:15.297909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:15.298782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:15.299245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:15.300349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:15.302185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:15.304372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:15.328394 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2338/4635] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:15.332854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:15.357065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a047-48ab-4ae0-bd61-8616f8102dce admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:15.373779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae9e9-fa91-4b3b-a1f2-053090de80db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:15.375120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae9e9-fa91-4b3b-a1f2-053090de80db admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:15.375836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae9e9-fa91-4b3b-a1f2-053090de80db admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:15.376430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae9e9-fa91-4b3b-a1f2-053090de80db admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:15.378264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ae9e9-fa91-4b3b-a1f2-053090de80db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:15.391257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ae9e9-fa91-4b3b-a1f2-053090de80db admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:15.414572 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2298/4636] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:15.418779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:15.442046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388d157-52d2-4ed5-a11a-73b3513b1817 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:15.458893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:15.461060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:15.461569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:15.462412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] PATH_INFO: `/v3/projects/0ae5b11810804fc0a2b68840f34572d4/users/f93bd04c86e64226b901abcd252234f7/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:15.487222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae5b11810804fc0a2b68840f34572d4, user_id=f93bd04c86e64226b901abcd252234f7, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:15.491805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:15.522311 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2339/4637] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:15 2022] PUT /identity/v3/projects/0ae5b11810804fc0a2b68840f34572d4/users/f93bd04c86e64226b901abcd252234f7/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:15.534642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2243c8-c043-4bbe-aa17-4ab7e0780979 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:15.534642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2243c8-c043-4bbe-aa17-4ab7e0780979 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:15.534823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2243c8-c043-4bbe-aa17-4ab7e0780979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:15.637968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7e2243c8-c043-4bbe-aa17-4ab7e0780979 None None] MFA Rules not processed for user `f93bd04c86e64226b901abcd252234f7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:15.640245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7e2243c8-c043-4bbe-aa17-4ab7e0780979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:15.765886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2299/4638] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:15.771096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:15.789111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:15.900181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae575b0-15c6-4ac4-9d72-df51314cf996 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:15.907882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:15.909301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:15.909841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:15.909968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:15.910575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:15.913096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:15.974619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2340/4639] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:15 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:15.975218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2243c8-c043-4bbe-aa17-4ab7e0780979 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:16.003604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2243c8-c043-4bbe-aa17-4ab7e0780979 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:16.183023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:16.185389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:16.186401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:16.187483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:16.190625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:16.201811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:16.260727 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2300/4640] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:15 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:14:16.264469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:16.282135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f937325-1dce-400c-9598-157dcc0ff240 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:16.302279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:16.303176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:16.304151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:16.304435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:16.305195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:16.309737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:16.338237 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2341/4641] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:16.342218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:16.355776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2232e2-9c8b-4bd7-8fbe-7718aa44a5c6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:16.361986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:16.362718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:16.362998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:16.363274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] PATH_INFO: `/v3/projects/a2f74f0f59d84b2691762f862d5d98fb/users/91fb56177c7443d2903b00d6c42d9182/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:16.380521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:16.382438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f74f0f59d84b2691762f862d5d98fb, user_id=91fb56177c7443d2903b00d6c42d9182, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:16.385984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:16.406513 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2301/4642] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:16 2022] PUT /identity/v3/projects/a2f74f0f59d84b2691762f862d5d98fb/users/91fb56177c7443d2903b00d6c42d9182/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:16.411108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:16.427503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cbd75-0de6-4bf5-8b45-c0402370a47f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:16.527315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7aa2c-0c6a-4eb1-8121-e096d4bd8a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:16.528039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7aa2c-0c6a-4eb1-8121-e096d4bd8a08 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:16.528758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7aa2c-0c6a-4eb1-8121-e096d4bd8a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:16.529562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7aa2c-0c6a-4eb1-8121-e096d4bd8a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:16.530683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7aa2c-0c6a-4eb1-8121-e096d4bd8a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:16.532747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7aa2c-0c6a-4eb1-8121-e096d4bd8a08 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:16.547013 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2342/4643] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:16.551287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:16.563264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dab38-bc81-4c7d-89ce-c74a5fdd3f12 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:16.572349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:16.573139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:16.573416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:16.573929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] PATH_INFO: `/v3/projects/a2f74f0f59d84b2691762f862d5d98fb/users/91fb56177c7443d2903b00d6c42d9182/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:16.576310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f74f0f59d84b2691762f862d5d98fb, user_id=91fb56177c7443d2903b00d6c42d9182, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:16.579973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:16.597239 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2302/4644] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:16 2022] PUT /identity/v3/projects/a2f74f0f59d84b2691762f862d5d98fb/users/91fb56177c7443d2903b00d6c42d9182/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:16.611070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:16.611890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:16.612206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:16.652417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] MFA Rules not processed for user `91fb56177c7443d2903b00d6c42d9182`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:16.654323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:16.774621 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2343/4645] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:16.798825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:16.801906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:16.928597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5807470-2cdf-40e0-aa62-117c3a463889 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:16.934028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:16.934602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:16.934961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:16.935237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:16.949661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:16.951702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:16.964313 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2303/4646] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:18.067636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:18.072405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:18.085890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:18.185512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:18.244330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050ef4-f444-4be3-a509-41813c16fe25 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:18.257070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:18.257801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:18.258083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:18.258361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] PATH_INFO: `/v3/users/241251b91c43444b900922f942681cc5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:18.259602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] RBAC: Authorizing `identity:delete_user(user_id=241251b91c43444b900922f942681cc5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:18.261566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:18.276508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826cec-d495-4ccd-bb8b-bb37fc3ca98c service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:18.282833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:18.283508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:18.284067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:18.284528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] PATH_INFO: `/v3/users/f93bd04c86e64226b901abcd252234f7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:18.287253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93bd04c86e64226b901abcd252234f7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:18.290495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:18.368939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241251b91c43444b900922f942681cc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.369809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241251b91c43444b900922f942681cc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.388175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241251b91c43444b900922f942681cc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.388420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93bd04c86e64226b901abcd252234f7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.400740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241251b91c43444b900922f942681cc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.405111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7203c133-db8a-4c58-8fa2-fff338c98add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241251b91c43444b900922f942681cc5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.416546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93bd04c86e64226b901abcd252234f7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.434600 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2344/4647] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:18 2022] DELETE /identity/v3/users/241251b91c43444b900922f942681cc5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:18.438261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93bd04c86e64226b901abcd252234f7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.458749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93bd04c86e64226b901abcd252234f7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.501834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103b99-26b6-4bd9-b87d-d9e42985b6d1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:18.502403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103b99-26b6-4bd9-b87d-d9e42985b6d1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:18.502855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103b99-26b6-4bd9-b87d-d9e42985b6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:18.517066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de26a622-232f-4766-9509-b14d2d1121ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93bd04c86e64226b901abcd252234f7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:18.521509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:18.542779 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2304/4648] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:18 2022] DELETE /identity/v3/users/f93bd04c86e64226b901abcd252234f7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:18.548363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:18.549101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:18.549477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:18.575563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2c103b99-26b6-4bd9-b87d-d9e42985b6d1 None None] MFA Rules not processed for user `c28c5e12dbb444cab4e376a3015c6bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:18.578400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2c103b99-26b6-4bd9-b87d-d9e42985b6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:18.602094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:18.604320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:18.753462 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2305/4649] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:18.795322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:18.796178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:18.823833 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2345/4650] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:18.838041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:18.838909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:18.839410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:18.875920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] MFA Rules not processed for user `c28c5e12dbb444cab4e376a3015c6bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:18.877846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:18.889613 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2346/4651] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:14:18.903478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:18.905249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:18.916830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:18.950469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe6518-730b-49ee-b992-68058e0a4582 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:18.958513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:18.959151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:18.959924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:18.961707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:18.984815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:18.989186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:19.015247 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2306/4652] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:19.048473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fccb0-64da-4a88-8ebf-91ffe70ce0e4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:19.056836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161db609-b10b-4863-a213-aa3b77c220f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:19.057824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161db609-b10b-4863-a213-aa3b77c220f5 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:19.058914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161db609-b10b-4863-a213-aa3b77c220f5 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:19.060072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161db609-b10b-4863-a213-aa3b77c220f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:19.072708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161db609-b10b-4863-a213-aa3b77c220f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:19.075780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161db609-b10b-4863-a213-aa3b77c220f5 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:19.089067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2347/4653] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:14:19.668112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:19.669728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:19.694977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115e4e6-a89d-4dfb-88f3-c93d90a6ffed service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:19.715821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:19.718183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:19.718655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:19.721222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] PATH_INFO: `/v3/projects/f23e44d8a5094104b35d0288d33f0ab6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:19.723226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f23e44d8a5094104b35d0288d33f0ab6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:19.732147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:19.916145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f23e44d8a5094104b35d0288d33f0ab6', 'request_id': 'req-e18e012e-4b04-4a79-85ad-b1e6dabc9661'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:19.935610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f23e44d8a5094104b35d0288d33f0ab6', 'request_id': 'req-e18e012e-4b04-4a79-85ad-b1e6dabc9661'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:19.951067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f23e44d8a5094104b35d0288d33f0ab6', 'request_id': 'req-e18e012e-4b04-4a79-85ad-b1e6dabc9661'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:19.980456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] The token cache is being invalidate because project f23e44d8a5094104b35d0288d33f0ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:19.981254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:19.981676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e18e012e-4b04-4a79-85ad-b1e6dabc9661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:19.983491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2307/4654] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:19 2022] DELETE /identity/v3/projects/f23e44d8a5094104b35d0288d33f0ab6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:19.989344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0839f31-3070-4332-adf9-2dbb672af902 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:19.990123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0839f31-3070-4332-adf9-2dbb672af902 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:19.990441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0839f31-3070-4332-adf9-2dbb672af902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.021629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c0839f31-3070-4332-adf9-2dbb672af902 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:20.025346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c0839f31-3070-4332-adf9-2dbb672af902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:20.130182 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2348/4655] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:14:20.140313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d77d-f6f4-4246-889c-6e3b6e78f7b0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.140644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d77d-f6f4-4246-889c-6e3b6e78f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.140965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d77d-f6f4-4246-889c-6e3b6e78f7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.186900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a722d77d-f6f4-4246-889c-6e3b6e78f7b0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:20.188867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a722d77d-f6f4-4246-889c-6e3b6e78f7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:20.212058 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2308/4656] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:20.213717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0839f31-3070-4332-adf9-2dbb672af902 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.234589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0839f31-3070-4332-adf9-2dbb672af902 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.249231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.250391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.250741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.251524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.255330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:14:20.257460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.266355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:20.305385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2349/4657] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:14:20.306708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d77d-f6f4-4246-889c-6e3b6e78f7b0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.328945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d77d-f6f4-4246-889c-6e3b6e78f7b0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.343017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.344228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.344519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.344826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.345481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.347811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:20.394960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:20.409515 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2309/4658] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:20 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:20.415282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.436776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f530fe-1da2-49c5-80c1-d80a9d035a99 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.489456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.491922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:20.581910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.583018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.583444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.583846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.586316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.589759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:20.638025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713552-3135-4117-8a02-123f7bb20e62 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.653290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.654542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.655017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.655517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] PATH_INFO: `/v3/projects/0ae5b11810804fc0a2b68840f34572d4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.658017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae5b11810804fc0a2b68840f34572d4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.663527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:20.673096 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2350/4659] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:20 2022] POST /identity/v3/users => generated 284 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:20.674464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.704819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094ceef-a4fe-4f0d-ab1a-85a69c5b98ed admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.713887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.716146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.716989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.717514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.718436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.722704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:20.733077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:20.753305 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2351/4660] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:20.759330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.789803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75abfad-3234-4226-891d-f52951262c17 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.818170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae5b11810804fc0a2b68840f34572d4', 'request_id': 'req-afd9b134-8844-43e9-8ac7-3580e9ce3c47'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:20.838625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae5b11810804fc0a2b68840f34572d4', 'request_id': 'req-afd9b134-8844-43e9-8ac7-3580e9ce3c47'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:20.860270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae5b11810804fc0a2b68840f34572d4', 'request_id': 'req-afd9b134-8844-43e9-8ac7-3580e9ce3c47'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:20.879185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] The token cache is being invalidate because project 0ae5b11810804fc0a2b68840f34572d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:20.887650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.888730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.888846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.889221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.890017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.892106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:20.895052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:20.897617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:20.906156 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2310/4661] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:20 2022] DELETE /identity/v3/projects/0ae5b11810804fc0a2b68840f34572d4 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:20.911585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.914063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:20.915940 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2352/4662] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:20.920308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:20.923855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:20.942603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0821f-26c4-4624-ad7b-80392acac6fd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.946495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9b134-8844-43e9-8ac7-3580e9ce3c47 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:20.950133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3d56c-cb53-42e4-8a20-fadc1aeba565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.951124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d56c-cb53-42e4-8a20-fadc1aeba565 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.951765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d56c-cb53-42e4-8a20-fadc1aeba565 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.952421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d56c-cb53-42e4-8a20-fadc1aeba565 admin admin] PATH_INFO: `/v3/projects/1d229e5b879b481082284236c9f8c446/users/a7b8599d0b40424087e52d8e35b71ec4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.964303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:20.965242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:20.965687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:20.966040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] PATH_INFO: `/v3/users/91fb56177c7443d2903b00d6c42d9182` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:20.969783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91fb56177c7443d2903b00d6c42d9182)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.971887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:20.981239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3d56c-cb53-42e4-8a20-fadc1aeba565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d229e5b879b481082284236c9f8c446, user_id=a7b8599d0b40424087e52d8e35b71ec4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:20.986745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3d56c-cb53-42e4-8a20-fadc1aeba565 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:21.060211 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2353/4663] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:20 2022] PUT /identity/v3/projects/1d229e5b879b481082284236c9f8c446/users/a7b8599d0b40424087e52d8e35b71ec4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:14:21.065433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:21.065747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:21.067154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:21.104183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91fb56177c7443d2903b00d6c42d9182'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:21.104964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91fb56177c7443d2903b00d6c42d9182'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:21.118558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:21.119759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:21.123162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91fb56177c7443d2903b00d6c42d9182'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:21.142831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91fb56177c7443d2903b00d6c42d9182'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:21.160719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91fb56177c7443d2903b00d6c42d9182'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:21.165456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:21.174969 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2311/4664] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:20 2022] DELETE /identity/v3/users/91fb56177c7443d2903b00d6c42d9182 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:21.176682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:21.182230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:21.332442 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2354/4665] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:21.373967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:21.377109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:21.486209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef54b08-0f1d-45dd-b7f1-f9d4f03ab457 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:21.489020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e029f6-1f20-42f1-9e33-97365f590c2a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:21.503837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:21.504507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:21.504802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:21.505077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:21.507410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:21.538987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:21.540645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:21.543393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:21.571060 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2312/4666] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:21.590420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:21.592259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:21.592861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:21.593641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:21.770996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:21.773541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:21.809241 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2355/4667] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:14:22.693507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:22.728544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2c11b-09ec-493e-b822-473eeff19368 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:22.743798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:22.745420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:22.746044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:22.746474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:22.750122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:22.784156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:22.789706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:22.790423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:22.792158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:22.824734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd07e73-bd19-4ab4-aadf-992383c5cc8f service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:22.828854 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2313/4668] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:22.831640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:22.832574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:22.833182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:22.833654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] PATH_INFO: `/v3/projects/a2f74f0f59d84b2691762f862d5d98fb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:22.836363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f74f0f59d84b2691762f862d5d98fb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:22.839819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:22.943227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f74f0f59d84b2691762f862d5d98fb', 'request_id': 'req-6e2c4a75-f716-48a7-8efa-0c623e94ef99'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:22.982980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f74f0f59d84b2691762f862d5d98fb', 'request_id': 'req-6e2c4a75-f716-48a7-8efa-0c623e94ef99'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:22.995119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f74f0f59d84b2691762f862d5d98fb', 'request_id': 'req-6e2c4a75-f716-48a7-8efa-0c623e94ef99'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.001481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:23.003874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:23.038045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbccc5-4009-4405-83ee-e65509efc576 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:23.047892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] The token cache is being invalidate because project a2f74f0f59d84b2691762f862d5d98fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:23.048200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.050110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e2c4a75-f716-48a7-8efa-0c623e94ef99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.058434 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2356/4669] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:22 2022] DELETE /identity/v3/projects/a2f74f0f59d84b2691762f862d5d98fb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:23.060236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c97a26-d81f-4238-839d-c679c82deacd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.060896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c97a26-d81f-4238-839d-c679c82deacd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.061157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c97a26-d81f-4238-839d-c679c82deacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.104290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-95c97a26-d81f-4238-839d-c679c82deacd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:23.108729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-95c97a26-d81f-4238-839d-c679c82deacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:23.231135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:23.256556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.262259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.262852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] PATH_INFO: `/v3/users/1adb648b5b9a4d448391ab2741e56130` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.269469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1adb648b5b9a4d448391ab2741e56130)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:23.275119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:23.347784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:23.384435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2357/4670] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:23.386050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efccccb-0aed-42d5-af42-6423c440fe83 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.386340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efccccb-0aed-42d5-af42-6423c440fe83 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.386616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efccccb-0aed-42d5-af42-6423c440fe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.452271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9efccccb-0aed-42d5-af42-6423c440fe83 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:23.453686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9efccccb-0aed-42d5-af42-6423c440fe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:23.473652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1adb648b5b9a4d448391ab2741e56130'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.474359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1adb648b5b9a4d448391ab2741e56130'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.479338 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2358/4671] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:23.484437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efccccb-0aed-42d5-af42-6423c440fe83 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:23.497356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efccccb-0aed-42d5-af42-6423c440fe83 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:23.505741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:23.506522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.507417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.507603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1adb648b5b9a4d448391ab2741e56130'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.507715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.508398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:14:23.508754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:23.510993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:23.529043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1adb648b5b9a4d448391ab2741e56130'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.537308 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2359/4672] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:14:23.537737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:23.539358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:23.542911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:23.564040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53feab25-38e9-4454-b418-8657f1da871a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:23.571598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:23.572950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.573272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.573527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.574152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5de89f40-683e-4b10-bc99-8193904ac273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1adb648b5b9a4d448391ab2741e56130'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:23.574287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:23.578647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:23.593863 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2314/4673] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:22 2022] DELETE /identity/v3/users/1adb648b5b9a4d448391ab2741e56130 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:14:23.595662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb913246-b787-4230-a62c-78f206025d53 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.596036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb913246-b787-4230-a62c-78f206025d53 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.596413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb913246-b787-4230-a62c-78f206025d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.635685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2360/4674] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:23 2022] POST /identity/v3/projects => generated 361 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:23.637659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:23.639400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cb913246-b787-4230-a62c-78f206025d53 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:23.640907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb913246-b787-4230-a62c-78f206025d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:23.796739 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2315/4675] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:23.811205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb913246-b787-4230-a62c-78f206025d53 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:23.813054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb913246-b787-4230-a62c-78f206025d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:23.813962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efd1f-40dd-40e7-80e9-e13ea3f55444 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:23.821617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:23.822599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.822884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.823732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.824394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:23.826765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:23.864586 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2361/4676] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:23 2022] POST /identity/v3/users => generated 281 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:23.866275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:23.898865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449b32b-3b79-4407-a070-82f8ea1fc6f7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:23.905906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:23.906588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.906873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.907147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.908156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:23.910190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:23.940256 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2362/4677] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:23.944616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:23.967433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660d664-bb72-4c3f-a226-41a364d5cf0f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:23.986317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:23.988208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:23.988493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:23.988791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:23.991767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:23.995967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:24.017522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb913246-b787-4230-a62c-78f206025d53 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:24.031314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2363/4678] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:24.035805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:24.036699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16529c20-4d67-4e5e-a45c-770727dae785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:24.042989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16529c20-4d67-4e5e-a45c-770727dae785 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.042989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16529c20-4d67-4e5e-a45c-770727dae785 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.042989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16529c20-4d67-4e5e-a45c-770727dae785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:24.065056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199eeb9-8171-4324-822a-1048ab8bbdbb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:24.080032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:24.082005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.082289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.082565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] PATH_INFO: `/v3/projects/709099acc61b422c877a2e2d5140611f/users/de0710c9ac03425893be016ccb8fd75a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:24.083097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16529c20-4d67-4e5e-a45c-770727dae785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:24.087483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16529c20-4d67-4e5e-a45c-770727dae785 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:24.128603 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2316/4679] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:14:24.146394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=709099acc61b422c877a2e2d5140611f, user_id=de0710c9ac03425893be016ccb8fd75a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:24.160777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:24.198649 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2364/4680] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:24 2022] PUT /identity/v3/projects/709099acc61b422c877a2e2d5140611f/users/de0710c9ac03425893be016ccb8fd75a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:24.210726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.210913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.211213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:24.272239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] MFA Rules not processed for user `de0710c9ac03425893be016ccb8fd75a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:24.276587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:24.396959 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2317/4681] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:24.438516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:24.441173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:24.518767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:24.520233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:24.573905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ebf76-34ee-48a4-9c76-8c04bc9e2f41 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:24.581132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:24.581551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:24.581853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.582507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.582773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:24.628595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:24.630578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:24.658834 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2365/4682] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:24.667863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:24.676771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:24.709388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b2b7f-4160-4d6a-997e-f30f59cd419e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:24.718322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:24.719545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.720703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.721246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] PATH_INFO: `/v3/projects/59136525ae52452490d15c6c2d623cad` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:24.722828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59136525ae52452490d15c6c2d623cad)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:24.725073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:24.774104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4896c-dc00-4fb5-ac38-b926059df8ae service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:24.779755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:24.780855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.781142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.781416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] PATH_INFO: `/v3/users/de0710c9ac03425893be016ccb8fd75a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:24.782942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de0710c9ac03425893be016ccb8fd75a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:24.785050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:24.797505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59136525ae52452490d15c6c2d623cad', 'request_id': 'req-e54c0bc1-381c-46cb-9c06-24976dadbd55'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.810479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59136525ae52452490d15c6c2d623cad', 'request_id': 'req-e54c0bc1-381c-46cb-9c06-24976dadbd55'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.827082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59136525ae52452490d15c6c2d623cad', 'request_id': 'req-e54c0bc1-381c-46cb-9c06-24976dadbd55'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.853869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] The token cache is being invalidate because project 59136525ae52452490d15c6c2d623cad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:24.854174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.857142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e54c0bc1-381c-46cb-9c06-24976dadbd55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.862467 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2318/4683] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:24 2022] DELETE /identity/v3/projects/59136525ae52452490d15c6c2d623cad => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:24.869151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb734b-a45f-4111-b28f-9c6bb7fcce23 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.869445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb734b-a45f-4111-b28f-9c6bb7fcce23 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.869859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb734b-a45f-4111-b28f-9c6bb7fcce23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:24.885430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de0710c9ac03425893be016ccb8fd75a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.887213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de0710c9ac03425893be016ccb8fd75a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.909927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-67bb734b-a45f-4111-b28f-9c6bb7fcce23 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:24.913256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de0710c9ac03425893be016ccb8fd75a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.921297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-67bb734b-a45f-4111-b28f-9c6bb7fcce23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:24.926231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de0710c9ac03425893be016ccb8fd75a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.928172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87cbe60b-8b05-4075-990d-267edd961906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de0710c9ac03425893be016ccb8fd75a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:24.987222 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2366/4684] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:24 2022] DELETE /identity/v3/users/de0710c9ac03425893be016ccb8fd75a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:24.989288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:24.989990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:24.990971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.045291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:25.047206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:25.109979 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2319/4685] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:25.121500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e528d72-f824-473a-b5b8-ea8fb6ceb309 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.122930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e528d72-f824-473a-b5b8-ea8fb6ceb309 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.123378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e528d72-f824-473a-b5b8-ea8fb6ceb309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.159704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5e528d72-f824-473a-b5b8-ea8fb6ceb309 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:25.164702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e528d72-f824-473a-b5b8-ea8fb6ceb309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:25.173028 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2367/4686] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:25.179238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.181414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:25.190020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2320/4687] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:14:25.194694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e528d72-f824-473a-b5b8-ea8fb6ceb309 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.206820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e528d72-f824-473a-b5b8-ea8fb6ceb309 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.224038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:25.229856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.231774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.232279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.233402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:14:25.235067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:25.236930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:25.241478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:25.270962 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2321/4688] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:14:25.276849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.300447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e0ff1-1d9a-449a-b86e-2655ecdeb88b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.301770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16562f3-4aaa-43f0-a2e9-993e7f5caea1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.307866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:25.308574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.308872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.309143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.311791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:25.313143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.313651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.314024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.314916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:25.318983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:25.340875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:25.342447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:25.357825 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2368/4689] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:14:25.373656 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2322/4690] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:25 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:25.377779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.396615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb6508-8761-4746-b004-bcc5d6ba42a7 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.487836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:25.488897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.489564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.489970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.490784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:25.495144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:25.535461 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2369/4691] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:25 2022] POST /identity/v3/users => generated 282 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:25.540689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.558849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a176cae-c450-42fe-bc8a-a604b65e12be admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.571798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:25.576101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.576646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.577130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.578312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:25.580827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:25.583509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:25.649232 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2323/4692] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:25.649578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.666052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc525b53-ba9b-408a-a9e0-9ea77591854d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.672284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:25.673036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.673506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.674125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.674877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:25.677607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:25.748246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.756007 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2370/4693] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:14:25.762690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fe07c-1c03-4597-9451-03c661c41915 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.769430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c872a-29ac-4009-a140-86872d5c6e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:25.770277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c872a-29ac-4009-a140-86872d5c6e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:25.770701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c872a-29ac-4009-a140-86872d5c6e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:25.771170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c872a-29ac-4009-a140-86872d5c6e60 admin admin] PATH_INFO: `/v3/projects/8f1ae55ddbd8453c885e531ab0620beb/users/e8d785381149436fb7caf8ff5e55c9eb/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:25.820341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c872a-29ac-4009-a140-86872d5c6e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1ae55ddbd8453c885e531ab0620beb, user_id=e8d785381149436fb7caf8ff5e55c9eb, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:25.830652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:25.835475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c872a-29ac-4009-a140-86872d5c6e60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:25.838944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:25.869437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aebe8c-5bea-4ae5-a59b-613e0bf78270 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:25.878685 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2324/4694] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:25 2022] PUT /identity/v3/projects/8f1ae55ddbd8453c885e531ab0620beb/users/e8d785381149436fb7caf8ff5e55c9eb/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:26.001575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e84201-5b10-4570-90e8-8450ec077796 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:26.002020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e84201-5b10-4570-90e8-8450ec077796 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:26.002295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e84201-5b10-4570-90e8-8450ec077796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:26.050277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:26.050999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:26.051168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:26.051442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] PATH_INFO: `/v3/projects/709099acc61b422c877a2e2d5140611f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:26.058937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=709099acc61b422c877a2e2d5140611f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:26.079732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:26.079921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-33e84201-5b10-4570-90e8-8450ec077796 None None] MFA Rules not processed for user `e8d785381149436fb7caf8ff5e55c9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:26.081101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-33e84201-5b10-4570-90e8-8450ec077796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:26.205684 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2325/4695] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:14:26.217031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e84201-5b10-4570-90e8-8450ec077796 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:26.243754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:26.246614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709099acc61b422c877a2e2d5140611f', 'request_id': 'req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:26.267869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709099acc61b422c877a2e2d5140611f', 'request_id': 'req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:26.288075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709099acc61b422c877a2e2d5140611f', 'request_id': 'req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:26.313371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] The token cache is being invalidate because project 709099acc61b422c877a2e2d5140611f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:26.314580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:26.317063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c0ac2a-8fc7-4ac9-957e-da6cf5e45263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:26.321271 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2371/4696] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:25 2022] DELETE /identity/v3/projects/709099acc61b422c877a2e2d5140611f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:14:26.326664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ad030-3c0c-4310-850b-81b783ae501c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:26.327017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ad030-3c0c-4310-850b-81b783ae501c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:26.327360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ad030-3c0c-4310-850b-81b783ae501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:26.341954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e84201-5b10-4570-90e8-8450ec077796 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:26.375920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bf3ad030-3c0c-4310-850b-81b783ae501c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:26.378222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bf3ad030-3c0c-4310-850b-81b783ae501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:26.455605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:26.458862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:26.459308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:26.459661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:26.463457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:26.560065 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2372/4697] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:26.576279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e82ba6-8068-4fd4-8c52-8220c6d1ab9c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:26.576584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e82ba6-8068-4fd4-8c52-8220c6d1ab9c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:26.581110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e82ba6-8068-4fd4-8c52-8220c6d1ab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:26.609069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:26.653836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-42e82ba6-8068-4fd4-8c52-8220c6d1ab9c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:26.655327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-42e82ba6-8068-4fd4-8c52-8220c6d1ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:26.697649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:26.699539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:26.704103 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2373/4698] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:26.717700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e82ba6-8068-4fd4-8c52-8220c6d1ab9c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:26.727322 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2326/4699] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:26.731056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e82ba6-8068-4fd4-8c52-8220c6d1ab9c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:26.755564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:26.756408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:26.756957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:26.757288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:26.757966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:14:26.758272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:26.760048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:26.779085 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2374/4700] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:14:26.783801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:26.804965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138de6a-83b9-4aef-927e-443cd49f143a service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:26.817916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:26.819580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:26.819938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:26.820293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:26.822686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:26.829868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:26.889076 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2327/4701] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:26 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:26.893868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:26.910041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792a44c-ca0b-4cfc-a171-27682171627d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:27.046194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:27.049071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:27.049615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:27.052195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:27.055444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:27.057845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:27.093001 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2375/4702] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:26 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:14:27.096979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:27.111154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84969-0523-4a16-b44d-b054164949d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:27.127115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:27.127777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:27.128036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:27.128315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:27.129033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:27.130804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:27.143189 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2328/4703] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:14:27.147778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:27.160965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fa130-b421-4285-b714-85eae6a6806c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:27.187160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13fedd-5405-4ae1-8a5b-15b62c7c86c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:27.187853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13fedd-5405-4ae1-8a5b-15b62c7c86c8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:27.188155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13fedd-5405-4ae1-8a5b-15b62c7c86c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:27.188412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13fedd-5405-4ae1-8a5b-15b62c7c86c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:27.189135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13fedd-5405-4ae1-8a5b-15b62c7c86c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:27.190730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13fedd-5405-4ae1-8a5b-15b62c7c86c8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:27.212761 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2376/4704] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:27.218103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:27.243285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb6760-d4b9-4bbd-abab-12272ff3c69c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:27.252079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:27.253629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:27.255629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:27.255935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] PATH_INFO: `/v3/projects/35e93ba1c9624e81aeed5f897483b28f/users/bdf718f851884452860701c998796495/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:27.321677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e93ba1c9624e81aeed5f897483b28f, user_id=bdf718f851884452860701c998796495, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:27.326937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:27.363805 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2329/4705] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:27 2022] PUT /identity/v3/projects/35e93ba1c9624e81aeed5f897483b28f/users/bdf718f851884452860701c998796495/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:27.375252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104d4d-7836-4918-b8d7-27e7568b4393 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:27.375822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104d4d-7836-4918-b8d7-27e7568b4393 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:27.376105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104d4d-7836-4918-b8d7-27e7568b4393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:27.433619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ca104d4d-7836-4918-b8d7-27e7568b4393 None None] MFA Rules not processed for user `bdf718f851884452860701c998796495`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:27.434875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ca104d4d-7836-4918-b8d7-27e7568b4393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:27.493065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:27.667589 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2377/4706] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:27.668793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:27.913864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3da6d7-778b-4d34-a8f6-5b250b0aa5f5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:27.932142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:27.933116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:27.933299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:27.933628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:27.935890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:27.964408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:28.038643 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2330/4707] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:27 2022] POST /identity/v3/projects => generated 385 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:28.042977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104d4d-7836-4918-b8d7-27e7568b4393 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:28.066100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104d4d-7836-4918-b8d7-27e7568b4393 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:28.230870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:28.231503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:28.231778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:28.232047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:28.232691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:28.234836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:28.294501 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2378/4708] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:28 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:28.297538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:28.314569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361376a7-0183-4575-8caf-1de6015a783b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:28.338342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:28.338926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:28.339228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:28.339541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:28.340217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:28.342008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:28.373106 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2331/4709] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:28.374422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:28.390551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea78f3-656b-4a3a-be5a-2e6f783a6969 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:28.406961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1dd5-2b49-41ed-ba20-adb0bf72407a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:28.408195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1dd5-2b49-41ed-ba20-adb0bf72407a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:28.408903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1dd5-2b49-41ed-ba20-adb0bf72407a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:28.409423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1dd5-2b49-41ed-ba20-adb0bf72407a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:28.410613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1dd5-2b49-41ed-ba20-adb0bf72407a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:28.417372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1dd5-2b49-41ed-ba20-adb0bf72407a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:28.462686 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2379/4710] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:28.467084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:28.479555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40bbc-feb9-42ea-a6ea-e6d41f9b16c8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:28.494986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:28.495730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:28.496002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:28.496286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] PATH_INFO: `/v3/projects/4eac99ba3e9848c1b23a8195da53b769/users/1147d5d62b3c406990ed49d0b3038882/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:28.547620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eac99ba3e9848c1b23a8195da53b769, user_id=1147d5d62b3c406990ed49d0b3038882, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:28.554342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:28.584792 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2332/4711] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:28 2022] PUT /identity/v3/projects/4eac99ba3e9848c1b23a8195da53b769/users/1147d5d62b3c406990ed49d0b3038882/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:28.597535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a894-9897-451d-a949-b0fa0848c8fa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:28.597535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a894-9897-451d-a949-b0fa0848c8fa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:28.597535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a894-9897-451d-a949-b0fa0848c8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:28.656921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4d56a894-9897-451d-a949-b0fa0848c8fa None None] MFA Rules not processed for user `1147d5d62b3c406990ed49d0b3038882`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:28.659462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4d56a894-9897-451d-a949-b0fa0848c8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:28.821134 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2380/4712] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:28.826452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:28.915044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:29.008138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd046e-1167-4217-84f0-f8be04a5dbf2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:29.023368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:29.024705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:29.025018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:29.025302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:29.026002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:29.048145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:29.126320 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2333/4713] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:28 2022] POST /identity/v3/projects => generated 383 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:29.128074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a894-9897-451d-a949-b0fa0848c8fa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:29.147066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a894-9897-451d-a949-b0fa0848c8fa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:29.199682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:29.301450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:29.304941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:29.304941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:29.304941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:29.304941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:29.307375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:29.346461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:29.348819 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2381/4714] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:29 2022] POST /identity/v3/users => generated 298 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:29.360375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e6855-00da-47da-ba02-dbc58799a892 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:29.376473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:29.377152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:29.378606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:29.378976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:29.379786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:29.382622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:29.396759 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2334/4715] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:29.401087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:29.412827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a827c-fe0e-48f1-b18f-c371d67097d8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:29.421210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:29.422340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:29.422861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:29.423366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] PATH_INFO: `/v3/projects/dd29f90d10454718b6c12a281befc50e/users/b36cb361d53e4b1c93ef0e54d1e449b6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:29.445332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd29f90d10454718b6c12a281befc50e, user_id=b36cb361d53e4b1c93ef0e54d1e449b6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:29.452768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:29.479069 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2382/4716] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:29 2022] PUT /identity/v3/projects/dd29f90d10454718b6c12a281befc50e/users/b36cb361d53e4b1c93ef0e54d1e449b6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:29.483661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:29.503986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ae4b7-58b4-4a69-b7c1-e492acb287c3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:29.655592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95002451-517c-4267-a036-3aa550bdb2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:29.657388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95002451-517c-4267-a036-3aa550bdb2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:29.657682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95002451-517c-4267-a036-3aa550bdb2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:29.657954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95002451-517c-4267-a036-3aa550bdb2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:29.660899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95002451-517c-4267-a036-3aa550bdb2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:29.665158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95002451-517c-4267-a036-3aa550bdb2d0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:29.697641 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2335/4717] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:14:29.697872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:29.727211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6b8d9-4cc8-4998-ae96-ea0cca509c7d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:29.744499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:29.747195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:29.747590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:29.748870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] PATH_INFO: `/v3/projects/dd29f90d10454718b6c12a281befc50e/users/b36cb361d53e4b1c93ef0e54d1e449b6/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:29.755067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd29f90d10454718b6c12a281befc50e, user_id=b36cb361d53e4b1c93ef0e54d1e449b6, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:29.773070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:29.815761 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2383/4718] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:29 2022] PUT /identity/v3/projects/dd29f90d10454718b6c12a281befc50e/users/b36cb361d53e4b1c93ef0e54d1e449b6/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:29.829724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:29.830228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:29.830509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:29.915591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] MFA Rules not processed for user `b36cb361d53e4b1c93ef0e54d1e449b6`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:29.917053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:30.088250 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2336/4719] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:30.102686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:30.105011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:30.192173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc775b-e32a-42c7-9af2-0aec0c8ab59b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:30.197143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:30.198426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:30.199074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:30.199783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:30.302042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:30.304438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:30.329963 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2384/4720] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:32.223871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:32.226048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:32.252470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec3c58-f5aa-4b0a-a1eb-84728b558c88 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:32.259370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331da8a-2489-43fe-9415-8ddf2e34ecad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:32.260272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331da8a-2489-43fe-9415-8ddf2e34ecad service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:32.260993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331da8a-2489-43fe-9415-8ddf2e34ecad service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:32.261416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331da8a-2489-43fe-9415-8ddf2e34ecad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:32.297171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:32.375529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331da8a-2489-43fe-9415-8ddf2e34ecad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:32.377869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331da8a-2489-43fe-9415-8ddf2e34ecad service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:32.393603 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2337/4721] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:33.484402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:33.498884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f096e-5dd5-44d4-8813-ec936138dda3 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:33.505001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114f2fa-3a8c-474c-ac84-181b1a5ff211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:33.505699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f2fa-3a8c-474c-ac84-181b1a5ff211 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:33.506193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f2fa-3a8c-474c-ac84-181b1a5ff211 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:33.506510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f2fa-3a8c-474c-ac84-181b1a5ff211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:33.516836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114f2fa-3a8c-474c-ac84-181b1a5ff211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:33.518471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114f2fa-3a8c-474c-ac84-181b1a5ff211 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:33.541788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2385/4722] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:37.923737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1895f2c-78ea-4f10-ab9a-8c5f46e7cc6b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:37.924399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1895f2c-78ea-4f10-ab9a-8c5f46e7cc6b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:37.924399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1895f2c-78ea-4f10-ab9a-8c5f46e7cc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:37.960384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e1895f2c-78ea-4f10-ab9a-8c5f46e7cc6b None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:37.962265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e1895f2c-78ea-4f10-ab9a-8c5f46e7cc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:38.061743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2338/4723] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:38.075128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:38.076078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:38.076601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:38.109189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:38.110985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:38.124941 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2386/4724] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:38.126977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1895f2c-78ea-4f10-ab9a-8c5f46e7cc6b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:38.137952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:38.139983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:38.187332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:38.241869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1895f2c-78ea-4f10-ab9a-8c5f46e7cc6b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:38.250640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:38.253091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:38.254802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:38.254802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] PATH_INFO: `/v3/users/bdf718f851884452860701c998796495` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:38.256516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf718f851884452860701c998796495)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:38.259103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:38.279108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8526b-8ca7-431b-b594-65ba0f4d683a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:38.288186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d98d33-327c-43b9-aadd-b73acc84c291 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:38.289405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d98d33-327c-43b9-aadd-b73acc84c291 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:38.289908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d98d33-327c-43b9-aadd-b73acc84c291 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:38.290383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d98d33-327c-43b9-aadd-b73acc84c291 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:38.301730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d98d33-327c-43b9-aadd-b73acc84c291 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:38.304683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d98d33-327c-43b9-aadd-b73acc84c291 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:38.327853 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2387/4725] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:14:38.345768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf718f851884452860701c998796495'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:38.346713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf718f851884452860701c998796495'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:38.349596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:38.358276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf718f851884452860701c998796495'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:38.375018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf718f851884452860701c998796495'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:38.398473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf718f851884452860701c998796495'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:38.409216 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2339/4726] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:38 2022] DELETE /identity/v3/users/bdf718f851884452860701c998796495 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:38.412396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:38.412920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:38.413741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:38.439046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:38.440405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:38.552962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2388/4727] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:38.593648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:38.595521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:38.735466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afb8cc-9d0f-412c-9e6d-3844a1912427 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:38.741419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:38.742128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:38.742414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:38.742681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:38.757211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:38.759253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:38.794765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2340/4728] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:39.891716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:39.893345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:39.923256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4187a8d-6154-416e-84e0-705694eec939 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:39.936637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:39.937757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:39.938514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:39.938970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] PATH_INFO: `/v3/projects/35e93ba1c9624e81aeed5f897483b28f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:39.942165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e93ba1c9624e81aeed5f897483b28f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:39.951939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:40.065193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e93ba1c9624e81aeed5f897483b28f', 'request_id': 'req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.086609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e93ba1c9624e81aeed5f897483b28f', 'request_id': 'req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.097884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e93ba1c9624e81aeed5f897483b28f', 'request_id': 'req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.119882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] The token cache is being invalidate because project 35e93ba1c9624e81aeed5f897483b28f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:40.120364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.122640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.127311 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2389/4729] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:39 2022] DELETE /identity/v3/projects/35e93ba1c9624e81aeed5f897483b28f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:40.128060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:40.140539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed04ebd-600c-487b-901f-d93f8243ae8b service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:40.236047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:40.238454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:40.238454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:40.238454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] PATH_INFO: `/v3/users/1147d5d62b3c406990ed49d0b3038882` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:40.239907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1147d5d62b3c406990ed49d0b3038882)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:40.242682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:40.377655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1147d5d62b3c406990ed49d0b3038882'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.379015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1147d5d62b3c406990ed49d0b3038882'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.396567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1147d5d62b3c406990ed49d0b3038882'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.413227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1147d5d62b3c406990ed49d0b3038882'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.464275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:40.484823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1147d5d62b3c406990ed49d0b3038882'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:40.505543 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2341/4730] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:40 2022] DELETE /identity/v3/users/1147d5d62b3c406990ed49d0b3038882 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:41.344178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:41.348718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:41.382759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3e7f7-1f03-44d1-8a77-e1b3736ebf16 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:41.490469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:41.563987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:41.564757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:41.565095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:41.565409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] PATH_INFO: `/v3/projects/4eac99ba3e9848c1b23a8195da53b769` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:41.569886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eac99ba3e9848c1b23a8195da53b769)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:41.577887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:41.764249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eac99ba3e9848c1b23a8195da53b769', 'request_id': 'req-ca97c4c7-0681-491d-9d75-02a89442fa1b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:41.786174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eac99ba3e9848c1b23a8195da53b769', 'request_id': 'req-ca97c4c7-0681-491d-9d75-02a89442fa1b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:41.797661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eac99ba3e9848c1b23a8195da53b769', 'request_id': 'req-ca97c4c7-0681-491d-9d75-02a89442fa1b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:41.849279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] The token cache is being invalidate because project 4eac99ba3e9848c1b23a8195da53b769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:41.851390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:41.854831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:41.856980 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2390/4731] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:41 2022] DELETE /identity/v3/projects/4eac99ba3e9848c1b23a8195da53b769 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:41.864096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:41.870093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:41.913446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da21c36-33ba-4f57-b9e8-ed313f83ad12 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:42.066228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:42.067632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:42.068096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:42.068468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] PATH_INFO: `/v3/users/b36cb361d53e4b1c93ef0e54d1e449b6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:42.071440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36cb361d53e4b1c93ef0e54d1e449b6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:42.075321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:42.185735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36cb361d53e4b1c93ef0e54d1e449b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:42.187686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36cb361d53e4b1c93ef0e54d1e449b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:42.210783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36cb361d53e4b1c93ef0e54d1e449b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:42.233361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36cb361d53e4b1c93ef0e54d1e449b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:42.259154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-100a4e6b-5d8e-4060-862d-ca72dd6982ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36cb361d53e4b1c93ef0e54d1e449b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:42.276620 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2342/4732] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:41 2022] DELETE /identity/v3/users/b36cb361d53e4b1c93ef0e54d1e449b6 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:42.796329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:42.800620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:42.873010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c4c7-0681-491d-9d75-02a89442fa1b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:43.035706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:43.036623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.037259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.038590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] PATH_INFO: `/v3/projects/dd29f90d10454718b6c12a281befc50e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.045988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd29f90d10454718b6c12a281befc50e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:43.049784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:43.138145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd29f90d10454718b6c12a281befc50e', 'request_id': 'req-34ae3197-e9f0-4510-a784-a1b4ba55972c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:43.161264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd29f90d10454718b6c12a281befc50e', 'request_id': 'req-34ae3197-e9f0-4510-a784-a1b4ba55972c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:43.170252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:43.173952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd29f90d10454718b6c12a281befc50e', 'request_id': 'req-34ae3197-e9f0-4510-a784-a1b4ba55972c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:43.228691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] The token cache is being invalidate because project dd29f90d10454718b6c12a281befc50e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:43.230497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:43.231661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ae3197-e9f0-4510-a784-a1b4ba55972c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:43.236630 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2391/4733] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:42 2022] DELETE /identity/v3/projects/dd29f90d10454718b6c12a281befc50e => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:43.242776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467d6fa-c7a7-41f5-a140-d20425549bcb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.243714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467d6fa-c7a7-41f5-a140-d20425549bcb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.244225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467d6fa-c7a7-41f5-a140-d20425549bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.283209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1467d6fa-c7a7-41f5-a140-d20425549bcb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:43.284671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1467d6fa-c7a7-41f5-a140-d20425549bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:43.415574 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2343/4734] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:43.422115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dd408-219d-4587-b80f-2d56bc34d62f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.422781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dd408-219d-4587-b80f-2d56bc34d62f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.423217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dd408-219d-4587-b80f-2d56bc34d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.469193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4e2dd408-219d-4587-b80f-2d56bc34d62f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:43.470718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4e2dd408-219d-4587-b80f-2d56bc34d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:43.479802 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2392/4735] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:14:43.484674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467d6fa-c7a7-41f5-a140-d20425549bcb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:43.509691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467d6fa-c7a7-41f5-a140-d20425549bcb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:43.518393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:43.519338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.519714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.520185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.523129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:14:43.523545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:43.525843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:43.534041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:43.549763 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2344/4736] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:14:43.554519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dd408-219d-4587-b80f-2d56bc34d62f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:43.585061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dd408-219d-4587-b80f-2d56bc34d62f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:43.602338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:43.602698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.603068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.603475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.605572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:43.607715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:43.683392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2393/4737] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:14:43 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:43.696103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:43.737352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654baaa-e4a9-4fe4-a43d-ee000184a49d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:43.872460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:43.873291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.873700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.874881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.874881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:43.876859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:43.907541 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2345/4738] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:14:43 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:14:43.912087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:43.925934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea176e1-e6eb-4c34-975c-8e5569e9cf01 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:43.936839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:43.940109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.940537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.940909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.941664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:43.944122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:43.956291 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2394/4739] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:43.961897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:43.975539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed85af-ef25-47eb-b741-f7752df93325 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:43.987024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16c34f-8dd9-4218-9d94-6ef192150114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:43.988089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c34f-8dd9-4218-9d94-6ef192150114 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:43.988586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c34f-8dd9-4218-9d94-6ef192150114 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:43.989206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c34f-8dd9-4218-9d94-6ef192150114 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:43.990036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16c34f-8dd9-4218-9d94-6ef192150114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:43.992280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16c34f-8dd9-4218-9d94-6ef192150114 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:44.010362 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2346/4740] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:14:44.012230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:44.025646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f810a82-b87e-4118-afbb-448bcbc65e74 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:44.032075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:44.032947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:44.034424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:44.034908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] PATH_INFO: `/v3/projects/27c0d3daa08340d89d288bbf811fece9/users/dc369ca971424247b96050e09368a7bb/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:44.063377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c0d3daa08340d89d288bbf811fece9, user_id=dc369ca971424247b96050e09368a7bb, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:44.068717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:44.093736 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2395/4741] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:14:44 2022] PUT /identity/v3/projects/27c0d3daa08340d89d288bbf811fece9/users/dc369ca971424247b96050e09368a7bb/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:44.097764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7938d-47be-4bcb-b06b-f6e957196963 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:44.098053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7938d-47be-4bcb-b06b-f6e957196963 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:44.098408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7938d-47be-4bcb-b06b-f6e957196963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:44.123669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2fa7938d-47be-4bcb-b06b-f6e957196963 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:44.125265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2fa7938d-47be-4bcb-b06b-f6e957196963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:44.208774 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2347/4742] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:14:44.223437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:44.225565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:44.353589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85fbe-1e41-4305-9d84-0306a20cb54a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:44.358681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:44.359472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:44.359756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:44.360028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:44.383619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:44.385625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:44.401541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2396/4743] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:45.643464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e87c8a-a9b8-409b-b111-ddd33f09020e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:45.644069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e87c8a-a9b8-409b-b111-ddd33f09020e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:45.644193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e87c8a-a9b8-409b-b111-ddd33f09020e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:45.687148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-86e87c8a-a9b8-409b-b111-ddd33f09020e None None] MFA Rules not processed for user `dc369ca971424247b96050e09368a7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:45.688444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-86e87c8a-a9b8-409b-b111-ddd33f09020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:45.824735 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2348/4744] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:45.839326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:45.841182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:45.971637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1c87-5b5c-4409-885b-51ea1a106314 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:45.990008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:45.990766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:45.991242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:45.991523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:46.021741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:46.029593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:46.051154 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2397/4745] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:14:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:14:47.727210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:47.727814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:47.727814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:47.739966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:47.782285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] MFA Rules not processed for user `dc369ca971424247b96050e09368a7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:14:47.784233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:47.809902 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2349/4746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:14:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:14:56.042496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=1,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:57.774357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:58.964395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:58.966031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:58.986958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b5c7c-6096-4c31-8976-d46bd779338f service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:58.993052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:58.993990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:58.994404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:58.994919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] PATH_INFO: `/v3/users/a5e050c7389844a78199faef5a0c9da4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:58.996250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e050c7389844a78199faef5a0c9da4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:58.998829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:59.076901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e050c7389844a78199faef5a0c9da4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.077890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e050c7389844a78199faef5a0c9da4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.090876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e050c7389844a78199faef5a0c9da4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.104176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e050c7389844a78199faef5a0c9da4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.105434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e050c7389844a78199faef5a0c9da4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.148553 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2398/4747] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:58 2022] DELETE /identity/v3/users/a5e050c7389844a78199faef5a0c9da4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:59.452930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:59.454483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:59.481600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4f404-7b3c-43f8-b045-4e908f3c7816 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:59.592485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:14:59.598584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:14:59.639840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:59.640930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:59.641435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:59.641932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] PATH_INFO: `/v3/users/c28c5e12dbb444cab4e376a3015c6bd1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:59.644376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28c5e12dbb444cab4e376a3015c6bd1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:59.648396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:59.761523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969637e-be6a-4db9-9721-1e549f36a8d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:14:59.767729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28c5e12dbb444cab4e376a3015c6bd1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.771224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28c5e12dbb444cab4e376a3015c6bd1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.771595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:14:59.772831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:14:59.773430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:14:59.773733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] PATH_INFO: `/v3/projects/326c35c0bc52463e95a56ba3371a276a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:14:59.775183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=326c35c0bc52463e95a56ba3371a276a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:14:59.777488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:14:59.787590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28c5e12dbb444cab4e376a3015c6bd1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.800643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28c5e12dbb444cab4e376a3015c6bd1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.826868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d549b1e3-6fdf-463e-897e-60735892c837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28c5e12dbb444cab4e376a3015c6bd1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.844249 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2350/4748] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:14:59 2022] DELETE /identity/v3/users/c28c5e12dbb444cab4e376a3015c6bd1 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:14:59.920607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326c35c0bc52463e95a56ba3371a276a', 'request_id': 'req-b7c17d08-852a-4818-be15-e00cf1ecf2df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.937345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326c35c0bc52463e95a56ba3371a276a', 'request_id': 'req-b7c17d08-852a-4818-be15-e00cf1ecf2df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.956525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326c35c0bc52463e95a56ba3371a276a', 'request_id': 'req-b7c17d08-852a-4818-be15-e00cf1ecf2df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.990808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] The token cache is being invalidate because project 326c35c0bc52463e95a56ba3371a276a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:14:59.994264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.995009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7c17d08-852a-4818-be15-e00cf1ecf2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:14:59.996585 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2399/4749] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:14:59 2022] DELETE /identity/v3/projects/326c35c0bc52463e95a56ba3371a276a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:15:00.001853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38734c-bf5c-4ac9-9481-55e9289b5567 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.002139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38734c-bf5c-4ac9-9481-55e9289b5567 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.002589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38734c-bf5c-4ac9-9481-55e9289b5567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.030304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8b38734c-bf5c-4ac9-9481-55e9289b5567 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:00.031833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8b38734c-bf5c-4ac9-9481-55e9289b5567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:00.120808 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2351/4750] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:00.130131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651472f5-c5c0-47c6-8ff4-f69842025d7f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.130498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651472f5-c5c0-47c6-8ff4-f69842025d7f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.130762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651472f5-c5c0-47c6-8ff4-f69842025d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.141606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:00.161788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-651472f5-c5c0-47c6-8ff4-f69842025d7f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:00.165220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-651472f5-c5c0-47c6-8ff4-f69842025d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:00.178319 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2400/4751] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:00.180011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38734c-bf5c-4ac9-9481-55e9289b5567 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.193184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38734c-bf5c-4ac9-9481-55e9289b5567 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:00.199714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:00.200264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.200605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.200914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.201603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:15:00.201914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:00.203617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:00.235191 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2352/4752] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:15:00.240249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651472f5-c5c0-47c6-8ff4-f69842025d7f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.251900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651472f5-c5c0-47c6-8ff4-f69842025d7f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:00.258509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:00.259454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.259984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.260346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.261754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:00.264049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:00.315337 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2401/4753] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:15:00.319829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.341619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4791d8c-3ca2-4a1d-9396-3be4768de73f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:00.348189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:00.399484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.401663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:00.481827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d0d8f-b0b1-4da1-a1db-ab70c0047ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:00.482572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d0d8f-b0b1-4da1-a1db-ab70c0047ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.486429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d0d8f-b0b1-4da1-a1db-ab70c0047ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.486751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d0d8f-b0b1-4da1-a1db-ab70c0047ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.487468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d0d8f-b0b1-4da1-a1db-ab70c0047ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:00.489486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d0d8f-b0b1-4da1-a1db-ab70c0047ca8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:00.532308 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2353/4754] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:15:00.534403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68510f8-c1e8-4942-ae50-2eb8cf4429a5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.534704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68510f8-c1e8-4942-ae50-2eb8cf4429a5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.535049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68510f8-c1e8-4942-ae50-2eb8cf4429a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.558132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c68510f8-c1e8-4942-ae50-2eb8cf4429a5 None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:00.559828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c68510f8-c1e8-4942-ae50-2eb8cf4429a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:00.590566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c02d14-9b53-47f9-b139-829e9a07a41c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:00.606505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:00.607091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.607369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.607639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] PATH_INFO: `/v3/projects/1e7e90f89b4844dbafdc1c5e5f17b407` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.609556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7e90f89b4844dbafdc1c5e5f17b407)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:00.611649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:00.668694 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2354/4755] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:00.670053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68510f8-c1e8-4942-ae50-2eb8cf4429a5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.689935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68510f8-c1e8-4942-ae50-2eb8cf4429a5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:00.705515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:00.713559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.727205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.727576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.728363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:00.730184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:00.750258 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2355/4756] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:15:00.757824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.771210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204405e-6fcc-4a08-8f8e-c0a483326560 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:00.782387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7e90f89b4844dbafdc1c5e5f17b407', 'request_id': 'req-02a999b0-5441-4b12-8acb-b6580f2a8e5b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:00.799007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7e90f89b4844dbafdc1c5e5f17b407', 'request_id': 'req-02a999b0-5441-4b12-8acb-b6580f2a8e5b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:00.811310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7e90f89b4844dbafdc1c5e5f17b407', 'request_id': 'req-02a999b0-5441-4b12-8acb-b6580f2a8e5b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:00.831842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] The token cache is being invalidate because project 1e7e90f89b4844dbafdc1c5e5f17b407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:15:00.832296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:00.834017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-02a999b0-5441-4b12-8acb-b6580f2a8e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:00.836722 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2402/4757] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:15:00 2022] DELETE /identity/v3/projects/1e7e90f89b4844dbafdc1c5e5f17b407 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:15:00.846261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e87dd9-4ead-49fd-a8ec-9847d6a95f36 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.847063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e87dd9-4ead-49fd-a8ec-9847d6a95f36 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.849054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e87dd9-4ead-49fd-a8ec-9847d6a95f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.870435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:00.871244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.871511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.871818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.872573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:00.875221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:00.883139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c6e87dd9-4ead-49fd-a8ec-9847d6a95f36 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:00.884598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c6e87dd9-4ead-49fd-a8ec-9847d6a95f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:00.894905 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2356/4758] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:15:00.896514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.898787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:00.909880 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2403/4759] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:00.913866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ee06-75d2-4e66-be11-dc5fa6ef5958 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:00.915826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ae4d3-0d37-450f-aaa3-fea297b8609e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.916127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ae4d3-0d37-450f-aaa3-fea297b8609e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.916412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ae4d3-0d37-450f-aaa3-fea297b8609e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.920717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb79d8-5500-4f98-8320-c4da0f685c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:00.921391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb79d8-5500-4f98-8320-c4da0f685c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.921672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb79d8-5500-4f98-8320-c4da0f685c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.921957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb79d8-5500-4f98-8320-c4da0f685c86 admin admin] PATH_INFO: `/v3/projects/c8e7620229c349319071db0ff7426b62/users/c43755ee4f0f49d8bf1d15921c2aed13/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.945166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb79d8-5500-4f98-8320-c4da0f685c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e7620229c349319071db0ff7426b62, user_id=c43755ee4f0f49d8bf1d15921c2aed13, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:00.949271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b01ae4d3-0d37-450f-aaa3-fea297b8609e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:00.949414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb79d8-5500-4f98-8320-c4da0f685c86 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:00.950947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b01ae4d3-0d37-450f-aaa3-fea297b8609e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:00.972843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2357/4760] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:15:00 2022] PUT /identity/v3/projects/c8e7620229c349319071db0ff7426b62/users/c43755ee4f0f49d8bf1d15921c2aed13/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:15:00.973818 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2404/4761] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:00.981045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ae4d3-0d37-450f-aaa3-fea297b8609e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:00.981298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50d8e-4410-4a6a-b15a-5126bd96c209 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:00.981588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50d8e-4410-4a6a-b15a-5126bd96c209 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:00.981865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50d8e-4410-4a6a-b15a-5126bd96c209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:00.990699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:00.995016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ae4d3-0d37-450f-aaa3-fea297b8609e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.015912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-85b50d8e-4410-4a6a-b15a-5126bd96c209 None None] MFA Rules not processed for user `c43755ee4f0f49d8bf1d15921c2aed13`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:01.017930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-85b50d8e-4410-4a6a-b15a-5126bd96c209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:01.052441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:01.066307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.067092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.067511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.067955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.069043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:15:01.069909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.071724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.086624 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2405/4762] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:15:01.091192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:01.102693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac45e9-5bb1-420d-9c75-14951325ae49 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.103035 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2358/4763] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:01.113787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624186f-4440-472b-bafc-457a484696b3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.114058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624186f-4440-472b-bafc-457a484696b3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.114277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624186f-4440-472b-bafc-457a484696b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.116936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.117567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.117915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.118188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.119083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.121906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.140636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4624186f-4440-472b-bafc-457a484696b3 None None] MFA Rules not processed for user `c43755ee4f0f49d8bf1d15921c2aed13`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:01.142060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4624186f-4440-472b-bafc-457a484696b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:01.156435 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2359/4764] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:15:01.158662 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2406/4765] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:15:01 2022] POST /identity/v3/projects => generated 403 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:15:01.162843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624186f-4440-472b-bafc-457a484696b3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:01.168276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:01.172097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:01.175624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624186f-4440-472b-bafc-457a484696b3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.253942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.254820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.255281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.255945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.257371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.260055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.283821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a587-6c87-4b58-abe3-69046af2b832 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.290781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae21551-d3d2-40e9-bd8d-ccb5cb0fc489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.291091 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2360/4766] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:15:01 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:15:01.291540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae21551-d3d2-40e9-bd8d-ccb5cb0fc489 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.291821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae21551-d3d2-40e9-bd8d-ccb5cb0fc489 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.292091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae21551-d3d2-40e9-bd8d-ccb5cb0fc489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.295351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:01.310318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b885128-8c34-4f07-9853-e9c6976d117c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.315550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.316471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.316959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.317233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.317892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.319767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.339998 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2361/4767] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:15:01.340975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:01.356282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40544b5-1ff5-4eac-b1c4-7e6f74768be7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.363369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.364077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.364981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.365272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.365994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.369253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.369390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae21551-d3d2-40e9-bd8d-ccb5cb0fc489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.371254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae21551-d3d2-40e9-bd8d-ccb5cb0fc489 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.386463 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2362/4768] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:15:01.387738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:01.396929 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2407/4769] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:15:01.400862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f22a95-4305-4ac2-95d9-1441cd818e2a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.411742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.412823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.413307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.413801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] PATH_INFO: `/v3/projects/4a125c6e9abd46b68b5b5c9cea53ec96/users/02c331abed624c5f88835fa9fdfcfa5c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.432042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a125c6e9abd46b68b5b5c9cea53ec96, user_id=02c331abed624c5f88835fa9fdfcfa5c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.435551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.458286 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2363/4770] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:15:01 2022] PUT /identity/v3/projects/4a125c6e9abd46b68b5b5c9cea53ec96/users/02c331abed624c5f88835fa9fdfcfa5c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:15:01.464005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.464437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.464803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.498721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:01.500814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:01.582697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:01.605310 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2408/4771] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:15:01.619124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:01.621643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:01.708265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:01.775469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb652f21-2ab1-43aa-9987-9d0842a0d80d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:01.786993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:01.788105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:01.788424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:01.788730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:01.813618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:01.815374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:01.830913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2364/4772] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:02.206824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:02.209553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:02.372346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ab601-897c-452c-ba8b-980f19b91b2f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:02.379332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:02.380765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:02.397575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ba2b3-a0ac-46af-8350-8d94548eb271 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:02.398241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ba2b3-a0ac-46af-8350-8d94548eb271 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:02.398518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ba2b3-a0ac-46af-8350-8d94548eb271 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:02.398962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ba2b3-a0ac-46af-8350-8d94548eb271 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:02.406848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6f2ba2b3-a0ac-46af-8350-8d94548eb271 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:02.417966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eeda34-7914-4754-9fa3-78605b0e27ac service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:02.441401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ba2b3-a0ac-46af-8350-8d94548eb271 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:02.442999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ba2b3-a0ac-46af-8350-8d94548eb271 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:02.445633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:02.446676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:02.446966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:02.447248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:02.452704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:02.470706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2409/4773] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:02.502011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:02.506821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:02.544557 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2365/4774] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:15:02.619557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049316f-e865-48f6-9d70-3aff29a166ae None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:02.620075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049316f-e865-48f6-9d70-3aff29a166ae None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:02.620545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049316f-e865-48f6-9d70-3aff29a166ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:02.671348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6049316f-e865-48f6-9d70-3aff29a166ae None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:02.674847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6049316f-e865-48f6-9d70-3aff29a166ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:02.773605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2410/4775] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:04.716794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:04.737546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c75d94-6cd3-467e-98d2-6ff4fefabd3f service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:04.763710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd4c0a-359d-4e86-a01b-a331ade4f428 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:04.764407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4c0a-359d-4e86-a01b-a331ade4f428 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:04.764746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4c0a-359d-4e86-a01b-a331ade4f428 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:04.765891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4c0a-359d-4e86-a01b-a331ade4f428 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:04.767523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9dcd4c0a-359d-4e86-a01b-a331ade4f428 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:04.799862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd4c0a-359d-4e86-a01b-a331ade4f428 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:04.802487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd4c0a-359d-4e86-a01b-a331ade4f428 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:04.816963 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2366/4776] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:12.545795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9f52-7b38-4767-80f8-72c9b0587df6 None None] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:12.546177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9f52-7b38-4767-80f8-72c9b0587df6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:12.546177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9f52-7b38-4767-80f8-72c9b0587df6 None None] PATH_INFO: `/` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:12.548063 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2411/4777] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:15:12.554091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e766fc-5659-40ab-be65-322e0941943f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:12.554622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e766fc-5659-40ab-be65-322e0941943f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:12.554622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e766fc-5659-40ab-be65-322e0941943f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:12.585288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-85e766fc-5659-40ab-be65-322e0941943f None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:12.587868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-85e766fc-5659-40ab-be65-322e0941943f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:12.602377 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2367/4778] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:15:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:12.629334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d9f52-7b38-4767-80f8-72c9b0587df6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:12.648054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d9f52-7b38-4767-80f8-72c9b0587df6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:12.667564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:12.671106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:12.671614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:12.672090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:12.674713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5422-31c5-4ff5-8692-378d19af286e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:12.680452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5422-31c5-4ff5-8692-378d19af286e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:12.681183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5422-31c5-4ff5-8692-378d19af286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:12.696742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:12.698964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:12.710690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2412/4779] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:12.741073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b23a5422-31c5-4ff5-8692-378d19af286e None None] MFA Rules not processed for user `02c331abed624c5f88835fa9fdfcfa5c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:12.742695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b23a5422-31c5-4ff5-8692-378d19af286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:12.875928 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2368/4780] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:12 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:15:12.881503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:12.907748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c72e-2d3d-42bb-a5c7-68e7b527d151 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:12.921544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:12.925719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:12.928286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:12.928798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:12.930168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:12.947947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:13.000440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a5422-31c5-4ff5-8692-378d19af286e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:13.015490 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2413/4781] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:15:12 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:15:13.019981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:13.021220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a5422-31c5-4ff5-8692-378d19af286e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:13.126690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:13.131554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:13.132571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:13.132948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:13.133479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] PATH_INFO: `/v3/users/c43755ee4f0f49d8bf1d15921c2aed13` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:13.135009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c43755ee4f0f49d8bf1d15921c2aed13)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:13.137247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:13.146729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fbc3d-52ac-4951-a5b0-5b413331d586 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:13.154073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:13.155480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:13.156225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:13.157637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:13.161548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:13.165134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:13.210935 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2414/4782] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:15:13 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:15:13.215144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:13.218323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:13.230722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdac3f-dc5e-4251-bdab-6310aaa25918 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:13.238552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:13.239179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:13.239642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:13.240015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:13.240974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:13.243039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:13.253587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c43755ee4f0f49d8bf1d15921c2aed13'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:13.254432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c43755ee4f0f49d8bf1d15921c2aed13'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:13.260461 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2415/4783] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:15:13.266562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:13.270393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c43755ee4f0f49d8bf1d15921c2aed13'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:13.270555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:13.284966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c43755ee4f0f49d8bf1d15921c2aed13'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:13.291458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c456-9100-4fb4-9a10-6883b80878c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:13.299232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:13.300113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:13.300863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:13.300863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:13.301603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:13.305152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:13.314605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3cdf2fcf-f73d-42fb-86b4-53b6e4cd8371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c43755ee4f0f49d8bf1d15921c2aed13'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:13.323688 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2416/4784] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:15:13.325659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:13.332727 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2369/4785] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:15:12 2022] DELETE /identity/v3/users/c43755ee4f0f49d8bf1d15921c2aed13 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:15:13.338712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:13.340942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:13.341213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:13.342635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38d18b-9577-4d41-9a09-b2b9eb76f825 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:13.375383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:13.377864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:13.476339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:13.484101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:13.484397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:13.484706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] PATH_INFO: `/v3/projects/3e98ba3d755a4493aa1f6f88259b47f1/users/3a42c94a1e6b45b8af1e99ccdc73edf4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:13.498093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2370/4786] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:13.513682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:13.516394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:13.530015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e98ba3d755a4493aa1f6f88259b47f1, user_id=3a42c94a1e6b45b8af1e99ccdc73edf4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:13.532992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:13.570348 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2417/4787] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:15:13 2022] PUT /identity/v3/projects/3e98ba3d755a4493aa1f6f88259b47f1/users/3a42c94a1e6b45b8af1e99ccdc73edf4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:15:13.638676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730ae8f-adde-4100-9426-15dc6479479f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:13.712073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:13.716999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:13.717378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:13.718563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:13.795159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:14.004418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:14.007011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:14.037989 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2371/4788] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:15:15.108914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:15.111480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:15.267020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf5935-6faf-4a3f-8dc9-42c322917a3b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:15.274201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7433b-a884-4c80-8c69-2522496336a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:15.274937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7433b-a884-4c80-8c69-2522496336a6 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:15.275261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7433b-a884-4c80-8c69-2522496336a6 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:15.275539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7433b-a884-4c80-8c69-2522496336a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:15.277770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-73a7433b-a884-4c80-8c69-2522496336a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:15.317790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:15.354933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:15.356474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:15.362243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7433b-a884-4c80-8c69-2522496336a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:15.364074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7433b-a884-4c80-8c69-2522496336a6 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:15.377752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560efa4-3060-4220-a363-ba4c7b870a03 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:15.390404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:15.391568 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2418/4789] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:15.404091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:15.404867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:15.405939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] PATH_INFO: `/v3/projects/c8e7620229c349319071db0ff7426b62` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:15.411381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e7620229c349319071db0ff7426b62)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:15.413918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:15.577509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e7620229c349319071db0ff7426b62', 'request_id': 'req-33abdff9-3d58-4adc-8d27-c43bb80b0822'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:15.596151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e7620229c349319071db0ff7426b62', 'request_id': 'req-33abdff9-3d58-4adc-8d27-c43bb80b0822'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:15.629217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e7620229c349319071db0ff7426b62', 'request_id': 'req-33abdff9-3d58-4adc-8d27-c43bb80b0822'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:15.661332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] The token cache is being invalidate because project c8e7620229c349319071db0ff7426b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:15:15.661807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:15.663093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-33abdff9-3d58-4adc-8d27-c43bb80b0822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:15:15.665210 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2372/4790] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:15:15 2022] DELETE /identity/v3/projects/c8e7620229c349319071db0ff7426b62 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:15:15.672544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5553b-4bba-49c9-a593-842d76a961bb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:15.672972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5553b-4bba-49c9-a593-842d76a961bb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:15.673380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5553b-4bba-49c9-a593-842d76a961bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:15.707372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-34f5553b-4bba-49c9-a593-842d76a961bb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:15.709244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34f5553b-4bba-49c9-a593-842d76a961bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:15.832716 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2419/4791] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:15.836245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35835d-dc0a-4a5d-9399-26034d627935 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:15.836542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35835d-dc0a-4a5d-9399-26034d627935 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:15.836848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35835d-dc0a-4a5d-9399-26034d627935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:15.867402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2a35835d-dc0a-4a5d-9399-26034d627935 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:15.869083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a35835d-dc0a-4a5d-9399-26034d627935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:15.883277 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2373/4792] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:15.885092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5553b-4bba-49c9-a593-842d76a961bb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:15.900516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5553b-4bba-49c9-a593-842d76a961bb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:15.913981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:15.914774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:15.915038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:15.915310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:15.915985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:15:15.916283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:15.917949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:15.972715 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2420/4793] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:15:15.977527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35835d-dc0a-4a5d-9399-26034d627935 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:15.994223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35835d-dc0a-4a5d-9399-26034d627935 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:16.009582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:16.010515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:16.011059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:16.011594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:16.012460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:16.014903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:16.082135 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2374/4794] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:15:15 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:15:16.086767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:16.101498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8535f5-35b9-4f82-9b13-bbbd2ea6f3bb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:16.219948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:16.222417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:16.222930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:16.223612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:16.225830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:16.229744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:16.274974 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2421/4795] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:15:16 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:15:16.279226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:16.296215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300cc89-ab69-4f4a-88c4-80a96711d376 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:16.304645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:16.305475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:16.305812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:16.307954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:16.308395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:16.310113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:16.327638 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2375/4796] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:15:16.328417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:16.341110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69660074-8ec4-480a-827c-a7da0ec68723 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:16.375387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78189b-c9c0-4b6f-bf4e-96baea86f96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:16.376848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78189b-c9c0-4b6f-bf4e-96baea86f96f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:16.377184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78189b-c9c0-4b6f-bf4e-96baea86f96f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:16.377461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78189b-c9c0-4b6f-bf4e-96baea86f96f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:16.379330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78189b-c9c0-4b6f-bf4e-96baea86f96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:16.382040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78189b-c9c0-4b6f-bf4e-96baea86f96f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:16.397052 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2422/4797] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:15:16.402528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:16.413612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d50532-0bc4-4b13-9a12-74f5a7749edf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:16.418765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:16.419443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:16.420009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:16.420296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] PATH_INFO: `/v3/projects/4b17e56b43f04d14b1c5135d1566bef9/users/219bef5cfcfc496892611dce4bc1d89d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:16.454271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b17e56b43f04d14b1c5135d1566bef9, user_id=219bef5cfcfc496892611dce4bc1d89d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:16.466678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:16.496200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2376/4798] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:15:16 2022] PUT /identity/v3/projects/4b17e56b43f04d14b1c5135d1566bef9/users/219bef5cfcfc496892611dce4bc1d89d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:15:16.519059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a727-9a73-486f-8dcf-bd275ab8d8ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:16.519457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a727-9a73-486f-8dcf-bd275ab8d8ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:16.519614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a727-9a73-486f-8dcf-bd275ab8d8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:16.546379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-73e8a727-9a73-486f-8dcf-bd275ab8d8ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:16.548038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-73e8a727-9a73-486f-8dcf-bd275ab8d8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:16.577569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:16.672515 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2423/4799] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:16.688409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:16.692307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:16.755450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:16.815563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b83f3-527d-40bf-8683-9f4b56d02d85 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:16.819840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7d178-b134-40f5-b4b3-ca67ccb06d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:16.820477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7d178-b134-40f5-b4b3-ca67ccb06d66 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:16.831318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7d178-b134-40f5-b4b3-ca67ccb06d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:16.831822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7d178-b134-40f5-b4b3-ca67ccb06d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:16.846010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7d178-b134-40f5-b4b3-ca67ccb06d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:16.848514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7d178-b134-40f5-b4b3-ca67ccb06d66 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:16.863502 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2377/4800] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:20.246795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867d40-5862-4a77-b718-f0fcb3cf525f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:20.247638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867d40-5862-4a77-b718-f0fcb3cf525f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:20.248235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867d40-5862-4a77-b718-f0fcb3cf525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:20.289961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3a867d40-5862-4a77-b718-f0fcb3cf525f None None] MFA Rules not processed for user `dc369ca971424247b96050e09368a7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:20.296854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3a867d40-5862-4a77-b718-f0fcb3cf525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:20.410143 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2424/4801] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:26.299387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:26.300699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:26.301274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:26.365288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] MFA Rules not processed for user `3a42c94a1e6b45b8af1e99ccdc73edf4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:26.366707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:26.527378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98cfdd-86b6-41a0-be97-ca9d709b7424 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:26.527936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98cfdd-86b6-41a0-be97-ca9d709b7424 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:26.527936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98cfdd-86b6-41a0-be97-ca9d709b7424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:26.528108 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2378/4802] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:26.573212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ea98cfdd-86b6-41a0-be97-ca9d709b7424 None None] MFA Rules not processed for user `02c331abed624c5f88835fa9fdfcfa5c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:26.574778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea98cfdd-86b6-41a0-be97-ca9d709b7424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:26.719217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2425/4803] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:26.955859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:26.957634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:27.199211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8594c-b119-4d36-8f4a-bd92ff15aeb9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:27.207282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82503a-9b7c-4985-aa97-bf1b7dbe397e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:27.208067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82503a-9b7c-4985-aa97-bf1b7dbe397e service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:27.208349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82503a-9b7c-4985-aa97-bf1b7dbe397e service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:27.208618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82503a-9b7c-4985-aa97-bf1b7dbe397e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:27.217234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ba82503a-9b7c-4985-aa97-bf1b7dbe397e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:27.232339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82503a-9b7c-4985-aa97-bf1b7dbe397e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:27.234023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82503a-9b7c-4985-aa97-bf1b7dbe397e service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:27.261692 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2379/4804] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:26 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:34.602055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:34.602489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:34.603505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:34.668144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] MFA Rules not processed for user `219bef5cfcfc496892611dce4bc1d89d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:34.670128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:34.802306 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2426/4805] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:15:34.818818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f31e-5dc4-4494-b203-74282aaec5d1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:34.819427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f31e-5dc4-4494-b203-74282aaec5d1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:34.819427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f31e-5dc4-4494-b203-74282aaec5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:34.841480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d622f31e-5dc4-4494-b203-74282aaec5d1 None None] MFA Rules not processed for user `219bef5cfcfc496892611dce4bc1d89d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:34.843017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d622f31e-5dc4-4494-b203-74282aaec5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:34.854699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2380/4806] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:34.946247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:34.947859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:34.971652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe3b5-db7d-4f68-8019-aa3cb888c083 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:34.981519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7beebc-0e5b-4be4-a07d-0dead28bd79c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:34.982039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7beebc-0e5b-4be4-a07d-0dead28bd79c service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:34.982315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7beebc-0e5b-4be4-a07d-0dead28bd79c service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:34.982609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7beebc-0e5b-4be4-a07d-0dead28bd79c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:34.991351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:34.999209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7beebc-0e5b-4be4-a07d-0dead28bd79c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:35.001122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7beebc-0e5b-4be4-a07d-0dead28bd79c service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:35.013406 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2427/4807] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:35.514572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:35.514865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:35.516064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:35.542345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:35.544400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:35.601770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:35.632477 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2381/4808] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:45.007300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:45.623700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:50.966190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f9a53-8402-41a3-bccf-d9d4737c6811 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:50.967310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f9a53-8402-41a3-bccf-d9d4737c6811 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:50.968071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f9a53-8402-41a3-bccf-d9d4737c6811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:51.001336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-486f9a53-8402-41a3-bccf-d9d4737c6811 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:15:51.004426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-486f9a53-8402-41a3-bccf-d9d4737c6811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:51.100810 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2428/4809] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:15:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:15:51.165169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:51.167170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:51.273983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8632a-2685-45e8-9713-3e87f8abbf72 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:51.281663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:51.282381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:51.282571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:51.283013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:51.301249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:51.303867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:51.317530 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2382/4810] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:15:52.792111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f9a53-8402-41a3-bccf-d9d4737c6811 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:52.813159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f9a53-8402-41a3-bccf-d9d4737c6811 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:52.820430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246418dc-3e6d-443e-8318-504ee2e8f8cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:52.821362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246418dc-3e6d-443e-8318-504ee2e8f8cc service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:52.821814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246418dc-3e6d-443e-8318-504ee2e8f8cc service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:52.822272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246418dc-3e6d-443e-8318-504ee2e8f8cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:52.832766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246418dc-3e6d-443e-8318-504ee2e8f8cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:52.834752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246418dc-3e6d-443e-8318-504ee2e8f8cc service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:52.846712 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2429/4811] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:52 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:15:53.859787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:15:53.864501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:53.907101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea1af3-4753-4b1c-8dc5-ae9e711c5e51 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:15:53.914105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:15:53.915067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:15:53.915767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:15:53.916276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:15:53.918339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:15:53.940313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:15:53.941939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:15:53.962323 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2383/4812] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:02.841676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=35 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:03.078818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e1279-7a29-48db-ae15-1dce5dc01ba6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:03.079203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e1279-7a29-48db-ae15-1dce5dc01ba6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:03.079483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e1279-7a29-48db-ae15-1dce5dc01ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:03.140426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-998e1279-7a29-48db-ae15-1dce5dc01ba6 None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:03.145743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-998e1279-7a29-48db-ae15-1dce5dc01ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:03.168680 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2430/4813] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:03.949003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=42 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:13.121692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:13.829390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:13.830884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:13.873106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701f791-50ac-4a1e-a278-d3b99276bbfb service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:13.914395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:13.915148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:13.915731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:13.916171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] PATH_INFO: `/v3/users/04a5f2b1d0134e529f25c227954b5f20` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:13.918930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a5f2b1d0134e529f25c227954b5f20)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:13.923336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:14.067594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a5f2b1d0134e529f25c227954b5f20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.068492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a5f2b1d0134e529f25c227954b5f20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.081092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a5f2b1d0134e529f25c227954b5f20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.109147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a5f2b1d0134e529f25c227954b5f20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.130458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a5f2b1d0134e529f25c227954b5f20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.150676 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2384/4814] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:13 2022] DELETE /identity/v3/users/04a5f2b1d0134e529f25c227954b5f20 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:14.413761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e1279-7a29-48db-ae15-1dce5dc01ba6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:14.435754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e1279-7a29-48db-ae15-1dce5dc01ba6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:14.539346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:14.540634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:14.541247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:14.541796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] PATH_INFO: `/v3/projects/f86d6a96f893464f81868717f0531bd3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:14.543467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86d6a96f893464f81868717f0531bd3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:14.545743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:14.672407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86d6a96f893464f81868717f0531bd3', 'request_id': 'req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.685090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86d6a96f893464f81868717f0531bd3', 'request_id': 'req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.696390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86d6a96f893464f81868717f0531bd3', 'request_id': 'req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.712498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] The token cache is being invalidate because project f86d6a96f893464f81868717f0531bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:14.712858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.714331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.718102 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2431/4815] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:14 2022] DELETE /identity/v3/projects/f86d6a96f893464f81868717f0531bd3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:14.723012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:14.725356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:14.745541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ef58d-5bd0-4c6a-b4a1-f4912628175f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:14.847457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:14.848128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:14.848519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:14.848888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] PATH_INFO: `/v3/users/95be68bcca8a4fa3a2d0a0d28173aa85` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:14.850815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] RBAC: Authorizing `identity:delete_user(user_id=95be68bcca8a4fa3a2d0a0d28173aa85)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:14.853190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:14.934551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95be68bcca8a4fa3a2d0a0d28173aa85'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.935649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95be68bcca8a4fa3a2d0a0d28173aa85'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.947046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95be68bcca8a4fa3a2d0a0d28173aa85'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.958224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95be68bcca8a4fa3a2d0a0d28173aa85'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:14.969179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:14.984328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d9ce169-77df-47c0-a731-db58af7641df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95be68bcca8a4fa3a2d0a0d28173aa85'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:15.001194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2385/4816] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:14 2022] DELETE /identity/v3/users/95be68bcca8a4fa3a2d0a0d28173aa85 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:15.442073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:15.444349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:15.481016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ef3e2-25a8-4b7a-bd32-16a706970a35 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:15.624494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:15.625386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:15.625796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:15.626073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] PATH_INFO: `/v3/projects/5f0d32ee9bd342c0949d28a270e3fb7e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:15.627388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0d32ee9bd342c0949d28a270e3fb7e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:15.635729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:15.743887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:15.769314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0d32ee9bd342c0949d28a270e3fb7e', 'request_id': 'req-ac215189-e779-4dc3-bba0-688c5d485d93'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:15.788694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0d32ee9bd342c0949d28a270e3fb7e', 'request_id': 'req-ac215189-e779-4dc3-bba0-688c5d485d93'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:15.800119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0d32ee9bd342c0949d28a270e3fb7e', 'request_id': 'req-ac215189-e779-4dc3-bba0-688c5d485d93'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:15.868142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] The token cache is being invalidate because project 5f0d32ee9bd342c0949d28a270e3fb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:15.868455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:15.869718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac215189-e779-4dc3-bba0-688c5d485d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:15.876191 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2432/4817] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:15 2022] DELETE /identity/v3/projects/5f0d32ee9bd342c0949d28a270e3fb7e => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:15.876831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c5c9c-84a0-4ee0-83a5-4717449c1abc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:15.877624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c5c9c-84a0-4ee0-83a5-4717449c1abc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:15.878053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c5c9c-84a0-4ee0-83a5-4717449c1abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:15.913680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-402c5c9c-84a0-4ee0-83a5-4717449c1abc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:15.915253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-402c5c9c-84a0-4ee0-83a5-4717449c1abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:16.079265 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2386/4818] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:16.080870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ef81-d054-4498-85de-9ab9dc865773 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.081713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ef81-d054-4498-85de-9ab9dc865773 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.082001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ef81-d054-4498-85de-9ab9dc865773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.134933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1ba4ef81-d054-4498-85de-9ab9dc865773 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:16.195810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ba4ef81-d054-4498-85de-9ab9dc865773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:16.264887 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2433/4819] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:16.269448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c5c9c-84a0-4ee0-83a5-4717449c1abc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:16.287172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c5c9c-84a0-4ee0-83a5-4717449c1abc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:16.306873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:16.307692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.308043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.308406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.309166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:16.309479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:16.311291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:16.345830 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2387/4820] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:16.353407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ef81-d054-4498-85de-9ab9dc865773 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:16.364333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ef81-d054-4498-85de-9ab9dc865773 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:16.382323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:16.383143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.383571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.384000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.384841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:16.387303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:16.458381 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2434/4821] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:16 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:16.462668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:16.476243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e16be-daa3-4072-994a-0c092e9b0195 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:16.625029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:16.625579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.625845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.626125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.627026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:16.629787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:16.668364 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2388/4822] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:16 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:16.669638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:16.692183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c8d70-bb8e-4f0f-9a52-f3a3398c204e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:16.705732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:16.706564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.706929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.707490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.708162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:16.710588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:16.728720 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2435/4823] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:16.731764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:16.745214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226cd20-694d-4dc2-bf5a-8572e1571d23 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:16.760724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc25a7-bdfc-4fd6-a224-7d0f7452d433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:16.761446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc25a7-bdfc-4fd6-a224-7d0f7452d433 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.761789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc25a7-bdfc-4fd6-a224-7d0f7452d433 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.762124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc25a7-bdfc-4fd6-a224-7d0f7452d433 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.763139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc25a7-bdfc-4fd6-a224-7d0f7452d433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:16.764838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc25a7-bdfc-4fd6-a224-7d0f7452d433 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:16.785058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:16.796453 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2389/4824] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:16.796725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec9785-22df-48a5-9ae7-d0c70983dcf7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:16.801989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:16.802821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.803191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.803574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] PATH_INFO: `/v3/projects/8070262519034021a2b6de1a1e1ec88a/users/dbf876e1c23642578363f9c5995f3d86/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.827647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8070262519034021a2b6de1a1e1ec88a, user_id=dbf876e1c23642578363f9c5995f3d86, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:16.830574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:16.873736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:16.874247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:16.874936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:16.878702 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2436/4825] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:16 2022] PUT /identity/v3/projects/8070262519034021a2b6de1a1e1ec88a/users/dbf876e1c23642578363f9c5995f3d86/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:16.902454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:16.903913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:16.974756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:17.005431 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2390/4826] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:17.030375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:17.032365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:17.186673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09a6e5-bc6a-4913-be52-e38bb0eb21e9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:17.198961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:17.201781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:17.202109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:17.202388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:17.233249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:17.236510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:17.264002 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2437/4827] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:21.934646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:21.939567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:22.060050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:22.062404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:22.096271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1704b2-5905-40c9-bbb2-3a557192b978 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:22.201996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:22.210160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:22.213946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:22.217163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] PATH_INFO: `/v3/users/02c331abed624c5f88835fa9fdfcfa5c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:22.224303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c331abed624c5f88835fa9fdfcfa5c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:22.235567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:22.237853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dccabf-a258-4193-b4bf-6b3771cd712a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:22.259041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:22.263969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:22.265363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:22.266041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:22.272090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:22.327583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:22.342951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:22.380194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c331abed624c5f88835fa9fdfcfa5c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:22.381465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c331abed624c5f88835fa9fdfcfa5c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:22.401305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c331abed624c5f88835fa9fdfcfa5c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:22.404927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:22.409075 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2391/4828] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:22.459815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c331abed624c5f88835fa9fdfcfa5c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:22.461273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c331abed624c5f88835fa9fdfcfa5c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:22.533508 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2438/4829] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:22 2022] DELETE /identity/v3/users/02c331abed624c5f88835fa9fdfcfa5c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:22.941728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:22.958056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ed410-19ec-4f13-9c57-8fd48f4bd215 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:23.088387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:23.089096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:23.089851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:23.089851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] PATH_INFO: `/v3/projects/4a125c6e9abd46b68b5b5c9cea53ec96` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:23.092175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a125c6e9abd46b68b5b5c9cea53ec96)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:23.094162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:23.241986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a125c6e9abd46b68b5b5c9cea53ec96', 'request_id': 'req-e05465fb-121c-483f-a763-c6729b7cd6c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.268797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a125c6e9abd46b68b5b5c9cea53ec96', 'request_id': 'req-e05465fb-121c-483f-a763-c6729b7cd6c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.298491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a125c6e9abd46b68b5b5c9cea53ec96', 'request_id': 'req-e05465fb-121c-483f-a763-c6729b7cd6c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.321774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] The token cache is being invalidate because project 4a125c6e9abd46b68b5b5c9cea53ec96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:23.321997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.322937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.332593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:23.333299 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2392/4830] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:22 2022] DELETE /identity/v3/projects/4a125c6e9abd46b68b5b5c9cea53ec96 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:23.335145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:23.358347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a7509-c8f3-403c-bba6-9b587a48f2e3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:23.491155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:23.491821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:23.492274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:23.492752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] PATH_INFO: `/v3/users/3a42c94a1e6b45b8af1e99ccdc73edf4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:23.494046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a42c94a1e6b45b8af1e99ccdc73edf4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:23.496267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:23.629224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a42c94a1e6b45b8af1e99ccdc73edf4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.631846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a42c94a1e6b45b8af1e99ccdc73edf4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.670017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a42c94a1e6b45b8af1e99ccdc73edf4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.683226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a42c94a1e6b45b8af1e99ccdc73edf4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.684228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0493e638-57c8-4f13-a17b-3b0a128369de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a42c94a1e6b45b8af1e99ccdc73edf4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:23.750355 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2439/4831] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:23 2022] DELETE /identity/v3/users/3a42c94a1e6b45b8af1e99ccdc73edf4 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:24.662304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:24.665115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:24.697494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05465fb-121c-483f-a763-c6729b7cd6c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:24.773815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:24.947776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:24.948454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:24.948828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:24.949105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] PATH_INFO: `/v3/projects/3e98ba3d755a4493aa1f6f88259b47f1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:24.952646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e98ba3d755a4493aa1f6f88259b47f1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:24.954652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:25.134299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e98ba3d755a4493aa1f6f88259b47f1', 'request_id': 'req-edb155ac-465c-4f97-a210-5cad352c003d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:25.157829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e98ba3d755a4493aa1f6f88259b47f1', 'request_id': 'req-edb155ac-465c-4f97-a210-5cad352c003d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:25.203914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e98ba3d755a4493aa1f6f88259b47f1', 'request_id': 'req-edb155ac-465c-4f97-a210-5cad352c003d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:25.228831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] The token cache is being invalidate because project 3e98ba3d755a4493aa1f6f88259b47f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:25.228831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:25.229538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-edb155ac-465c-4f97-a210-5cad352c003d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:25.241141 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2393/4832] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:24 2022] DELETE /identity/v3/projects/3e98ba3d755a4493aa1f6f88259b47f1 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:25.247874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096180a5-7c59-4335-a3f0-28b59f56b29a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:25.248449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096180a5-7c59-4335-a3f0-28b59f56b29a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:25.248449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096180a5-7c59-4335-a3f0-28b59f56b29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:25.291711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-096180a5-7c59-4335-a3f0-28b59f56b29a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:25.295273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-096180a5-7c59-4335-a3f0-28b59f56b29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:25.421309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:25.456878 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2440/4833] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:25.464889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5b580-954b-4106-a89e-17bb09c26a99 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:25.465249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5b580-954b-4106-a89e-17bb09c26a99 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:25.465451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5b580-954b-4106-a89e-17bb09c26a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:25.511841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-24a5b580-954b-4106-a89e-17bb09c26a99 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:25.513174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-24a5b580-954b-4106-a89e-17bb09c26a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:25.542705 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2394/4834] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:25.544818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096180a5-7c59-4335-a3f0-28b59f56b29a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:25.559144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096180a5-7c59-4335-a3f0-28b59f56b29a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:25.578355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:25.579002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:25.579275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:25.579535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:25.580197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:25.580501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:25.582331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:25.630082 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2441/4835] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:25.632388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5b580-954b-4106-a89e-17bb09c26a99 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:25.651560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5b580-954b-4106-a89e-17bb09c26a99 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:25.660689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba701309-0917-4123-a80b-619283817107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:25.661798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba701309-0917-4123-a80b-619283817107 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:25.662485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba701309-0917-4123-a80b-619283817107 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:25.663843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba701309-0917-4123-a80b-619283817107 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:25.665490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba701309-0917-4123-a80b-619283817107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:25.667795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba701309-0917-4123-a80b-619283817107 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:25.781265 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2395/4836] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:25 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:25.786110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:25.797605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5988a90-c43d-4bf9-8225-d5d0d519a6ac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:25.884339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:25.885362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:25.886449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:25.887029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:25.887842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:25.893836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:25.929029 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2442/4837] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:25 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:25.933623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba701309-0917-4123-a80b-619283817107 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:25.950919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba701309-0917-4123-a80b-619283817107 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:25.958755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:25.959874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:25.960446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:25.960947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:25.961759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:25.964488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:25.982364 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2396/4838] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:25.986459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:26.007138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba8a0-cf21-4bfb-a0ff-3761c13fa5ac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:26.013892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6df42d-89db-40ee-a475-1635469d78b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:26.015701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df42d-89db-40ee-a475-1635469d78b9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:26.016205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df42d-89db-40ee-a475-1635469d78b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:26.016747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6df42d-89db-40ee-a475-1635469d78b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:26.017565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6df42d-89db-40ee-a475-1635469d78b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:26.020502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6df42d-89db-40ee-a475-1635469d78b9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:26.048459 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2443/4839] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:26.050891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:26.074445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751b6e1-e0d0-4825-be5b-c2a3e486380f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:26.086510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:26.088639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:26.089514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:26.089802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] PATH_INFO: `/v3/projects/e350c6d323bb4635a612e94d21f99bf1/users/73586af7bfa14c578de7bc8313a816c4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:26.136582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e350c6d323bb4635a612e94d21f99bf1, user_id=73586af7bfa14c578de7bc8313a816c4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:26.142659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:26.186852 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2397/4840] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:26 2022] PUT /identity/v3/projects/e350c6d323bb4635a612e94d21f99bf1/users/73586af7bfa14c578de7bc8313a816c4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:26.199223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398f195-62fa-4115-99cc-c281589ef1a9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:26.200127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398f195-62fa-4115-99cc-c281589ef1a9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:26.200624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398f195-62fa-4115-99cc-c281589ef1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:26.269046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2398f195-62fa-4115-99cc-c281589ef1a9 None None] MFA Rules not processed for user `73586af7bfa14c578de7bc8313a816c4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:26.272744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2398f195-62fa-4115-99cc-c281589ef1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:26.488682 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2444/4841] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:26.493608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:26.614959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:26.652299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a2f30-9e7f-45c6-810e-b129d87c8614 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:26.664073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:26.664976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:26.665408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:26.665776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:26.668181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:26.674714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:26.742623 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2398/4842] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:26 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:26.747042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398f195-62fa-4115-99cc-c281589ef1a9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:26.818268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398f195-62fa-4115-99cc-c281589ef1a9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:27.001662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:27.039084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:27.042113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:27.042688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:27.044224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:27.045303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:27.047589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:27.124700 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2445/4843] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:26 2022] POST /identity/v3/users => generated 295 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:27.129259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:27.155660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51abfe7-271d-46ed-963c-3be1cf08519b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:27.172894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:27.173694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:27.174111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:27.174455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:27.175420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:27.179798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:27.213114 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2399/4844] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:27.218199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:27.238322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0bd0f-416c-45ff-8b63-e75089a4a373 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:27.254141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:27.255253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:27.255732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:27.256425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] PATH_INFO: `/v3/projects/3992721a498e4f488e66475637f66b72/users/d1b23217758a4512a5c5607df183ca83/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:27.277780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:27.280496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:27.290199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3992721a498e4f488e66475637f66b72, user_id=d1b23217758a4512a5c5607df183ca83, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:27.307593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b14bef-c7c0-462e-91fe-3691a0fdcdab admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:27.309221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:27.318058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:27.319996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:27.321573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:27.323073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] PATH_INFO: `/v3/users/219bef5cfcfc496892611dce4bc1d89d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:27.326285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219bef5cfcfc496892611dce4bc1d89d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:27.331146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:27.360342 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2446/4845] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:27 2022] PUT /identity/v3/projects/3992721a498e4f488e66475637f66b72/users/d1b23217758a4512a5c5607df183ca83/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:27.363862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:27.459809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219bef5cfcfc496892611dce4bc1d89d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:27.461465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219bef5cfcfc496892611dce4bc1d89d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:27.479428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219bef5cfcfc496892611dce4bc1d89d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:27.499737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219bef5cfcfc496892611dce4bc1d89d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:27.522092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219bef5cfcfc496892611dce4bc1d89d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:27.550196 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2400/4846] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:27 2022] DELETE /identity/v3/users/219bef5cfcfc496892611dce4bc1d89d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:27.555319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210228e-b820-4460-b12d-1fcc648c95a2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:27.765690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab36bf3-efec-43d4-bd73-bc12d291a50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:27.767749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36bf3-efec-43d4-bd73-bc12d291a50b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:27.768381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36bf3-efec-43d4-bd73-bc12d291a50b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:27.769649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36bf3-efec-43d4-bd73-bc12d291a50b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:27.772725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab36bf3-efec-43d4-bd73-bc12d291a50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:27.775627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab36bf3-efec-43d4-bd73-bc12d291a50b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:27.807384 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2447/4847] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:27.810073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:27.811645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:27.839263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a2bc7-a49e-476a-838a-aa8eaa8ddaa5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:27.855982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:27.856914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:27.857339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:27.857755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] PATH_INFO: `/v3/projects/3992721a498e4f488e66475637f66b72/users/d1b23217758a4512a5c5607df183ca83/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:27.861242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3992721a498e4f488e66475637f66b72, user_id=d1b23217758a4512a5c5607df183ca83, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:27.868687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:27.898192 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2401/4848] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:27 2022] PUT /identity/v3/projects/3992721a498e4f488e66475637f66b72/users/d1b23217758a4512a5c5607df183ca83/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:27.910740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c81f-8ff5-44c6-8b6f-13844f6615ec None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:27.911196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c81f-8ff5-44c6-8b6f-13844f6615ec None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:27.911508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c81f-8ff5-44c6-8b6f-13844f6615ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:27.973467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4c83c81f-8ff5-44c6-8b6f-13844f6615ec None None] MFA Rules not processed for user `d1b23217758a4512a5c5607df183ca83`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:27.974833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c83c81f-8ff5-44c6-8b6f-13844f6615ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:28.095945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:28.153035 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2448/4849] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:28.156616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.176649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272abe1-a704-41ae-b3a9-92ad65bbf58c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.189450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:28.190933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:28.191494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:28.192229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:28.193267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:28.194806 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:28.194806 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:28.194806 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:28.196959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:28.247012 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2402/4850] 10.222.0.26 () {60 vars in 1185 bytes} [Tue Sep 27 10:16:28 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:28.248784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83c81f-8ff5-44c6-8b6f-13844f6615ec None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.264060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83c81f-8ff5-44c6-8b6f-13844f6615ec None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.281410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:28.284979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:28.285327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:28.294632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f0a04601c29940008c50b8a10bf43af1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:28.296723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=f0a04601c29940008c50b8a10bf43af1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:28.300068 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:28.300068 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:16:28.300068 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:16:28.303734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:28.329366 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2449/4851] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:16:28 2022] GET /identity/v3/OS-OAUTH1/consumers/f0a04601c29940008c50b8a10bf43af1 => generated 221 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:28.331230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.348031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09fe7c-c82b-46c2-8702-b8b2e6c38673 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.360404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:28.363600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:28.364132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:28.364414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f0a04601c29940008c50b8a10bf43af1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:28.366360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=f0a04601c29940008c50b8a10bf43af1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:28.367834 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:28.367834 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:28.367834 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:28.374604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:28.374888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invalidating token cache because consumer f0a04601c29940008c50b8a10bf43af1 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:28.375228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.379595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.445201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.445372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'f0a04601c29940008c50b8a10bf43af1', 'request_id': 'req-dc04f80e-0729-4280-98a9-37d90f74a306'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.447617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:28.470024 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2403/4852] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:16:28 2022] DELETE /identity/v3/OS-OAUTH1/consumers/f0a04601c29940008c50b8a10bf43af1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:28.476017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.477857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:28.503861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04f80e-0729-4280-98a9-37d90f74a306 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.515048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:28.515734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:28.515979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:28.516278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:28.520581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:28.523746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:28.561501 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2404/4853] 10.222.0.26 () {60 vars in 1185 bytes} [Tue Sep 27 10:16:28 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:28.565595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.607486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38048e82-05b2-494e-8040-82eb185643b3 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.616611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:28.618063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:28.618440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:28.618851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:28.619596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=f7d61cb2fac8420f9f0ed21d3d38a7ca)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:28.620745 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:28.620745 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:28.620745 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:28.622068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:28.632036 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2405/4854] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:16:28 2022] GET /identity/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca => generated 221 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:28.639105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.667140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62c26e-9088-4d22-8ac0-a555d9ee3837 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.677138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d9979-d66a-4678-8fb1-390bd0295f14 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.680134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:28.682189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:28.683916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:28.684340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:28.685340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=f7d61cb2fac8420f9f0ed21d3d38a7ca)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:28.690902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:28.691339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:28.693532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:28.694292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:28.694431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invalidating token cache because consumer f7d61cb2fac8420f9f0ed21d3d38a7ca has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:28.694991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] PATH_INFO: `/v3/projects/4b17e56b43f04d14b1c5135d1566bef9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:28.696441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.700791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b17e56b43f04d14b1c5135d1566bef9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:28.701344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.709454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:28.722757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:INSERT=6,keystone:DELETE=13 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:28.743412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'f7d61cb2fac8420f9f0ed21d3d38a7ca', 'request_id': 'req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.868624 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2406/4855] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:16:28 2022] DELETE /identity/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:28.892195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:28.894125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:28.912813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe534a51-4a86-4b96-bb30-ddb1bb1c93e2 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:28.943616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b17e56b43f04d14b1c5135d1566bef9', 'request_id': 'req-ffb2099d-6cdb-4f93-bf08-58fe943834fc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.965947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b17e56b43f04d14b1c5135d1566bef9', 'request_id': 'req-ffb2099d-6cdb-4f93-bf08-58fe943834fc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:28.977661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b17e56b43f04d14b1c5135d1566bef9', 'request_id': 'req-ffb2099d-6cdb-4f93-bf08-58fe943834fc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.001281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] The token cache is being invalidate because project 4b17e56b43f04d14b1c5135d1566bef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:29.001457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.002298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffb2099d-6cdb-4f93-bf08-58fe943834fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.009099 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2450/4856] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:28 2022] DELETE /identity/v3/projects/4b17e56b43f04d14b1c5135d1566bef9 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:16:29.011204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f843-0202-49ca-934e-378a6ec75198 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.011204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f843-0202-49ca-934e-378a6ec75198 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.011581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f843-0202-49ca-934e-378a6ec75198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.044291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e121f843-0202-49ca-934e-378a6ec75198 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:29.046045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e121f843-0202-49ca-934e-378a6ec75198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:29.048289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.049155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.049540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.049932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.050745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=f7d61cb2fac8420f9f0ed21d3d38a7ca)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.053161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Sep 27 10:16:29.089086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 27 10:16:29.090460 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:16:29.094092 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2407/4857] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:16:28 2022] GET /identity/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca => generated 74 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 10:16:29.095039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.096931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:29.133947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2b04e-6471-4c96-b64e-fdb8fa564065 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.150329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.150918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.151227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.151506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.152107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=f7d61cb2fac8420f9f0ed21d3d38a7ca)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.153795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.154140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invalidating token cache because consumer f7d61cb2fac8420f9f0ed21d3d38a7ca has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:29.154353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.169304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.214511 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2451/4858] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Sep 27 10:16:29.221187 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2408/4859] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:16:29 2022] DELETE /identity/v3/OS-OAUTH1/consumers/f7d61cb2fac8420f9f0ed21d3d38a7ca => generated 74 bytes in 126 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 10:16:29.222107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.223461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:29.231611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add4838-bcf8-44d9-a070-9afc5f4cb6e4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.232773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add4838-bcf8-44d9-a070-9afc5f4cb6e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.233490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add4838-bcf8-44d9-a070-9afc5f4cb6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.251179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:29.272231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6cf8d-c232-45be-af44-527241d69847 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.283279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4add4838-bcf8-44d9-a070-9afc5f4cb6e4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:29.284903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4add4838-bcf8-44d9-a070-9afc5f4cb6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:29.297426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.298977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.298977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.298977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.299292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.301627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.328883 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2452/4860] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:29.338482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add4838-bcf8-44d9-a070-9afc5f4cb6e4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.345586 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2409/4861] 10.222.0.26 () {60 vars in 1185 bytes} [Tue Sep 27 10:16:29 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:29.360224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.374187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add4838-bcf8-44d9-a070-9afc5f4cb6e4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.383327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acf4fa-2512-4825-853d-8a362ffd4243 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.394854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.396289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.399547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.400044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.408931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:29.409679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.412171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.418756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.419407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.419693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.419966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.420626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.422323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.451212 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2453/4862] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:29.453222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.481254 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2410/4863] 10.222.0.26 () {60 vars in 1185 bytes} [Tue Sep 27 10:16:29 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:29.485513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.502014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2e39-3a1a-4294-b139-b18e9299a648 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.515434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.517251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.517637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.518036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.519215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.519501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d638d5f-d3b7-4f8d-922c-ad0049011e25 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.526500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.527418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.527870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.528315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.529170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.529944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.531396 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:29.531396 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:29.531396 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:29.532943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.551157 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2411/4864] 10.222.0.26 () {58 vars in 1164 bytes} [Tue Sep 27 10:16:29 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 535 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:29.555626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.583718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5819b-5a05-4293-a634-a80fe5c5af4a tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.594853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.595839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.596429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.596890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/534d443bdde342019d6679aeff669941` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.597759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=534d443bdde342019d6679aeff669941)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.599909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.600478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invalidating token cache because consumer 534d443bdde342019d6679aeff669941 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:29.600988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.605709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.632707 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2454/4865] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:29 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:29.634756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.638272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:29.651223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '534d443bdde342019d6679aeff669941', 'request_id': 'req-184c7e36-0413-46e3-a8a7-0fa5d280c14e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.678910 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2412/4866] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:16:29 2022] DELETE /identity/v3/OS-OAUTH1/consumers/534d443bdde342019d6679aeff669941 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:29.683756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.688461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:29.727014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c7e36-0413-46e3-a8a7-0fa5d280c14e tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.808389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c290d-e6fd-4cda-bcd8-3e00b25b4ea4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.826032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.826666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.826963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.827227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.829185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.842637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.905549 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2455/4867] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:29 2022] POST /identity/v3/users => generated 291 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:29.909609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.926776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.929534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.929819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.932138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/2c122dec40104766b47bec22678b6edf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.932832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=2c122dec40104766b47bec22678b6edf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.936364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3922-4149-4588-93ac-60085aaf3438 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:29.950794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:29.951431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:29.951743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:29.952013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:29.953497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:29.958117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.960810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invalidating token cache because consumer 2c122dec40104766b47bec22678b6edf has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:29.960963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:29.962853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.966064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:29.985999 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2456/4868] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:16:29.987538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:29.989666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:30.009995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34016d0b-b559-4e4f-800e-7b0e793d4b4f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.018357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.019158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.019678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.020036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.021246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.023174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.023796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '2c122dec40104766b47bec22678b6edf', 'request_id': 'req-bf7bd777-1717-427b-aa03-bc30ca60aa32'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.033912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:30.044592 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2457/4869] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:30.050652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:30.067077 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2413/4870] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:16:29 2022] DELETE /identity/v3/OS-OAUTH1/consumers/2c122dec40104766b47bec22678b6edf => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:30.072790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54470b3-809e-4e62-9cee-1f414af77866 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.072989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:30.075097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:30.085319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d19d42-59ff-4769-8619-e7353e89ec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.088265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19d42-59ff-4769-8619-e7353e89ec80 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.089241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19d42-59ff-4769-8619-e7353e89ec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.089532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19d42-59ff-4769-8619-e7353e89ec80 admin admin] PATH_INFO: `/v3/projects/898c5ffacd6247c6a8bff965ebaf0591/users/04440378e21043b2a287e9b4c800080d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.115527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd777-1717-427b-aa03-bc30ca60aa32 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.116996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d19d42-59ff-4769-8619-e7353e89ec80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898c5ffacd6247c6a8bff965ebaf0591, user_id=04440378e21043b2a287e9b4c800080d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.121067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d19d42-59ff-4769-8619-e7353e89ec80 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.130086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.130991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.131366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.131795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.132616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.136150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.152893 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2458/4871] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:30 2022] PUT /identity/v3/projects/898c5ffacd6247c6a8bff965ebaf0591/users/04440378e21043b2a287e9b4c800080d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:30.159076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2ea04-aa4a-496a-94c8-1e237b9fb0ac None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.159575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2ea04-aa4a-496a-94c8-1e237b9fb0ac None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.159859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2ea04-aa4a-496a-94c8-1e237b9fb0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.181873 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2414/4872] 10.222.0.26 () {60 vars in 1185 bytes} [Tue Sep 27 10:16:30 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:30.184793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:30.226462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b10e0-97e5-4d1d-b038-45afb2909d89 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.231087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-87b2ea04-aa4a-496a-94c8-1e237b9fb0ac None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:30.234683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87b2ea04-aa4a-496a-94c8-1e237b9fb0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:30.392353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.394575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.394575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.394575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/df0f65d49c564bd1a8e6b56a9b619467` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.394904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=df0f65d49c564bd1a8e6b56a9b619467)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.395672 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:30.395672 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:30.395672 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:30.397119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.433298 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2415/4873] 10.222.0.26 () {60 vars in 1285 bytes} [Tue Sep 27 10:16:30 2022] PATCH /identity/v3/OS-OAUTH1/consumers/df0f65d49c564bd1a8e6b56a9b619467 => generated 221 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:30.436194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:30.449492 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2459/4874] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:30.467372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a331a-20f5-4c50-9cac-24b30c58f336 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.480692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.481863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.483011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.483629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/df0f65d49c564bd1a8e6b56a9b619467` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.484496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=df0f65d49c564bd1a8e6b56a9b619467)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.492424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.512996 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2416/4875] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:16:30 2022] GET /identity/v3/OS-OAUTH1/consumers/df0f65d49c564bd1a8e6b56a9b619467 => generated 221 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:30.519251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2ea04-aa4a-496a-94c8-1e237b9fb0ac None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:30.531954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:30.546661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2ea04-aa4a-496a-94c8-1e237b9fb0ac None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.579242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1b23217758a4512a5c5607df183ca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3992721a498e4f488e66475637f66b72', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.580835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.582373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.582865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/df0f65d49c564bd1a8e6b56a9b619467` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.583675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=df0f65d49c564bd1a8e6b56a9b619467)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.585368 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:30.585368 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:16:30.585368 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:16:30.587414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.587898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invalidating token cache because consumer df0f65d49c564bd1a8e6b56a9b619467 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:30.588398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.590013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.671513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:30.673627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:30.688232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'df0f65d49c564bd1a8e6b56a9b619467', 'request_id': 'req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.720152 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2460/4876] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:16:30 2022] DELETE /identity/v3/OS-OAUTH1/consumers/df0f65d49c564bd1a8e6b56a9b619467 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:30.727914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:30.735975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:30.795344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e956-d1fb-4ba9-9644-8ff8d6493253 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.827675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.828405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.833674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.834722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] PATH_INFO: `/v3/users/73586af7bfa14c578de7bc8313a816c4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.836431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] RBAC: Authorizing `identity:delete_user(user_id=73586af7bfa14c578de7bc8313a816c4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.840511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.910982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41769b3e-1268-44f1-bcfa-bc4bd54811a4 tempest-OAUTHConsumersV3Test-1559020135 tempest-OAUTHConsumersV3Test-1559020135-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:30.915909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6b08c-9be6-4004-9ff9-8fdcbc289bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:30.916633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b08c-9be6-4004-9ff9-8fdcbc289bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:30.916952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b08c-9be6-4004-9ff9-8fdcbc289bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:30.917223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b08c-9be6-4004-9ff9-8fdcbc289bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:30.923970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f4c6b08c-9be6-4004-9ff9-8fdcbc289bd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:30.958665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73586af7bfa14c578de7bc8313a816c4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.959726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73586af7bfa14c578de7bc8313a816c4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.962693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6b08c-9be6-4004-9ff9-8fdcbc289bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:30.964727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6b08c-9be6-4004-9ff9-8fdcbc289bd1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:30.982683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73586af7bfa14c578de7bc8313a816c4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.990967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73586af7bfa14c578de7bc8313a816c4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.992267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73586af7bfa14c578de7bc8313a816c4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:30.994540 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2417/4877] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:31.016389 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2461/4878] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:30 2022] DELETE /identity/v3/users/73586af7bfa14c578de7bc8313a816c4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:31.021193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eec3ce-3d1e-45a5-aa98-7301068e5e94 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:31.021916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eec3ce-3d1e-45a5-aa98-7301068e5e94 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:31.022375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eec3ce-3d1e-45a5-aa98-7301068e5e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:31.044399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b6eec3ce-3d1e-45a5-aa98-7301068e5e94 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:31.046096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b6eec3ce-3d1e-45a5-aa98-7301068e5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:31.148344 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2418/4879] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:31.180039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:31.182105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:31.374031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9d143-2da1-4945-906e-1e40e85dc2db admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:31.386922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcdd4f-cdab-4ff7-9869-543d4c23ef92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:31.387571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcdd4f-cdab-4ff7-9869-543d4c23ef92 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:31.387851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcdd4f-cdab-4ff7-9869-543d4c23ef92 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:31.388123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcdd4f-cdab-4ff7-9869-543d4c23ef92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:31.432619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcdd4f-cdab-4ff7-9869-543d4c23ef92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:31.435376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcdd4f-cdab-4ff7-9869-543d4c23ef92 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:31.476118 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2462/4880] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:16:32.512194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:32.513979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:32.514850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:32.550972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:32.584041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:32.585912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:32.730656 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2419/4881] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:32.768991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:32.770043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:32.770575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:32.785476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:32.787344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:32.814900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:32.817415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:32.894144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:32.914680 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2463/4882] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:16:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:32.918843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:32.922276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:32.955307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc206fd-3a2a-463b-9e5c-26c2858e1ae4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:32.966369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:32.969013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:32.970670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:32.971617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] PATH_INFO: `/v3/projects/e350c6d323bb4635a612e94d21f99bf1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:32.972287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085fd17-57bc-4d43-bb35-511c0c619edb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:32.976706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e350c6d323bb4635a612e94d21f99bf1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:32.978945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:32.983809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:32.985095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:32.985860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:32.986744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:32.996576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:33.110732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e350c6d323bb4635a612e94d21f99bf1', 'request_id': 'req-865c482e-a086-41ce-82e1-a9f119ace360'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.142440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e350c6d323bb4635a612e94d21f99bf1', 'request_id': 'req-865c482e-a086-41ce-82e1-a9f119ace360'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.156905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e350c6d323bb4635a612e94d21f99bf1', 'request_id': 'req-865c482e-a086-41ce-82e1-a9f119ace360'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.178896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] The token cache is being invalidate because project e350c6d323bb4635a612e94d21f99bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:33.179734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.182260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.185863 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2464/4883] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:32 2022] DELETE /identity/v3/projects/e350c6d323bb4635a612e94d21f99bf1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:33.187324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:33.187528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:33.191365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:33.192848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:33.222672 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2420/4884] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:33.223634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:33.250504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc99f6-b93f-4183-92ce-2ef4cb120741 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:33.375723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c482e-a086-41ce-82e1-a9f119ace360 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:33.377591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:33.389867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8774d6-5993-48b1-af38-3de266062b48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:33.390514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8774d6-5993-48b1-af38-3de266062b48 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:33.393762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8774d6-5993-48b1-af38-3de266062b48 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:33.394166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8774d6-5993-48b1-af38-3de266062b48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:33.403376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6d8774d6-5993-48b1-af38-3de266062b48 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:33.407684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:33.410291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:33.411958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:33.413114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] PATH_INFO: `/v3/users/d1b23217758a4512a5c5607df183ca83` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:33.417495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b23217758a4512a5c5607df183ca83)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:33.429479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:33.549323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b23217758a4512a5c5607df183ca83'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.551071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b23217758a4512a5c5607df183ca83'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.562547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b23217758a4512a5c5607df183ca83'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.576949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b23217758a4512a5c5607df183ca83'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.592619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8774d6-5993-48b1-af38-3de266062b48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:33.595279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8774d6-5993-48b1-af38-3de266062b48 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:33.597467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b23217758a4512a5c5607df183ca83'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:33.607417 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2465/4885] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:16:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:16:33.613209 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2421/4886] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:33 2022] DELETE /identity/v3/users/d1b23217758a4512a5c5607df183ca83 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:35.306002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15c38-970d-431b-9152-a64082742fed None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:35.306488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15c38-970d-431b-9152-a64082742fed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:35.306589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15c38-970d-431b-9152-a64082742fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:35.354576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-78b15c38-970d-431b-9152-a64082742fed None None] MFA Rules not processed for user `279d3662f60c4a979c22ee3e8fa92865`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:35.357147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78b15c38-970d-431b-9152-a64082742fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:35.441367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:35.582565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2466/4887] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:35.641399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:35.645506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:35.717295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15c38-970d-431b-9152-a64082742fed None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:35.727792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78b15c38-970d-431b-9152-a64082742fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:35.869662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2aad0-b87d-4115-8689-7206d1c77766 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:35.882873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:35.885774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:35.886281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:35.888036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] PATH_INFO: `/v3/projects/3992721a498e4f488e66475637f66b72` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:35.891736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3992721a498e4f488e66475637f66b72)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:35.897956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:35.899413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15c38-970d-431b-9152-a64082742fed None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:35.907118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f267a85-43b3-45bf-9427-b78917b56852 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:35.907956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f267a85-43b3-45bf-9427-b78917b56852 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:35.909155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f267a85-43b3-45bf-9427-b78917b56852 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:35.909502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f267a85-43b3-45bf-9427-b78917b56852 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:35.914031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f267a85-43b3-45bf-9427-b78917b56852 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:36.059117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3992721a498e4f488e66475637f66b72', 'request_id': 'req-192836d7-7b3f-4414-aff3-ff3be323e959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:36.098809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3992721a498e4f488e66475637f66b72', 'request_id': 'req-192836d7-7b3f-4414-aff3-ff3be323e959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:36.130436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3992721a498e4f488e66475637f66b72', 'request_id': 'req-192836d7-7b3f-4414-aff3-ff3be323e959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:36.148052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] The token cache is being invalidate because project 3992721a498e4f488e66475637f66b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:36.148610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:36.156683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-192836d7-7b3f-4414-aff3-ff3be323e959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:36.158558 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2422/4888] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:35 2022] DELETE /identity/v3/projects/3992721a498e4f488e66475637f66b72 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:36.165125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a576-16c8-4b8a-992d-7f5b34475d7b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:36.166275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a576-16c8-4b8a-992d-7f5b34475d7b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:36.168343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a576-16c8-4b8a-992d-7f5b34475d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:36.191475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:36.215657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f267a85-43b3-45bf-9427-b78917b56852 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:36.217805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f267a85-43b3-45bf-9427-b78917b56852 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:36.227043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1165a576-16c8-4b8a-992d-7f5b34475d7b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:36.228491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1165a576-16c8-4b8a-992d-7f5b34475d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:36.272818 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2467/4889] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:16:36.436255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673ca5-ae77-4019-9eae-414ce853105d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:36.436935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673ca5-ae77-4019-9eae-414ce853105d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:36.436935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673ca5-ae77-4019-9eae-414ce853105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:36.440243 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2423/4890] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:36.477591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-82673ca5-ae77-4019-9eae-414ce853105d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:36.480480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-82673ca5-ae77-4019-9eae-414ce853105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:36.514026 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2468/4891] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:36.518796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a576-16c8-4b8a-992d-7f5b34475d7b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:36.532448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a576-16c8-4b8a-992d-7f5b34475d7b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:36.561898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:36.562307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:36.563836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:36.563836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:36.563836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:36.563836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:36.566085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:36.614766 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2424/4892] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:36.615579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82673ca5-ae77-4019-9eae-414ce853105d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:36.630597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82673ca5-ae77-4019-9eae-414ce853105d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:36.647330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e4089-fdd9-4f26-b87d-a5f7b3a886e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:36.648089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e4089-fdd9-4f26-b87d-a5f7b3a886e8 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:36.651254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e4089-fdd9-4f26-b87d-a5f7b3a886e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:36.652425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e4089-fdd9-4f26-b87d-a5f7b3a886e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:36.653749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e4089-fdd9-4f26-b87d-a5f7b3a886e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:36.661029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e4089-fdd9-4f26-b87d-a5f7b3a886e8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:36.757790 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2469/4893] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:36 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:36.761885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:36.780419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5a437-10e1-4922-8117-ab0e9cba311f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:36.876534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1caabb-9020-4860-b168-95a5b7d13937 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:36.876880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1caabb-9020-4860-b168-95a5b7d13937 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:36.877154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1caabb-9020-4860-b168-95a5b7d13937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:36.960329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6f1caabb-9020-4860-b168-95a5b7d13937 None None] MFA Rules not processed for user `dbf876e1c23642578363f9c5995f3d86`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:36.962695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6f1caabb-9020-4860-b168-95a5b7d13937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:36.966458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:36.967351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:36.967783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:36.968306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:36.969321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:36.971579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:37.026781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2425/4894] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:36 2022] POST /identity/v3/users => generated 285 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:37.031274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:37.053418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:37.062649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45344-7ac3-4675-9053-a828ee34fafe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:37.076859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:37.090851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:37.092970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:37.095204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:37.095941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:37.101884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:37.132223 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2426/4895] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:37.136175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:37.149635 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2470/4896] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:16:37.161984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:37.162241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:37.163659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:37.165466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe955a-9c21-438d-8b13-31c385b9311b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:37.176163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:37.176988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:37.177335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:37.178136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:37.179479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:37.184366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:37.205474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] MFA Rules not processed for user `dbf876e1c23642578363f9c5995f3d86`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:37.211249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:37.221750 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2427/4897] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:16:37.226072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:37.235061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2471/4898] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:37.245812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:37.247989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:37.261081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def32af-b0ba-43a5-9a96-1c1692ccdbdb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:37.273360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0d760-e589-480e-bddb-a63e0bf1783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:37.274356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d760-e589-480e-bddb-a63e0bf1783c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:37.275062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d760-e589-480e-bddb-a63e0bf1783c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:37.275605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0d760-e589-480e-bddb-a63e0bf1783c admin admin] PATH_INFO: `/v3/projects/acb5d8053c984b62aadb0f9b5e0e3df2/users/fb61a61f501e4532840c81629f48d689/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:37.310612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0d760-e589-480e-bddb-a63e0bf1783c admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb5d8053c984b62aadb0f9b5e0e3df2, user_id=fb61a61f501e4532840c81629f48d689, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:37.319096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0d760-e589-480e-bddb-a63e0bf1783c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:37.361726 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2428/4899] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:37 2022] PUT /identity/v3/projects/acb5d8053c984b62aadb0f9b5e0e3df2/users/fb61a61f501e4532840c81629f48d689/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:37.381280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0290a-6cc6-4a96-a9f9-0c27125e52a9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:37.381971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0290a-6cc6-4a96-a9f9-0c27125e52a9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:37.383247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0290a-6cc6-4a96-a9f9-0c27125e52a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:37.431221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a1f0290a-6cc6-4a96-a9f9-0c27125e52a9 None None] MFA Rules not processed for user `fb61a61f501e4532840c81629f48d689`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:37.432604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a1f0290a-6cc6-4a96-a9f9-0c27125e52a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:37.458838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84470c44-52b5-4bf2-b277-2562c7cffab8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:37.463427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:37.552120 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2429/4900] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:37.556947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0290a-6cc6-4a96-a9f9-0c27125e52a9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:37.610553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:37.611211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:37.611526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:37.611756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:37.714420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0290a-6cc6-4a96-a9f9-0c27125e52a9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:37.726840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:37.729695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:37.730058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:37.730386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:37.731894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:37.746335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:37.748924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:37.756421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:37.762128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:37.778538 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2472/4901] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:37.830689 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2430/4902] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:37 2022] POST /identity/v3/projects => generated 369 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:37.834476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:37.864443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d8739-5163-4c08-92d2-def39dc4f8dd service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:38.038857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:38.039647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:38.040057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:38.040422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:38.041655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:38.053581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:38.103877 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2473/4903] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:37 2022] POST /identity/v3/users => generated 291 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:38.108077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:38.123255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44304d0-1a3c-44e7-8489-9eeca09ac7a9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:38.132558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:38.133401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:38.133515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:38.133745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:38.134471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:38.140551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:38.159478 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2431/4904] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:38.164650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:38.177041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226fe50-d402-451a-8c35-40af0335b5c4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:38.183207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:38.184126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:38.184474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:38.184872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] PATH_INFO: `/v3/projects/a949fc42c82b40399c24ad17a2a55665/users/cf3ec5f09a5e484295d4fba1fcdc935a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:38.207306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a949fc42c82b40399c24ad17a2a55665, user_id=cf3ec5f09a5e484295d4fba1fcdc935a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:38.211149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:38.234950 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2474/4905] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:38 2022] PUT /identity/v3/projects/a949fc42c82b40399c24ad17a2a55665/users/cf3ec5f09a5e484295d4fba1fcdc935a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:16:38.237027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:38.254118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe9933-1903-4b11-b882-8972105cf75b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:38.407207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b626e9e-dc16-4ef9-a3b7-5df692b3a88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:38.408557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b626e9e-dc16-4ef9-a3b7-5df692b3a88b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:38.408956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b626e9e-dc16-4ef9-a3b7-5df692b3a88b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:38.409246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b626e9e-dc16-4ef9-a3b7-5df692b3a88b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:38.412959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b626e9e-dc16-4ef9-a3b7-5df692b3a88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:38.416803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b626e9e-dc16-4ef9-a3b7-5df692b3a88b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:38.466457 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2432/4906] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:38.468386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:38.507138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a886b4-4832-4da5-9366-a3abeae379ab admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:38.522371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:38.523111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:38.523403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:38.528025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] PATH_INFO: `/v3/projects/a949fc42c82b40399c24ad17a2a55665/users/cf3ec5f09a5e484295d4fba1fcdc935a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:38.535077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a949fc42c82b40399c24ad17a2a55665, user_id=cf3ec5f09a5e484295d4fba1fcdc935a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:38.553640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:38.615522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2475/4907] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:38 2022] PUT /identity/v3/projects/a949fc42c82b40399c24ad17a2a55665/users/cf3ec5f09a5e484295d4fba1fcdc935a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:38.625948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be064-ebb2-4af4-8343-6f0708cf5984 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:38.626641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be064-ebb2-4af4-8343-6f0708cf5984 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:38.626976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be064-ebb2-4af4-8343-6f0708cf5984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:38.703025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c68be064-ebb2-4af4-8343-6f0708cf5984 None None] MFA Rules not processed for user `cf3ec5f09a5e484295d4fba1fcdc935a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:38.704527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c68be064-ebb2-4af4-8343-6f0708cf5984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:38.761056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:38.898124 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2433/4908] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:38.899329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:38.916864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130bc81-4bf6-4f5f-bfd2-e45c7afa6098 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:38.929687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:38.931746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:38.934456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:38.935792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:38.937359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:38.939258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:39.003782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2476/4909] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:38 2022] POST /identity/v3/projects => generated 359 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:39.007100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68be064-ebb2-4af4-8343-6f0708cf5984 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:39.021759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68be064-ebb2-4af4-8343-6f0708cf5984 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:39.188904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:39.189515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:39.189794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:39.190066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:39.190720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:39.193318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:39.240575 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2434/4910] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:39 2022] POST /identity/v3/users => generated 413 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:39.242650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:39.259632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f5ac5-0f34-437a-92e5-d854ea2cd2aa tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:39.267381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:39.268317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:39.268635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:39.269100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/users/fed568a09d2749b498e4568c868ea30e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:39.293950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_user(user_id=fed568a09d2749b498e4568c868ea30e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:39.297507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:39.299725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2477/4911] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:16:39 2022] GET /identity/v3/users/fed568a09d2749b498e4568c868ea30e => generated 413 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:39.303934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:39.320168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8d2c-c368-4793-9e7d-ebf707c57125 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:39.326517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:39.327404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:39.327990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:39.328400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/users/fed568a09d2749b498e4568c868ea30e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:39.329684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_user(user_id=fed568a09d2749b498e4568c868ea30e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:39.332178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:39.512348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed568a09d2749b498e4568c868ea30e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:39.514977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed568a09d2749b498e4568c868ea30e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:39.549351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed568a09d2749b498e4568c868ea30e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:39.573167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed568a09d2749b498e4568c868ea30e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:39.609688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed568a09d2749b498e4568c868ea30e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:39.632512 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2435/4912] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:39 2022] DELETE /identity/v3/users/fed568a09d2749b498e4568c868ea30e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:39.636435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:39.639173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:39.666736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28100cdc-4ca1-4797-ba28-fa6ad6d7bcea tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:39.932604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:39.937430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:39.937711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:39.939129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/a28e3cbcc33047f8aef2be28d0127c2e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:39.946157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a28e3cbcc33047f8aef2be28d0127c2e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:39.953121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:40.014724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:40.123698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28e3cbcc33047f8aef2be28d0127c2e', 'request_id': 'req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:40.150881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28e3cbcc33047f8aef2be28d0127c2e', 'request_id': 'req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:40.176576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28e3cbcc33047f8aef2be28d0127c2e', 'request_id': 'req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:40.212494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project a28e3cbcc33047f8aef2be28d0127c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:40.214588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:40.218623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:40.231695 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2478/4913] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:39 2022] DELETE /identity/v3/projects/a28e3cbcc33047f8aef2be28d0127c2e => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:40.238021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:40.263714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231a1e4-ff11-4cef-8fef-5462e9b8df36 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:40.316270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:40.319692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:40.335606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:40.473630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:40.474275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:40.474630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:40.474907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:40.475620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:40.477532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:40.527644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2436/4914] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:40 2022] POST /identity/v3/projects => generated 348 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:40.529104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:40.540358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:40.627236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ffdc-5b36-4a6e-8e28-e7c73e7194e2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:40.829446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c954-d930-43f4-bd20-f29b8a1bf870 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:40.841423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:40.843240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:40.843587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:40.843953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:40.844725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:40.845215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:40.848170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:40.851621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:40.857822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:40.859971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:40.860506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] PATH_INFO: `/v3/users/dc369ca971424247b96050e09368a7bb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:40.863816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc369ca971424247b96050e09368a7bb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:40.879518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:40.887760 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2437/4915] 10.222.0.26 () {58 vars in 1160 bytes} [Tue Sep 27 10:16:40 2022] GET /identity/v3/projects?is_domain=True => generated 1020 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:40.892000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:40.911476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c688-be11-4468-9270-af5eec97e9a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:40.921052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:40.921969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:40.922466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:40.922905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:40.928787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:40.931874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:40.990784 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2438/4916] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 10:16:40 2022] GET /identity/v3/domains => generated 829 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:16:40.992003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:41.027539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59485d-a917-45f9-b475-b1eacbf11cb9 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:41.038096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc369ca971424247b96050e09368a7bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.040852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc369ca971424247b96050e09368a7bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.042286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:41.043277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:41.044468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:41.044930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/fc81ca9602ed4a53889cc356c89f7c28` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:41.047068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:update_project(project_id=fc81ca9602ed4a53889cc356c89f7c28)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:41.049624 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:41.049624 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:41.049624 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:41.051658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:41.057105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:41.072564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc369ca971424247b96050e09368a7bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.089337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc369ca971424247b96050e09368a7bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.091841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc369ca971424247b96050e09368a7bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.094287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.095975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.144207 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2479/4917] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:40 2022] DELETE /identity/v3/users/dc369ca971424247b96050e09368a7bb => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:41.153769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.154538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.160290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2439/4918] 10.222.0.26 () {60 vars in 1252 bytes} [Tue Sep 27 10:16:40 2022] PATCH /identity/v3/projects/fc81ca9602ed4a53889cc356c89f7c28 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:41.164604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:41.166277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:41.380266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892756e-b6c3-4ca7-aa75-abcf5bd19ce2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:41.390047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:41.390990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:41.392638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:41.395741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/fc81ca9602ed4a53889cc356c89f7c28` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:41.423882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fc81ca9602ed4a53889cc356c89f7c28)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:41.432706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:41.502460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.515442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.615494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28', 'request_id': 'req-e70dbcb3-de0f-47ed-b701-f53c8269da48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.638799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28', 'request_id': 'req-e70dbcb3-de0f-47ed-b701-f53c8269da48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.654032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28', 'request_id': 'req-e70dbcb3-de0f-47ed-b701-f53c8269da48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.672477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project fc81ca9602ed4a53889cc356c89f7c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:41.673123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.673968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.686622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28', 'request_id': 'req-e70dbcb3-de0f-47ed-b701-f53c8269da48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.688304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28', 'request_id': 'req-e70dbcb3-de0f-47ed-b701-f53c8269da48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.693798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'fc81ca9602ed4a53889cc356c89f7c28', 'request_id': 'req-e70dbcb3-de0f-47ed-b701-f53c8269da48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:41.705236 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2480/4919] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:41 2022] DELETE /identity/v3/projects/fc81ca9602ed4a53889cc356c89f7c28 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:41.710221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:41.711799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:41.943746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cab40-163a-476c-8393-22975a434270 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:41.960888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:42.125495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:42.126292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:42.126703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:42.127085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:42.128164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:42.130009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:42.237807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:42.239953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:42.249494 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2440/4920] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:41 2022] POST /identity/v3/projects => generated 360 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:42.270426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dbcb3-de0f-47ed-b701-f53c8269da48 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:42.275783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:42.395466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:42.396320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:42.397585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:42.397966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/5afdf44a50d345d88b12c8923cfb7eb6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:42.402322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=5afdf44a50d345d88b12c8923cfb7eb6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:42.406496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:42.416068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:42.426780 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2481/4921] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:42 2022] GET /identity/v3/projects/5afdf44a50d345d88b12c8923cfb7eb6 => generated 360 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:16:42.437717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d78bf-5a8b-40bf-a764-d0f0f9eac45a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:42.451783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:42.452919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:42.452919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:42.454882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/5afdf44a50d345d88b12c8923cfb7eb6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:42.458105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5afdf44a50d345d88b12c8923cfb7eb6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:42.465295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:42.589630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afdf44a50d345d88b12c8923cfb7eb6', 'request_id': 'req-849bf85c-c925-4215-bf41-05ccef8df98f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:42.633084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afdf44a50d345d88b12c8923cfb7eb6', 'request_id': 'req-849bf85c-c925-4215-bf41-05ccef8df98f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:42.648533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afdf44a50d345d88b12c8923cfb7eb6', 'request_id': 'req-849bf85c-c925-4215-bf41-05ccef8df98f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:42.676608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 5afdf44a50d345d88b12c8923cfb7eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:42.678848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:42.681113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:42.684112 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2441/4922] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:42 2022] DELETE /identity/v3/projects/5afdf44a50d345d88b12c8923cfb7eb6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:42.689203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:42.691101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:42.716196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12651f0-11b1-4ecf-b7fe-1e06b7325f71 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:42.844840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:42.845240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:42.845572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:42.845941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:42.846849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:42.849000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:42.919100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:42.922306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:42.962087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2482/4923] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:42 2022] POST /identity/v3/projects => generated 360 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:42.963477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:43.117250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2b859-eb8e-4a6e-9aa5-b1514c214da3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.126771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bf85c-c925-4215-bf41-05ccef8df98f tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.132254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.132254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.132737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.133115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/e7c8bb7240fa43c88327fdee5f2701d6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.137183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=e7c8bb7240fa43c88327fdee5f2701d6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.142342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.148922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2483/4924] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:42 2022] GET /identity/v3/projects/e7c8bb7240fa43c88327fdee5f2701d6 => generated 360 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:43.158094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:43.188280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a970f8-bec5-49eb-9508-d7adf9bd593a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.193457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:43.202894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.203766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.204138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.204418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/e7c8bb7240fa43c88327fdee5f2701d6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.211041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e7c8bb7240fa43c88327fdee5f2701d6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.214759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.296541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.297327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.297683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.298024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] PATH_INFO: `/v3/projects/27c0d3daa08340d89d288bbf811fece9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.299356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c0d3daa08340d89d288bbf811fece9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.301399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.332190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:43.358557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c8bb7240fa43c88327fdee5f2701d6', 'request_id': 'req-df863595-4214-4ad6-9b7b-94cefd6760a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.384370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c8bb7240fa43c88327fdee5f2701d6', 'request_id': 'req-df863595-4214-4ad6-9b7b-94cefd6760a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.386499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c0d3daa08340d89d288bbf811fece9', 'request_id': 'req-bba04202-71b5-4848-9759-e2dd485f26c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.395844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c0d3daa08340d89d288bbf811fece9', 'request_id': 'req-bba04202-71b5-4848-9759-e2dd485f26c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.398187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c8bb7240fa43c88327fdee5f2701d6', 'request_id': 'req-df863595-4214-4ad6-9b7b-94cefd6760a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.412746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c0d3daa08340d89d288bbf811fece9', 'request_id': 'req-bba04202-71b5-4848-9759-e2dd485f26c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.426437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project e7c8bb7240fa43c88327fdee5f2701d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:43.426826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.427532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.430716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] The token cache is being invalidate because project 27c0d3daa08340d89d288bbf811fece9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:43.431598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.432564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba04202-71b5-4848-9759-e2dd485f26c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:43.433740 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2484/4925] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:43 2022] DELETE /identity/v3/projects/e7c8bb7240fa43c88327fdee5f2701d6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:43.434655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:43.435433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2442/4926] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:42 2022] DELETE /identity/v3/projects/27c0d3daa08340d89d288bbf811fece9 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:16:43.440941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6160d-6816-4f70-b1e8-b4aeac673afb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.440941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6160d-6816-4f70-b1e8-b4aeac673afb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.441223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6160d-6816-4f70-b1e8-b4aeac673afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.442899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:43.479902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-12f6160d-6816-4f70-b1e8-b4aeac673afb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:43.482207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-12f6160d-6816-4f70-b1e8-b4aeac673afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:43.553507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df863595-4214-4ad6-9b7b-94cefd6760a1 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.638657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.640350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.640828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.641121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2443/4927] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:16:43.641532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.643108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.645403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b69e2-77c1-4c9c-a8d6-abf6a12722f9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.645403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b69e2-77c1-4c9c-a8d6-abf6a12722f9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.645403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b69e2-77c1-4c9c-a8d6-abf6a12722f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.646787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.671615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d47b69e2-77c1-4c9c-a8d6-abf6a12722f9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:43.673238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d47b69e2-77c1-4c9c-a8d6-abf6a12722f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:43.689136 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2444/4928] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:43.692989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b69e2-77c1-4c9c-a8d6-abf6a12722f9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:43.706028 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2485/4929] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:16:43 2022] POST /identity/v3/projects => generated 347 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:43.707987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:43.710583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:43.723418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b69e2-77c1-4c9c-a8d6-abf6a12722f9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.826150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:43.843425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:43.862716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.863801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.864498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.866437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.870875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:43.871422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.875318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.880956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e1b8c-1079-49e1-874c-52bca85776df tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.892392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.893080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.893359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.893629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/56874d567fe44c25be731c35999253d5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.897616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=56874d567fe44c25be731c35999253d5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.900507 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2445/4930] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:16:43.900762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.905632 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2486/4931] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:43 2022] GET /identity/v3/projects/56874d567fe44c25be731c35999253d5 => generated 347 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:43.905784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:43.914852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:43.929132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ea65d-dc5e-4d99-808c-d69066a24ef0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.935656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.936183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c335a-699a-4625-af04-0a1d33287e4a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:43.936328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.936720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.937021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.937677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.939569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.944967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:43.945910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:43.946387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:43.946920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/56874d567fe44c25be731c35999253d5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:43.949447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=56874d567fe44c25be731c35999253d5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:43.952473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:43.994126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2446/4932] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:43 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:43.999454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.017609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56874d567fe44c25be731c35999253d5', 'request_id': 'req-8310decd-42e3-4f12-927e-95ba75384ecf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.035816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56874d567fe44c25be731c35999253d5', 'request_id': 'req-8310decd-42e3-4f12-927e-95ba75384ecf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.044972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56874d567fe44c25be731c35999253d5', 'request_id': 'req-8310decd-42e3-4f12-927e-95ba75384ecf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.065734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 56874d567fe44c25be731c35999253d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:44.066058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.066892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.069067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2487/4933] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:43 2022] DELETE /identity/v3/projects/56874d567fe44c25be731c35999253d5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:44.074362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.076809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:44.085407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f3361-c797-429b-ad1a-2f0a37388c9f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.187752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.190072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.190364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.190640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.192400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.195045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310decd-42e3-4f12-927e-95ba75384ecf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.195322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.202373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.203374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.203862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.204375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.205437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.207629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.235138 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2447/4934] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:43 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:44.236721 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2488/4935] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:16:44 2022] POST /identity/v3/domains => generated 276 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:44.240394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.240768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.242928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:44.263881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bf03e-56e4-4444-9627-dfcdb5d8a31c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.333888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:44.342143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.343427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.345485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.346467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.347431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.349619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860826a-afa9-490a-9be1-db4a65248fb2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.350723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.361333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.362295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.362679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.363050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.365794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.368085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.374868 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2448/4936] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:44.376163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.389814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e076540-0e88-498e-97c3-82ddaa2b5c53 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.395550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.396392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.396813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.397296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.398302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.400335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.409755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:44.417076 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2449/4937] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:44.417809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.433110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7f4e6-743e-4e87-bcaa-45c085e100d6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.439787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db2366-e4a6-4364-9091-5afdbb30ade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.440640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2366-e4a6-4364-9091-5afdbb30ade6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.441096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2366-e4a6-4364-9091-5afdbb30ade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.441472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2366-e4a6-4364-9091-5afdbb30ade6 admin admin] PATH_INFO: `/v3/projects/466f9ae4f69f4ba6a9484b5a93bb6284/users/5a34669b510b41eea8b10ba3c9f650ef/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.454853 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2489/4938] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:44 2022] POST /identity/v3/projects => generated 405 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:44.457691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.464297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db2366-e4a6-4364-9091-5afdbb30ade6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=466f9ae4f69f4ba6a9484b5a93bb6284, user_id=5a34669b510b41eea8b10ba3c9f650ef, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.468775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db2366-e4a6-4364-9091-5afdbb30ade6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.492711 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2450/4939] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:44 2022] PUT /identity/v3/projects/466f9ae4f69f4ba6a9484b5a93bb6284/users/5a34669b510b41eea8b10ba3c9f650ef/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:16:44.499705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa3fa-a323-4aff-86f5-fb27af369ccb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.501946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa3fa-a323-4aff-86f5-fb27af369ccb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.502665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa3fa-a323-4aff-86f5-fb27af369ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.576685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-66eaa3fa-a323-4aff-86f5-fb27af369ccb None None] MFA Rules not processed for user `5a34669b510b41eea8b10ba3c9f650ef`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:44.578637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66eaa3fa-a323-4aff-86f5-fb27af369ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:44.589591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a4490-f5d9-4386-99d6-8b7de468deb3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.693833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.695590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.696036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.696499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/5a6e03dcc89249038896de1bb6d0fd0c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.698199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=5a6e03dcc89249038896de1bb6d0fd0c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.701387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.702853 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2451/4940] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:44.708135 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2490/4941] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:44 2022] GET /identity/v3/projects/5a6e03dcc89249038896de1bb6d0fd0c => generated 405 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:44.712415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa3fa-a323-4aff-86f5-fb27af369ccb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.723871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.726452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:44.759839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa3fa-a323-4aff-86f5-fb27af369ccb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.768625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.769437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.769881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.770558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/5a6e03dcc89249038896de1bb6d0fd0c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.771658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5a6e03dcc89249038896de1bb6d0fd0c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:44.776059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:44.879136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a60562-637e-4420-8ba8-c9b10c872bc2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:44.885040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6e03dcc89249038896de1bb6d0fd0c', 'request_id': 'req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.903715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6e03dcc89249038896de1bb6d0fd0c', 'request_id': 'req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.924433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6e03dcc89249038896de1bb6d0fd0c', 'request_id': 'req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.939851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 5a6e03dcc89249038896de1bb6d0fd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:44.940955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.941869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:44.943605 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2452/4942] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:44 2022] DELETE /identity/v3/projects/5a6e03dcc89249038896de1bb6d0fd0c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:44.945546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:44.948885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:44.983640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:44.984300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:44.984562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:44.985058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:44.990111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:45.001736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:45.027469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:45.091392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac567e7a-2fa9-4a1f-82b1-b2b8b488cd3e tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:45.102846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:45.103696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:45.104334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:45.104736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] PATH_INFO: `/v3/users/e8d785381149436fb7caf8ff5e55c9eb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:45.107476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d785381149436fb7caf8ff5e55c9eb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:45.111086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:45.170263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:45.174420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:45.200434 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2491/4943] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:45.200434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:45.203101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:45.259616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d785381149436fb7caf8ff5e55c9eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.261151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d785381149436fb7caf8ff5e55c9eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.277099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d785381149436fb7caf8ff5e55c9eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.303818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d785381149436fb7caf8ff5e55c9eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.349658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7eeb584-ee09-40ca-b66f-499dcb3446b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d785381149436fb7caf8ff5e55c9eb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.364780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216bf40-a7cb-4a9a-b30f-9ee6c45f924d service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:45.370193 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2453/4944] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:44 2022] DELETE /identity/v3/users/e8d785381149436fb7caf8ff5e55c9eb => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:45.371851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:45.372140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:45.372413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:45.456337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:45.461228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:45.568571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:45.569707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:45.570195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:45.570954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains/8d716accb1a64005a2b6109124d90606` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:45.575510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=8d716accb1a64005a2b6109124d90606)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:45.579781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:45.673927 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2454/4945] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:45.705460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:45.707117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:45.725879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '8d716accb1a64005a2b6109124d90606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.729789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '8d716accb1a64005a2b6109124d90606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.817821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '8d716accb1a64005a2b6109124d90606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.818555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '8d716accb1a64005a2b6109124d90606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:45.822061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2492/4946] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:16:45 2022] PATCH /identity/v3/domains/8d716accb1a64005a2b6109124d90606 => generated 277 bytes in 623 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:45.826213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:45.827934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:45.852829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ff362-3c37-49bc-83a7-00fd6da7e3ed None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:45.864029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:45.890895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:46.012910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:46.016241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:46.016761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:46.018586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:46.025410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:46.047837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13718273-4f33-4375-b902-a537369c78ab tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:46.060823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:46.061847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:46.062202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:46.062636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains/8d716accb1a64005a2b6109124d90606` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:46.093691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=8d716accb1a64005a2b6109124d90606)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:46.095230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:46.156139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '8d716accb1a64005a2b6109124d90606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.188119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '8d716accb1a64005a2b6109124d90606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.191918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:46.195205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:46.227123 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2455/4947] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:46.347735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d716accb1a64005a2b6109124d90606', 'request_id': 'req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.376377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d716accb1a64005a2b6109124d90606', 'request_id': 'req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.386454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d716accb1a64005a2b6109124d90606', 'request_id': 'req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.408529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 8d716accb1a64005a2b6109124d90606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:46.408998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.410561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.427249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '8d716accb1a64005a2b6109124d90606', 'request_id': 'req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.428533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '8d716accb1a64005a2b6109124d90606', 'request_id': 'req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.439894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '8d716accb1a64005a2b6109124d90606', 'request_id': 'req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:46.453086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2493/4948] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:45 2022] DELETE /identity/v3/domains/8d716accb1a64005a2b6109124d90606 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:46.457915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:46.475278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303aa66-d4d6-461d-9514-01b97e98fecd service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:46.584222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:46.585036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:46.585408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:46.585850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:46.588099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:46.590125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:46.631760 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2456/4949] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:16:46 2022] POST /identity/v3/domains => generated 276 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:46.634700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:46.637252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:46.673079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7f7a1-c2dc-48c1-9a14-7de44b7a73ee tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:46.787966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:46.789167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:46.789630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:46.790190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:46.791171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:46.795875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:46.811225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:46.875295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:46.877803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:46.891314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2494/4950] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:46 2022] POST /identity/v3/projects => generated 410 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:46.895313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:47.043814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446a5c1-351c-44f3-a1a4-23b213907082 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:47.050928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:47.051715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.052158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.052861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.054999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:47.058356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:47.066324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:47.093495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a5121-2555-4b80-8111-147fa3cf07de tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:47.114579 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2495/4951] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:46 2022] POST /identity/v3/projects => generated 403 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:47.120151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:47.190423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:47.192966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.193646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.194291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] PATH_INFO: `/v3/projects/8f1ae55ddbd8453c885e531ab0620beb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.197360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f1ae55ddbd8453c885e531ab0620beb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:47.200625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:47.219396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cac866-d7c8-4af0-8789-d4897cb89775 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:47.227985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:47.229486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.229486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.229803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/1f19d5b23ba3494e897f322e84579150` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.231978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1f19d5b23ba3494e897f322e84579150)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:47.234338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:47.285676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f1ae55ddbd8453c885e531ab0620beb', 'request_id': 'req-fb960ba4-d15c-4093-aec7-98cdf7fa764b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.301295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f1ae55ddbd8453c885e531ab0620beb', 'request_id': 'req-fb960ba4-d15c-4093-aec7-98cdf7fa764b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.320595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f1ae55ddbd8453c885e531ab0620beb', 'request_id': 'req-fb960ba4-d15c-4093-aec7-98cdf7fa764b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.352327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f19d5b23ba3494e897f322e84579150', 'request_id': 'req-252766c5-988e-4bef-80fa-bd19c26330dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.356436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] The token cache is being invalidate because project 8f1ae55ddbd8453c885e531ab0620beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:47.356749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.358562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb960ba4-d15c-4093-aec7-98cdf7fa764b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.366986 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2457/4952] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:46 2022] DELETE /identity/v3/projects/8f1ae55ddbd8453c885e531ab0620beb => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:47.368670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f7d5-0ab0-4549-a45f-5d404e649b43 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.369052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f7d5-0ab0-4549-a45f-5d404e649b43 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.369417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f7d5-0ab0-4549-a45f-5d404e649b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.377211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f19d5b23ba3494e897f322e84579150', 'request_id': 'req-252766c5-988e-4bef-80fa-bd19c26330dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.382371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:47.411336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f19d5b23ba3494e897f322e84579150', 'request_id': 'req-252766c5-988e-4bef-80fa-bd19c26330dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.437545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fe46f7d5-0ab0-4549-a45f-5d404e649b43 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:47.440351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe46f7d5-0ab0-4549-a45f-5d404e649b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:47.514682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 1f19d5b23ba3494e897f322e84579150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:47.516137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.518795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.529243 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2496/4953] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:47 2022] DELETE /identity/v3/projects/1f19d5b23ba3494e897f322e84579150 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:47.533879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:47.535459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:47.539119 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2458/4954] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:47.547918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31054b2f-e5f9-4e73-a825-01c9aa87f363 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.548286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31054b2f-e5f9-4e73-a825-01c9aa87f363 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.548620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31054b2f-e5f9-4e73-a825-01c9aa87f363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.607899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-31054b2f-e5f9-4e73-a825-01c9aa87f363 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:47.620545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-31054b2f-e5f9-4e73-a825-01c9aa87f363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:47.646137 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2459/4955] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:47.650980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31054b2f-e5f9-4e73-a825-01c9aa87f363 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:47.671630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31054b2f-e5f9-4e73-a825-01c9aa87f363 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:47.686441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:47.687650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.690067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.690579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.692651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:47.693264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:47.695266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:47.732746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252766c5-988e-4bef-80fa-bd19c26330dc tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:47.737776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:47.738461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.738896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.739031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/d62dc621944d4f7ca7d8419bc80d149e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.740363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d62dc621944d4f7ca7d8419bc80d149e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:47.742538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:47.748152 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2460/4956] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:16:47.753164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:47.773999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abddcf-595f-42c0-88db-c0880def19f1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:47.789874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:47.791668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:47.792362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:47.793991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:47.796214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:47.798939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:47.810726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:47.860696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62dc621944d4f7ca7d8419bc80d149e', 'request_id': 'req-e32365f8-5058-4d91-b907-f1392aed7648'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.877978 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2461/4957] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:47 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:47.882886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:47.887145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62dc621944d4f7ca7d8419bc80d149e', 'request_id': 'req-e32365f8-5058-4d91-b907-f1392aed7648'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.898886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62dc621944d4f7ca7d8419bc80d149e', 'request_id': 'req-e32365f8-5058-4d91-b907-f1392aed7648'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.945932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project d62dc621944d4f7ca7d8419bc80d149e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:47.946433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.948214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:47.954853 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2497/4958] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:47 2022] DELETE /identity/v3/projects/d62dc621944d4f7ca7d8419bc80d149e => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:16:47.960856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:47.963569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:48.059805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccdce4-403c-4a09-9c7d-07a7bb25d4ed admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:48.090062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:48.090712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.090979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.091297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.092000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:48.094075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:48.149897 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2462/4959] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:47 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:48.151504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32365f8-5058-4d91-b907-f1392aed7648 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:48.154527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:48.161082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:48.206151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1227c6-c4da-4c65-9307-4090cdb97878 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:48.211409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:48.213524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:48.214155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.214437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.215543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.216174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:48.217888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:48.248393 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2463/4960] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:48.251929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:48.268357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcec879-4df5-4c72-b409-38a4b724c97f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:48.274378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:48.275180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.275452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.275891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.276314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:48.279610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:48.293214 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2464/4961] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:48.297364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:48.303545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:48.305543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.306174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.306670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains/dd29f5eab466436a84297e1827688e59` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.309117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=dd29f5eab466436a84297e1827688e59)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:48.317138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:48.318956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e67d55-ff25-4b7b-acbc-b0106a27d4d6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:48.325482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51044ff-9175-4e22-8de7-d2b42b8d7eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:48.325607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51044ff-9175-4e22-8de7-d2b42b8d7eff admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.325892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51044ff-9175-4e22-8de7-d2b42b8d7eff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.326171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51044ff-9175-4e22-8de7-d2b42b8d7eff admin admin] PATH_INFO: `/v3/projects/a6c1259e89284611bc708904b122eb7e/users/095e814e7164440a9dddb2095a2912ef/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.353870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51044ff-9175-4e22-8de7-d2b42b8d7eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c1259e89284611bc708904b122eb7e, user_id=095e814e7164440a9dddb2095a2912ef, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:48.357978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51044ff-9175-4e22-8de7-d2b42b8d7eff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:48.386767 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2465/4962] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:48 2022] PUT /identity/v3/projects/a6c1259e89284611bc708904b122eb7e/users/095e814e7164440a9dddb2095a2912ef/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:48.387058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'dd29f5eab466436a84297e1827688e59'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.388601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'dd29f5eab466436a84297e1827688e59'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.397912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.398195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.398468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.436007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] MFA Rules not processed for user `095e814e7164440a9dddb2095a2912ef`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:48.437665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:48.452148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'dd29f5eab466436a84297e1827688e59'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.455028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'dd29f5eab466436a84297e1827688e59'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.461885 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2498/4963] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:16:47 2022] PATCH /identity/v3/domains/dd29f5eab466436a84297e1827688e59 => generated 277 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:48.463661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:48.465618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:48.647429 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2466/4964] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:16:48.649826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754cdbc-77ba-4f51-9034-db3fc010f691 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:48.658913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:48.660626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:48.685423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:48.686182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.686678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.687016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains/dd29f5eab466436a84297e1827688e59` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.699226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:48.703121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=dd29f5eab466436a84297e1827688e59)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:48.704977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:48.743438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'dd29f5eab466436a84297e1827688e59'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.761080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'dd29f5eab466436a84297e1827688e59'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.775705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475a262-02fb-4f26-ad39-02831c6c9c34 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:48.779777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:48.780491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:48.780927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:48.781281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:48.804617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:48.806291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:48.817647 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2467/4965] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:16:48.853041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd29f5eab466436a84297e1827688e59', 'request_id': 'req-bb880b80-e5ab-4ef2-a109-803309cf78c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.885702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd29f5eab466436a84297e1827688e59', 'request_id': 'req-bb880b80-e5ab-4ef2-a109-803309cf78c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.887427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:UPDATE=1,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:48.894273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd29f5eab466436a84297e1827688e59', 'request_id': 'req-bb880b80-e5ab-4ef2-a109-803309cf78c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.921131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project dd29f5eab466436a84297e1827688e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:48.921514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.924844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.944251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'dd29f5eab466436a84297e1827688e59', 'request_id': 'req-bb880b80-e5ab-4ef2-a109-803309cf78c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.945240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'dd29f5eab466436a84297e1827688e59', 'request_id': 'req-bb880b80-e5ab-4ef2-a109-803309cf78c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.952571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'dd29f5eab466436a84297e1827688e59', 'request_id': 'req-bb880b80-e5ab-4ef2-a109-803309cf78c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:48.967083 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2499/4966] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:48 2022] DELETE /identity/v3/domains/dd29f5eab466436a84297e1827688e59 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:48.967523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:48.989937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749856e-548f-4542-9e82-4a037d22d976 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:49.101641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:49.102504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:49.102857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:49.103274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:49.104199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:49.107521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:49.177242 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2468/4967] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:48 2022] POST /identity/v3/projects => generated 378 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:49.179892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:49.189851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:49.225136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880b80-e5ab-4ef2-a109-803309cf78c7 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:49.342967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:49.343752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:49.344101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:49.344501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/7c3a761360384f7a80af3c71393ebbc4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:49.345878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=7c3a761360384f7a80af3c71393ebbc4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:49.348386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:49.355766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2500/4968] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:49 2022] GET /identity/v3/projects/7c3a761360384f7a80af3c71393ebbc4 => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:49.356005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:49.376024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d172b96-be8e-4be8-89c2-c6b26c07630a tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:49.387581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:49.388430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:49.388620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:49.388890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:49.389971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:49.392177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:49.447591 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2469/4969] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:16:49 2022] GET /identity/v3/projects => generated 6541 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:16:49.455294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:49.468877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4261954-2657-4266-9381-1e529f716272 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:49.475994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:49.477171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:49.477721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:49.478299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/7c3a761360384f7a80af3c71393ebbc4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:49.479890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7c3a761360384f7a80af3c71393ebbc4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:49.482400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:49.557245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c3a761360384f7a80af3c71393ebbc4', 'request_id': 'req-f76a8d05-8895-4ee6-a925-06c2dc535b03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:49.572741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c3a761360384f7a80af3c71393ebbc4', 'request_id': 'req-f76a8d05-8895-4ee6-a925-06c2dc535b03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:49.585481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c3a761360384f7a80af3c71393ebbc4', 'request_id': 'req-f76a8d05-8895-4ee6-a925-06c2dc535b03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:49.610680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 7c3a761360384f7a80af3c71393ebbc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:49.610896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:49.611515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:49.620416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2501/4970] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:49 2022] DELETE /identity/v3/projects/7c3a761360384f7a80af3c71393ebbc4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:49.621236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:49.623424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:49.638147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07e3d7-a914-4a91-9531-f12abfa2f742 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:49.712226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:49.713713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:49.713713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:49.713866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:49.714491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:49.716399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:49.788031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2470/4971] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:16:49 2022] POST /identity/v3/projects => generated 347 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:49.792094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:49.822784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a8d05-8895-4ee6-a925-06c2dc535b03 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:49.890250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:49.909189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:49.910860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:49.911642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:49.912096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/3bd75e5b337f475a8a6d4d8aabef495e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:49.914440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:update_project(project_id=3bd75e5b337f475a8a6d4d8aabef495e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:49.916294 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:49.916294 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:16:49.916294 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:16:49.917855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:49.984996 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2502/4972] 10.222.0.26 () {60 vars in 1252 bytes} [Tue Sep 27 10:16:49 2022] PATCH /identity/v3/projects/3bd75e5b337f475a8a6d4d8aabef495e => generated 361 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:49.988791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:50.002420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cd07-50dc-474d-bb78-7bd325ae1d12 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:50.007387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:50.008109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.008460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.008844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/3bd75e5b337f475a8a6d4d8aabef495e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.022958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=3bd75e5b337f475a8a6d4d8aabef495e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:50.024891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:50.028851 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2471/4973] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:49 2022] GET /identity/v3/projects/3bd75e5b337f475a8a6d4d8aabef495e => generated 348 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:50.031948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:50.044754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c1551-e837-4a22-98ae-dff872dd350b tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:50.051407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:50.052139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.052541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.053238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/3bd75e5b337f475a8a6d4d8aabef495e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.055156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3bd75e5b337f475a8a6d4d8aabef495e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:50.057728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:50.155244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd75e5b337f475a8a6d4d8aabef495e', 'request_id': 'req-35e45b41-b6b1-4eb8-8b25-02662c445ade'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:50.178519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd75e5b337f475a8a6d4d8aabef495e', 'request_id': 'req-35e45b41-b6b1-4eb8-8b25-02662c445ade'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:50.192370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd75e5b337f475a8a6d4d8aabef495e', 'request_id': 'req-35e45b41-b6b1-4eb8-8b25-02662c445ade'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:50.211952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 3bd75e5b337f475a8a6d4d8aabef495e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:50.212258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:50.213036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35e45b41-b6b1-4eb8-8b25-02662c445ade tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:50.219991 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2503/4974] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:50 2022] DELETE /identity/v3/projects/3bd75e5b337f475a8a6d4d8aabef495e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:16:50.222014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:50.224521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:50.235550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:50.241620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc923e-d542-421c-ae40-27ba92e2a550 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:50.382005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:50.382894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.383218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.383968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.385998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:50.387892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461f07a-0102-462c-856f-5481eced3e8a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.388261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461f07a-0102-462c-856f-5481eced3e8a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.392053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:50.392220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461f07a-0102-462c-856f-5481eced3e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.438354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0461f07a-0102-462c-856f-5481eced3e8a None None] MFA Rules not processed for user `04440378e21043b2a287e9b4c800080d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:50.441164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0461f07a-0102-462c-856f-5481eced3e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:50.477105 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2472/4975] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:50 2022] POST /identity/v3/projects => generated 361 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:50.478240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:50.637082 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2504/4976] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:50.638706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.638998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.639590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.663580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] MFA Rules not processed for user `04440378e21043b2a287e9b4c800080d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:50.665070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:50.675433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61c9e2-d700-41d1-959a-7e68de6df612 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:50.687467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:50.689906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.690321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.691228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/e2fd818ec3814085aacab4560273d77d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.693090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:update_project(project_id=e2fd818ec3814085aacab4560273d77d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:50.697350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:50.738125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:50.783373 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2505/4977] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:50.797366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:50.798321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:50.815198 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2473/4978] 10.222.0.26 () {60 vars in 1252 bytes} [Tue Sep 27 10:16:50 2022] PATCH /identity/v3/projects/e2fd818ec3814085aacab4560273d77d => generated 373 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:50.816772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:50.836810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ddb7-23f2-4905-9a36-2c7272f3e7f3 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:50.849573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:50.850980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.851676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.852171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/e2fd818ec3814085aacab4560273d77d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.882856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=e2fd818ec3814085aacab4560273d77d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:50.887366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:50.894406 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2474/4979] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:50 2022] GET /identity/v3/projects/e2fd818ec3814085aacab4560273d77d => generated 360 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:16:50.895559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:50.924299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ceec-5986-422a-942d-980f7d502db5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:50.942403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:50.944750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.945497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.946419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/e2fd818ec3814085aacab4560273d77d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:50.949622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e2fd818ec3814085aacab4560273d77d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:50.957043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:50.983761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83d0a-5550-4116-9caa-abcb8492246a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:50.995460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:50.996517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:50.996933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:50.997302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:51.007190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:51.009103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:51.020417 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2506/4980] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:51.049325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:51.100480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2fd818ec3814085aacab4560273d77d', 'request_id': 'req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.117110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2fd818ec3814085aacab4560273d77d', 'request_id': 'req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.139061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2fd818ec3814085aacab4560273d77d', 'request_id': 'req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.163053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project e2fd818ec3814085aacab4560273d77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:51.163353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.164421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.166371 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2475/4981] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:50 2022] DELETE /identity/v3/projects/e2fd818ec3814085aacab4560273d77d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:51.171307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:51.183287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a4158-baa0-46e2-967f-6a458f4708da service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:51.329713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:51.331314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:51.331764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:51.332226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:51.333129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:51.335567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:51.417250 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2507/4982] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:51 2022] POST /identity/v3/projects => generated 354 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:51.419262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:51.421096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:51.439157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6351d-c6a0-40a3-81d9-1eb9abea5896 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:51.618486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:51.619335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:51.619753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:51.620346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/5c41f37c774c48d8ac2ba2e28c96550f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:51.621867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:update_project(project_id=5c41f37c774c48d8ac2ba2e28c96550f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:51.625827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:51.688298 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2476/4983] 10.222.0.26 () {60 vars in 1252 bytes} [Tue Sep 27 10:16:51 2022] PATCH /identity/v3/projects/5c41f37c774c48d8ac2ba2e28c96550f => generated 369 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:51.691766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:51.711031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908364c1-f366-413f-a17a-985551f0deaf tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:51.722091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:51.722742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:51.723018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:51.723293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/5c41f37c774c48d8ac2ba2e28c96550f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:51.744883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:get_project(project_id=5c41f37c774c48d8ac2ba2e28c96550f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:51.747135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:51.752014 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2508/4984] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:16:51 2022] GET /identity/v3/projects/5c41f37c774c48d8ac2ba2e28c96550f => generated 356 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:51.753540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:51.783509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9f8b-ad9e-4548-9fb8-f612e835fdda tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:51.801404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:51.802081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:51.802360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:51.802663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/projects/5c41f37c774c48d8ac2ba2e28c96550f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:51.805450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5c41f37c774c48d8ac2ba2e28c96550f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:51.807860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:51.929399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c41f37c774c48d8ac2ba2e28c96550f', 'request_id': 'req-5a7157e2-83cd-4177-91e2-19297d1d3786'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.941081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c41f37c774c48d8ac2ba2e28c96550f', 'request_id': 'req-5a7157e2-83cd-4177-91e2-19297d1d3786'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.967595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c41f37c774c48d8ac2ba2e28c96550f', 'request_id': 'req-5a7157e2-83cd-4177-91e2-19297d1d3786'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.992288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] The token cache is being invalidate because project 5c41f37c774c48d8ac2ba2e28c96550f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:51.992624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.993669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:51.995639 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2477/4985] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:51 2022] DELETE /identity/v3/projects/5c41f37c774c48d8ac2ba2e28c96550f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:52.001009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:52.003103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:52.028546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ad295-6a7a-49d0-98b1-1039f2f29084 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:52.164041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:52.201589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:52.205796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:52.206452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:52.206994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains/dd29f5eab466436a84297e1827688e59` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:52.228635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Could not find domain: dd29f5eab466436a84297e1827688e59.: keystone.exception.DomainNotFound: Could not find domain: dd29f5eab466436a84297e1827688e59. Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: dd29f5eab466436a84297e1827688e59. Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:16:52.241356 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:16:52.242478 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: dd29f5eab466436a84297e1827688e59. Sep 27 10:16:52.243506 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:16:52.244575 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2509/4986] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:16:51 2022] PATCH /identity/v3/domains/dd29f5eab466436a84297e1827688e59 => generated 112 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:16:52.247366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:52.262031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7157e2-83cd-4177-91e2-19297d1d3786 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:52.300622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ceafa-c0b8-4f73-9fb7-5277b52659a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf3ec5f09a5e484295d4fba1fcdc935a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a949fc42c82b40399c24ad17a2a55665', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:52.301356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ceafa-c0b8-4f73-9fb7-5277b52659a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:52.301657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ceafa-c0b8-4f73-9fb7-5277b52659a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:52.301937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ceafa-c0b8-4f73-9fb7-5277b52659a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] PATH_INFO: `/v3/domains/8d716accb1a64005a2b6109124d90606` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:52.328100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ceafa-c0b8-4f73-9fb7-5277b52659a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-485ceafa-c0b8-4f73-9fb7-5277b52659a2 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Could not find domain: 8d716accb1a64005a2b6109124d90606.: keystone.exception.DomainNotFound: Could not find domain: 8d716accb1a64005a2b6109124d90606. Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 8d716accb1a64005a2b6109124d90606. Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:16:52.346043 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:16:52.347195 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 8d716accb1a64005a2b6109124d90606. Sep 27 10:16:52.348162 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:16:52.350742 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2478/4987] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:16:52 2022] PATCH /identity/v3/domains/8d716accb1a64005a2b6109124d90606 => generated 112 bytes in 103 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:16:52.356377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:52.358617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:52.561062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b177f-9bc6-4991-8339-6768e93a2ab5 tempest-ProjectsTestJSON-1431733017 tempest-ProjectsTestJSON-1431733017-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:52.573710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:52.574442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:52.574717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:52.575488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] PATH_INFO: `/v3/users/fb61a61f501e4532840c81629f48d689` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:52.581304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb61a61f501e4532840c81629f48d689)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:52.583442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:52.686739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb61a61f501e4532840c81629f48d689'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:52.687645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb61a61f501e4532840c81629f48d689'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:52.704269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb61a61f501e4532840c81629f48d689'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:52.720453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb61a61f501e4532840c81629f48d689'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:52.721569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb61a61f501e4532840c81629f48d689'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:52.769040 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2510/4988] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:52 2022] DELETE /identity/v3/users/fb61a61f501e4532840c81629f48d689 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:52.775465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:52.776024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:52.776129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:52.823400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:52.824934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:52.873592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:53.011023 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2479/4989] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:53.035645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:53.037726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:53.208733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5f840-2a46-4f45-800f-847af6f1d176 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:53.244094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:53.244774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:53.245060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:53.245335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:53.270031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:53.271805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:53.301402 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2511/4990] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:16:54.229948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:54.231690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:54.290220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bb423-411c-4813-af91-efeb60dd35f8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:54.298387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:54.299169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:54.299316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:54.299880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] PATH_INFO: `/v3/projects/acb5d8053c984b62aadb0f9b5e0e3df2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:54.302263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb5d8053c984b62aadb0f9b5e0e3df2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:54.308905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:54.431648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb5d8053c984b62aadb0f9b5e0e3df2', 'request_id': 'req-991cf78f-fd87-43f8-923f-52144b49ec49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.448269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb5d8053c984b62aadb0f9b5e0e3df2', 'request_id': 'req-991cf78f-fd87-43f8-923f-52144b49ec49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.490829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb5d8053c984b62aadb0f9b5e0e3df2', 'request_id': 'req-991cf78f-fd87-43f8-923f-52144b49ec49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.524074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] The token cache is being invalidate because project acb5d8053c984b62aadb0f9b5e0e3df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:54.526069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.527432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.529424 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2480/4991] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:54 2022] DELETE /identity/v3/projects/acb5d8053c984b62aadb0f9b5e0e3df2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:54.534801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:54.546713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:54.560213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e7217-cc63-4f17-a255-9dd408def3ab service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:54.721530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:54.725061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:54.725061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:54.728467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] PATH_INFO: `/v3/users/cf3ec5f09a5e484295d4fba1fcdc935a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:54.737925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3ec5f09a5e484295d4fba1fcdc935a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:54.741622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:54.877783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3ec5f09a5e484295d4fba1fcdc935a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.878606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3ec5f09a5e484295d4fba1fcdc935a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.900932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3ec5f09a5e484295d4fba1fcdc935a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.910974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3ec5f09a5e484295d4fba1fcdc935a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.912291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-49a52091-50f9-4c47-8e3a-a5cb7c1cfc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3ec5f09a5e484295d4fba1fcdc935a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:54.950828 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2512/4992] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:54 2022] DELETE /identity/v3/users/cf3ec5f09a5e484295d4fba1fcdc935a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:56.396780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:56.398881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:56.414069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cf78f-fd87-43f8-923f-52144b49ec49 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:56.531192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:56.532027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:56.533161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:56.534034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] PATH_INFO: `/v3/projects/a949fc42c82b40399c24ad17a2a55665` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:56.536885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a949fc42c82b40399c24ad17a2a55665)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:56.542790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:56.658527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:56.672762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a949fc42c82b40399c24ad17a2a55665', 'request_id': 'req-3075986c-e4ef-4730-a9fd-ad37db5cff27'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:56.687815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a949fc42c82b40399c24ad17a2a55665', 'request_id': 'req-3075986c-e4ef-4730-a9fd-ad37db5cff27'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:56.735143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a949fc42c82b40399c24ad17a2a55665', 'request_id': 'req-3075986c-e4ef-4730-a9fd-ad37db5cff27'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:56.753222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] The token cache is being invalidate because project a949fc42c82b40399c24ad17a2a55665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:16:56.753662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:56.754678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3075986c-e4ef-4730-a9fd-ad37db5cff27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:16:56.757424 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2481/4993] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:16:56 2022] DELETE /identity/v3/projects/a949fc42c82b40399c24ad17a2a55665 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:16:56.767069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d38ae-13a2-4d04-8770-bc14ab3dde3a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:56.768036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d38ae-13a2-4d04-8770-bc14ab3dde3a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:56.768488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d38ae-13a2-4d04-8770-bc14ab3dde3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:56.813377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f70d38ae-13a2-4d04-8770-bc14ab3dde3a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:56.815773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f70d38ae-13a2-4d04-8770-bc14ab3dde3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:56.954189 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2513/4994] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:56.957274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6e464-7649-491a-85d4-10503709aaa5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:56.957641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6e464-7649-491a-85d4-10503709aaa5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:56.958002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6e464-7649-491a-85d4-10503709aaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:56.990106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dcb6e464-7649-491a-85d4-10503709aaa5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:56.991537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dcb6e464-7649-491a-85d4-10503709aaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:57.002010 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2482/4995] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:57.007283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d38ae-13a2-4d04-8770-bc14ab3dde3a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:57.022772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d38ae-13a2-4d04-8770-bc14ab3dde3a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:57.030344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:57.031413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:57.031891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:57.032363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:57.033254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:16:57.033804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:57.036128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:57.068851 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2514/4996] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:16:57.071707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6e464-7649-491a-85d4-10503709aaa5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:57.106983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6e464-7649-491a-85d4-10503709aaa5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:57.114821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:57.115627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:57.115938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:57.116167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:57.116760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:57.121263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:57.198950 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2483/4997] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:57 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:57.202538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:57.216368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb4e85-f3ea-434d-ad46-66849a5b0f98 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:57.224938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:57.375880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:57.376783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:57.377238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:57.377589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:57.378544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:57.383316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:57.438087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2515/4998] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:57 2022] POST /identity/v3/users => generated 283 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:57.442965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:57.460342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928ed80-f897-4e19-8579-0bf46e7c016a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:57.488647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:57.489806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:57.490167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:57.490456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:57.496208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:57.499621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:57.534526 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2484/4999] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:57.538105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:57.554368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871d99-f39b-4103-b434-0d998e3d8abe admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:57.564853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12b470-569f-453f-a1f9-bc38f05330d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:57.565499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12b470-569f-453f-a1f9-bc38f05330d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:57.565781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12b470-569f-453f-a1f9-bc38f05330d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:57.567339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12b470-569f-453f-a1f9-bc38f05330d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:57.569500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12b470-569f-453f-a1f9-bc38f05330d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:57.571169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12b470-569f-453f-a1f9-bc38f05330d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:57.597837 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2516/5000] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:57.613116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:57.640303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffba97-0261-4d0b-a89f-453a77a03cd8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:57.657776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:57.658848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:57.659292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:57.659722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] PATH_INFO: `/v3/projects/31d5b9b67fbe41079c7cba8ae39c161e/users/5567d7d2026742b9b46bbfda36c4702a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:57.694180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d5b9b67fbe41079c7cba8ae39c161e, user_id=5567d7d2026742b9b46bbfda36c4702a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:57.697900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:57.740269 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2485/5001] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:57 2022] PUT /identity/v3/projects/31d5b9b67fbe41079c7cba8ae39c161e/users/5567d7d2026742b9b46bbfda36c4702a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:57.753709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba0219-51ab-4049-97a4-87561c4a8ecd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:57.753987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba0219-51ab-4049-97a4-87561c4a8ecd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:57.754283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba0219-51ab-4049-97a4-87561c4a8ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:57.798576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fdba0219-51ab-4049-97a4-87561c4a8ecd None None] MFA Rules not processed for user `5567d7d2026742b9b46bbfda36c4702a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:57.800930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fdba0219-51ab-4049-97a4-87561c4a8ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:57.927378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2517/5002] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:57.932526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:58.058329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6687-6483-46db-8e0d-075e414fbf6c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:58.064884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:58.065772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:58.066199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:58.066764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:58.067814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:58.070075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:58.113402 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2486/5003] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:57 2022] POST /identity/v3/projects => generated 365 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:58.115235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba0219-51ab-4049-97a4-87561c4a8ecd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:58.133789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba0219-51ab-4049-97a4-87561c4a8ecd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:58.244087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:58.244793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:58.245078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:58.245354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:58.246199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:58.252338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:58.302112 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2518/5004] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:58 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:58.306728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:58.330248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b58c1b-f5db-464f-84ac-0d31e09002b3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:58.338728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:58.339452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:58.339744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:58.340025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:58.340931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:58.342952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:58.362626 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2487/5005] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:58.362867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:58.382367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edc9ac-af69-4835-9ed6-021b7a932f2b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:58.391776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:58.393709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:58.394065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:58.396694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] PATH_INFO: `/v3/projects/ceead42ebe69410a87378451144494d2/users/6beefa908daf4834afa1c476bc26267b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:58.401167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:58.433241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceead42ebe69410a87378451144494d2, user_id=6beefa908daf4834afa1c476bc26267b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:58.438327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:58.469187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2519/5006] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:58 2022] PUT /identity/v3/projects/ceead42ebe69410a87378451144494d2/users/6beefa908daf4834afa1c476bc26267b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:58.473029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:58.501430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5c894-9289-4069-944d-f02959b4d0aa admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:58.596699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:58.685071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9628af-6682-4b4c-b749-88ff305501f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:58.685980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9628af-6682-4b4c-b749-88ff305501f4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:58.686441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9628af-6682-4b4c-b749-88ff305501f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:58.686962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9628af-6682-4b4c-b749-88ff305501f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:58.691282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9628af-6682-4b4c-b749-88ff305501f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:58.706407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9628af-6682-4b4c-b749-88ff305501f4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:58.746590 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2488/5007] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:16:58.753434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:58.767912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b09e-f67b-4005-9f46-a751da8bc98b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:58.780974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:58.781734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:58.782011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:58.782280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] PATH_INFO: `/v3/projects/ceead42ebe69410a87378451144494d2/users/6beefa908daf4834afa1c476bc26267b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:58.786568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceead42ebe69410a87378451144494d2, user_id=6beefa908daf4834afa1c476bc26267b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:58.794753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:58.839575 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2520/5008] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:58 2022] PUT /identity/v3/projects/ceead42ebe69410a87378451144494d2/users/6beefa908daf4834afa1c476bc26267b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:58.850679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d54d45-67ec-48e5-8855-44d224f1822b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:58.851282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d54d45-67ec-48e5-8855-44d224f1822b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:58.851811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d54d45-67ec-48e5-8855-44d224f1822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:58.899708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-70d54d45-67ec-48e5-8855-44d224f1822b None None] MFA Rules not processed for user `6beefa908daf4834afa1c476bc26267b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:16:58.901409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-70d54d45-67ec-48e5-8855-44d224f1822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:59.057003 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2489/5009] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:16:59.058964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.081331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c9529-1688-42ed-a3e4-4e26de907722 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.088421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.089483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.089847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.090326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.091841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.093708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.126854 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2521/5010] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/roles => generated 234 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.131387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d54d45-67ec-48e5-8855-44d224f1822b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.141806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d54d45-67ec-48e5-8855-44d224f1822b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.147105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.147882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.148212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.148544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.149418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.150467 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:59.150467 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:59.150467 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:59.151754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.172705 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2490/5011] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/roles => generated 234 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.175089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.203477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab15b-5bef-4953-b93a-76b3b509e76e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.209185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.209812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.210097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.210371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.211053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.212687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.232055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2522/5012] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/roles => generated 233 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:16:59.236263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.249476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04767ad0-22c2-46e0-9d2f-6e507584a583 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.256688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.257889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.258223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.258452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.259135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.261006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.295551 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2491/5013] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/domains => generated 277 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.299746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.310501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8a792-ad5e-4454-a774-86d0718d6699 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.376431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.377009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.377285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.377553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.378196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.380117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.429277 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2523/5014] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.432962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.443542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7428a3a-bc50-48ca-af87-5097eb92b1d1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.511953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.512594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.512920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.513116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.513816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.514828 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:59.514828 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:16:59.514828 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:16:59.516036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.543114 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2492/5015] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/groups => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.547173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.559830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ff9d5-a7ce-4fd7-840e-f8f1ad5f91be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.566953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.568009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.568700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.569078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.570068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.571860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.595638 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2524/5016] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/roles => generated 234 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.597275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.610937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d28353-62be-4466-b0f7-b865d31a7880 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.617787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.618507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.618860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.619253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.620129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.622766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.657107 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2493/5017] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/users => generated 439 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.663653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.681356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b60bc-bd38-49bf-b2ea-d90a779aa1f1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.688436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.689613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.689895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.690246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.690930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.692713 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:16:59.692713 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:16:59.692713 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:16:59.693331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.720725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.727076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:16:59.753717 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2525/5018] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:16:59 2022] POST /identity/v3/roles => generated 269 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:16:59.757471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.773357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20273416-f02e-46d2-80ec-1fbe724938a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.780074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:16:59.781386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:16:59.782052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:16:59.782603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/a508b77d6e3746a4a300d1ef437fa37d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:16:59.791889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:59.803356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=a508b77d6e3746a4a300d1ef437fa37d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:16:59.806906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:16:59.831958 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2526/5019] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:16:59 2022] PUT /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/a508b77d6e3746a4a300d1ef437fa37d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:16:59.833668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:16:59.897210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67b79-ef30-46ed-ba38-5f30727c5ee2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:16:59.919880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:00.003764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f60d-5f38-4d63-9c23-9d6729e76494 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:00.021709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.022907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.022907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.023166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.024053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Adding query filter params (scope.project.id=339c23e8cefe4ccdb98aa25939614bc0, user.id=3c6740d580674798bbd1e2c8c79e1465) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:00.024358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.027739 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:00.027739 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:00.027739 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:00.028996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.092974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.093779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.094243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.094720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] PATH_INFO: `/v3/users/5a34669b510b41eea8b10ba3c9f650ef` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.097432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a34669b510b41eea8b10ba3c9f650ef)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.101248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.173315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2527/5020] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:16:59 2022] GET /identity/v3/role_assignments?scope.project.id=339c23e8cefe4ccdb98aa25939614bc0&user.id=3c6740d580674798bbd1e2c8c79e1465&effective => generated 228 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:00.175849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:00.203825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972137a-ad2e-4986-a6ef-ed92e89880ba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:00.219667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.220529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.220864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.221148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/a508b77d6e3746a4a300d1ef437fa37d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.221730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a34669b510b41eea8b10ba3c9f650ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.222705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a34669b510b41eea8b10ba3c9f650ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.226445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=a508b77d6e3746a4a300d1ef437fa37d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.229129 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:00.229129 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:00.229129 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:00.235978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.238938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a34669b510b41eea8b10ba3c9f650ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.250787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role a508b77d6e3746a4a300d1ef437fa37d was removed from user 3c6740d580674798bbd1e2c8c79e1465 on project 339c23e8cefe4ccdb98aa25939614bc0. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:00.251098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.251893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.254017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a34669b510b41eea8b10ba3c9f650ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.281977 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2528/5021] 10.222.0.26 () {58 vars in 1467 bytes} [Tue Sep 27 10:17:00 2022] DELETE /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/a508b77d6e3746a4a300d1ef437fa37d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:00.285111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0923d4db-f9a0-41f3-8961-92a657b3742c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a34669b510b41eea8b10ba3c9f650ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.288771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:00.290547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:00.311603 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2494/5022] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:16:59 2022] DELETE /identity/v3/users/5a34669b510b41eea8b10ba3c9f650ef => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:00.316847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10203d6-69ae-4894-9cc5-20226c29b59c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.317459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10203d6-69ae-4894-9cc5-20226c29b59c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.317734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10203d6-69ae-4894-9cc5-20226c29b59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.356605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d10203d6-69ae-4894-9cc5-20226c29b59c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:00.358181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d10203d6-69ae-4894-9cc5-20226c29b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:00.381322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de809a9d-4f97-4bd4-b6d8-5ebf13a1977f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:00.401634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:00.486173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.488003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.489397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.489519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/a508b77d6e3746a4a300d1ef437fa37d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.490983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=a508b77d6e3746a4a300d1ef437fa37d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.492230 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:00.492230 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:00.492230 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:00.493408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.511180 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2495/5023] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:00.517279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:00.557045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role a508b77d6e3746a4a300d1ef437fa37d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:00.557359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.558093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:00.566429 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2529/5024] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:00 2022] DELETE /identity/v3/roles/a508b77d6e3746a4a300d1ef437fa37d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:00.567462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10203d6-69ae-4894-9cc5-20226c29b59c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:00.585082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10203d6-69ae-4894-9cc5-20226c29b59c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:00.587647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:00.590183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:00.687758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.689042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.689891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.689891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/7722957c891e43498a31f4ce9abf17f0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.694276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=7722957c891e43498a31f4ce9abf17f0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.698635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.712173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aaee63-6660-4c01-86b8-37eb39797c34 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:00.718137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.719106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.719805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.720322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.722486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:00.724523 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2496/5025] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:00 2022] PUT /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/7722957c891e43498a31f4ce9abf17f0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:00.734817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:00.737117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:00.814838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abc6bf-aa1c-41bd-a775-d49984c12b05 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:00.821116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.821604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.822293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.822578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.822854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.823759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.824563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.826140 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:00.826140 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:00.826140 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:00.827020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.838726 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2530/5026] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:00.860509 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2497/5027] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:00 2022] PUT /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 519 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:00.861870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:00.873169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb00d3-ff26-448b-8f79-2a3a7640ba0d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:00.931907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:00.932554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:00.932830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:00.933108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:00.933811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Adding query filter params (scope.project.id=339c23e8cefe4ccdb98aa25939614bc0, user.id=3c6740d580674798bbd1e2c8c79e1465) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:00.934118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:00.936036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:00.976602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:00.993157 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2531/5028] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:17:00 2022] GET /identity/v3/role_assignments?scope.project.id=339c23e8cefe4ccdb98aa25939614bc0&user.id=3c6740d580674798bbd1e2c8c79e1465&effective => generated 1061 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:00.997298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.008157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec405b-f5fe-4556-9a4a-06ac3d2ad451 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.018768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.019661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.020225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.020764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.023028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.024189 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:01.024189 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:01.024189 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:01.025552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.040835 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2498/5029] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:00 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:01.044842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.057956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1738b8-5936-451d-af6f-6b8926da4d6e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.136462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.137266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.137625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.137970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.138770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Adding query filter params (scope.project.id=339c23e8cefe4ccdb98aa25939614bc0, user.id=3c6740d580674798bbd1e2c8c79e1465) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:01.139147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.141126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.194453 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2532/5030] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:17:01 2022] GET /identity/v3/role_assignments?scope.project.id=339c23e8cefe4ccdb98aa25939614bc0&user.id=3c6740d580674798bbd1e2c8c79e1465&effective => generated 577 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:01.198382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.211235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001152cc-8620-4600-84aa-63a68393b076 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.216602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.217474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.217708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.218006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.220213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.222178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189.: keystone.exception.ImpliedRoleNotFound: 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189. Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = query.one() Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._iter().one() Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._only_one_row( Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:01.263512 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 27 10:17:01.265118 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 27 10:17:01.266075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 27 10:17:01.266075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 27 10:17:01.266075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 27 10:17:01.266075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189. Sep 27 10:17:01.266075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:01.271901 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2499/5031] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:01 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 137 bytes in 74 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:17:01.272915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.289160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1b531-945e-410b-a213-0ad7623548c4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.297523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.298958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.299365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.300249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/7722957c891e43498a31f4ce9abf17f0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.316535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=7722957c891e43498a31f4ce9abf17f0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.323373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.338319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role 7722957c891e43498a31f4ce9abf17f0 was removed from user 3c6740d580674798bbd1e2c8c79e1465 on project 339c23e8cefe4ccdb98aa25939614bc0. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:01.338892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:01.340231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:01.378130 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2533/5032] 10.222.0.26 () {58 vars in 1467 bytes} [Tue Sep 27 10:17:01 2022] DELETE /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/7722957c891e43498a31f4ce9abf17f0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:01.385010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.389052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:01.421679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e7fbb-2c28-4cb7-8cfc-eabcd84431e4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.517514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:01.585891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.587588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.587875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.588160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.588898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.589945 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:01.589945 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:01.589945 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:01.591587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.626937 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2500/5033] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:01 2022] POST /identity/v3/roles => generated 271 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:01.631808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.647111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2174ac-f42c-42a8-b56d-44f4cc57827a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.677619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.678666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.679390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.679842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.680690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Adding query filter params (domain_id=faa766ce7d784ab3b5a3b882841e8f49) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:01.681038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.682137 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:01.682137 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:01.682137 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:01.683540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.694690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2534/5034] 10.222.0.26 () {58 vars in 1207 bytes} [Tue Sep 27 10:17:01 2022] GET /identity/v3/roles?domain_id=faa766ce7d784ab3b5a3b882841e8f49 => generated 408 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:01.698755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.725375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fa355-61f0-43f3-9939-a375b9ad82a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.731684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.732337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.732616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.732930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/f769690424e04ee7b2175b9ba548e679` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.735951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=f769690424e04ee7b2175b9ba548e679)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.738597 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:01.738597 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:01.738597 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:01.739560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.819631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role f769690424e04ee7b2175b9ba548e679 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:01.820076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:01.821698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:01.837878 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2501/5035] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:01 2022] DELETE /identity/v3/roles/f769690424e04ee7b2175b9ba548e679 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:01.842560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.846589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:01.887360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:01.889488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865363-070e-458d-9be4-a003e39c77f0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:01.891768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:01.984189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:01.984935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:01.985246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:01.985516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:01.986183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Adding query filter params (domain_id=faa766ce7d784ab3b5a3b882841e8f49) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:01.986484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:01.988108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:01.998760 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2535/5036] 10.222.0.26 () {58 vars in 1207 bytes} [Tue Sep 27 10:17:01 2022] GET /identity/v3/roles?domain_id=faa766ce7d784ab3b5a3b882841e8f49 => generated 148 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:01.999853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.016313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6b64a-3848-4e19-9c94-44f6cdbe542c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.024575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.025341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.025625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.025896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/f769690424e04ee7b2175b9ba548e679` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.041215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Could not find role: f769690424e04ee7b2175b9ba548e679.: keystone.exception.RoleNotFound: Could not find role: f769690424e04ee7b2175b9ba548e679. Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Sep 27 10:17:02.047433 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:17:02.048504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: f769690424e04ee7b2175b9ba548e679. Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:02.049499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2536/5037] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:01 2022] DELETE /identity/v3/roles/f769690424e04ee7b2175b9ba548e679 => generated 110 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:17:02.052859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dd4b4-f797-4a5e-8634-394f294cb95c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.065368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.067397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.070101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:02.071157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.071445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.071722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] PATH_INFO: `/v3/projects/466f9ae4f69f4ba6a9484b5a93bb6284` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.073081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466f9ae4f69f4ba6a9484b5a93bb6284)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.077876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfef01b-c2d7-4a36-8d11-26738e31a092 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.079815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.087821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.089348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.089701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.090092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.101316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=faa766ce7d784ab3b5a3b882841e8f49, group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.104371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.133484 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2537/5038] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:02 2022] PUT /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:02.134699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.178423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466f9ae4f69f4ba6a9484b5a93bb6284', 'request_id': 'req-685fd49d-e23d-4d18-8ebe-6ce7033ee987'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:02.186578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466f9ae4f69f4ba6a9484b5a93bb6284', 'request_id': 'req-685fd49d-e23d-4d18-8ebe-6ce7033ee987'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:02.198381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466f9ae4f69f4ba6a9484b5a93bb6284', 'request_id': 'req-685fd49d-e23d-4d18-8ebe-6ce7033ee987'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:02.218523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] The token cache is being invalidate because project 466f9ae4f69f4ba6a9484b5a93bb6284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:02.218983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:02.220268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-685fd49d-e23d-4d18-8ebe-6ce7033ee987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:02.227610 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2502/5039] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:01 2022] DELETE /identity/v3/projects/466f9ae4f69f4ba6a9484b5a93bb6284 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:02.234421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ee8fe-16dc-486f-b6b4-78e7bcaefb03 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.234761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ee8fe-16dc-486f-b6b4-78e7bcaefb03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.235336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ee8fe-16dc-486f-b6b4-78e7bcaefb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.255459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816829b-c687-44da-8fb2-686128b896b1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.267525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-814ee8fe-16dc-486f-b6b4-78e7bcaefb03 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:02.269197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-814ee8fe-16dc-486f-b6b4-78e7bcaefb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:02.295044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:02.360023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.360738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.361020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.361294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.362938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=faa766ce7d784ab3b5a3b882841e8f49, group_id=2d07ce47431146d4abbca7fbbf44b9b2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.363951 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:02.363951 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:02.363951 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:02.366021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.372818 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2503/5040] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:02.375183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bc96f-0de6-46a6-9351-8158b238471f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.375407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bc96f-0de6-46a6-9351-8158b238471f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.375676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bc96f-0de6-46a6-9351-8158b238471f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.381800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2538/5041] 10.222.0.26 () {58 vars in 1365 bytes} [Tue Sep 27 10:17:02 2022] GET /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles => generated 409 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:02.385228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.387518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:02.404519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df670-4d51-4fea-848d-f98e7a34d3f7 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.406440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6b7bc96f-0de6-46a6-9351-8158b238471f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:02.408351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6b7bc96f-0de6-46a6-9351-8158b238471f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:02.411330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.412395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.413272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.413628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.416168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=faa766ce7d784ab3b5a3b882841e8f49, group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.417467 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:02.417467 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:02.417467 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:02.417886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2504/5042] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:02.422691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.425544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bc96f-0de6-46a6-9351-8158b238471f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.440569 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2539/5043] 10.222.0.26 () {58 vars in 1465 bytes} [Tue Sep 27 10:17:02 2022] HEAD /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:02.441512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.452967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bc96f-0de6-46a6-9351-8158b238471f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.452967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.452967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.452967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.453303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.453303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:02.453303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.459078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.463617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27415694-c4ca-4597-95fc-8552af596f48 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.468683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.469385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.469677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.469955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.470092 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2505/5044] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:02.472141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=faa766ce7d784ab3b5a3b882841e8f49, group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.474500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.474984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.481252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role d66df6111bc44bf7b60729fa93422584 was removed from group 2d07ce47431146d4abbca7fbbf44b9b2 on domain faa766ce7d784ab3b5a3b882841e8f49. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:02.481612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:02.482386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:02.491357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b7e7f-a1d0-4ecc-892a-a41da763f496 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.496819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.497438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.497712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.498004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.498651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.500539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.503948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2540/5045] 10.222.0.26 () {58 vars in 1467 bytes} [Tue Sep 27 10:17:02 2022] DELETE /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:02.509850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.511796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:02.548712 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2506/5046] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:02 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:02.557066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.558451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:02.595702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:02.600793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d81f-a155-49b1-be55-6d77ae3f5efc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.638554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c5f3c-c783-489b-991a-d390376f5aad admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.644848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.645521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.645890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.646197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.647008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.651632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.680521 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2507/5047] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:02 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:02.680940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.680940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.680940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.681244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.682219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.682849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.684925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.694570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fbc44-6a96-4212-9643-a99f9e6cae33 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.702495 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2541/5048] 10.222.0.26 () {60 vars in 1486 bytes} [Tue Sep 27 10:17:02 2022] PUT /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:02.707151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.777028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.778121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.778494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.778965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.779812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.781673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.788352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ed93f-91ed-4742-8917-3230a9a16aef tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.790355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:02.794394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.795103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.795385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.795659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.797352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_grants(project_id=339c23e8cefe4ccdb98aa25939614bc0, group_id=2d07ce47431146d4abbca7fbbf44b9b2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.800082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.801329 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2508/5049] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:02 2022] GET /identity/v3/roles => generated 2319 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:02.805705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.815463 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2542/5050] 10.222.0.26 () {58 vars in 1368 bytes} [Tue Sep 27 10:17:02 2022] GET /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles => generated 410 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:02.817589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4936c-44d3-4038-a0b3-b2687a4dcca3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.821357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.823898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.825621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.825966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.826310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.827199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.829593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.832374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2991-643f-4989-bb80-9b63d51278cd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.838073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9aa2dd-5bf4-4f69-a102-8b8a72369c8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.838908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9aa2dd-5bf4-4f69-a102-8b8a72369c8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.839189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9aa2dd-5bf4-4f69-a102-8b8a72369c8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.839459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9aa2dd-5bf4-4f69-a102-8b8a72369c8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/groups/2d07ce47431146d4abbca7fbbf44b9b2/users/3c6740d580674798bbd1e2c8c79e1465` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.842868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9aa2dd-5bf4-4f69-a102-8b8a72369c8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=2d07ce47431146d4abbca7fbbf44b9b2, user_id=3c6740d580674798bbd1e2c8c79e1465)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.843944 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-1f9aa2dd-5bf4-4f69-a102-8b8a72369c8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:02.843944 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:02.843944 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:02.845098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9aa2dd-5bf4-4f69-a102-8b8a72369c8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.846673 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2509/5051] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:02 2022] GET /identity/v3/roles => generated 2319 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:02.850995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:02.862219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c55f0-65fa-4fa2-9c3e-d6e599c0caf6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:02.868235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576657ba-dd36-42d6-a2a0-3a72f81e3b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:02.868953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576657ba-dd36-42d6-a2a0-3a72f81e3b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.869233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576657ba-dd36-42d6-a2a0-3a72f81e3b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.869513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576657ba-dd36-42d6-a2a0-3a72f81e3b3c admin admin] PATH_INFO: `/v3/projects/dbfb9f03e6de43a6bd5e7a6641ac8088/users/1d92f7d2ee2d47beaa4f5d43ac40cd82/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.890175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576657ba-dd36-42d6-a2a0-3a72f81e3b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbfb9f03e6de43a6bd5e7a6641ac8088, user_id=1d92f7d2ee2d47beaa4f5d43ac40cd82, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:02.892187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2543/5052] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:17:02 2022] PUT /identity/v3/groups/2d07ce47431146d4abbca7fbbf44b9b2/users/3c6740d580674798bbd1e2c8c79e1465 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:02.893657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576657ba-dd36-42d6-a2a0-3a72f81e3b3c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:02.897756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e50a2-ace9-40e6-a98a-d3336bf3b302 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.898126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e50a2-ace9-40e6-a98a-d3336bf3b302 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.898612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e50a2-ace9-40e6-a98a-d3336bf3b302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.934823 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2510/5053] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:02 2022] PUT /identity/v3/projects/dbfb9f03e6de43a6bd5e7a6641ac8088/users/1d92f7d2ee2d47beaa4f5d43ac40cd82/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:02.947189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49ee4-72f0-4744-b6a0-d609ba22cfd9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:02.947472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49ee4-72f0-4744-b6a0-d609ba22cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:02.947860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49ee4-72f0-4744-b6a0-d609ba22cfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:02.951598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b83e50a2-ace9-40e6-a98a-d3336bf3b302 None None] MFA Rules not processed for user `3c6740d580674798bbd1e2c8c79e1465`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:02.952917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b83e50a2-ace9-40e6-a98a-d3336bf3b302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:02.987302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-13b49ee4-72f0-4744-b6a0-d609ba22cfd9 None None] MFA Rules not processed for user `1d92f7d2ee2d47beaa4f5d43ac40cd82`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:02.988624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-13b49ee4-72f0-4744-b6a0-d609ba22cfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:03.038284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:03.108706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2544/5054] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:03.114906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e50a2-ace9-40e6-a98a-d3336bf3b302 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.125763 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2511/5055] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:03.127186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b49ee4-72f0-4744-b6a0-d609ba22cfd9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.228111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e50a2-ace9-40e6-a98a-d3336bf3b302 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.241852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.241852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.242067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.244750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.248788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:check_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.251529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b49ee4-72f0-4744-b6a0-d609ba22cfd9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.252734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.257595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.258664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.258955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.259780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.260933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.263068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.265156 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2545/5056] 10.222.0.26 () {58 vars in 1468 bytes} [Tue Sep 27 10:17:03 2022] HEAD /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:03.266104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.279187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fe4c1-8071-464d-aa2b-a0e17540ddb1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.285745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.286476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.286771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.287317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.289606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.292589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.299426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role d66df6111bc44bf7b60729fa93422584 was removed from group 2d07ce47431146d4abbca7fbbf44b9b2 on project 339c23e8cefe4ccdb98aa25939614bc0. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:03.299715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:03.300403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:03.301066 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2512/5057] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:03 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:03.305365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.308455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:03.324078 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2546/5058] 10.222.0.26 () {58 vars in 1470 bytes} [Tue Sep 27 10:17:03 2022] DELETE /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:03.333793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.335390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:03.373563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:03.405018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a26346-3d2a-46d5-aa38-63091a1bac0e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.418131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca613c1-8a34-4209-aafa-6f523338bb57 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.423548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.425493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.425493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.425493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/groups/2d07ce47431146d4abbca7fbbf44b9b2/users/3c6740d580674798bbd1e2c8c79e1465` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.427186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=2d07ce47431146d4abbca7fbbf44b9b2, user_id=3c6740d580674798bbd1e2c8c79e1465)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.428720 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:03.428720 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:03.428720 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:03.430565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.447742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '3c6740d580674798bbd1e2c8c79e1465'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:03.476571 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2547/5059] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:17:03 2022] DELETE /identity/v3/groups/2d07ce47431146d4abbca7fbbf44b9b2/users/3c6740d580674798bbd1e2c8c79e1465 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:03.496070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.500261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.500973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.501545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.501821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.502468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.504554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.540704 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2513/5060] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:03 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:03.542724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:03.545490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.559882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3413b-bdf7-4e90-94a9-12d121612360 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.581767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3a88-80dd-4b68-846c-29a9a28b577c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.588262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.589449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.589723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.589999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.592487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.593562 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:03.593562 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:03.593562 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:03.594463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.621054 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2548/5061] 10.222.0.26 () {60 vars in 1381 bytes} [Tue Sep 27 10:17:03 2022] PUT /identity/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:03.623117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.642863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0636a6-e2d3-407c-8f6d-c703fa7468b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.648433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.649137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.650039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.650039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.650912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=2d07ce47431146d4abbca7fbbf44b9b2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.651925 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:03.651925 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:03.651925 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:03.652854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.668692 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2549/5062] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:17:03 2022] GET /identity/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles => generated 375 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:03.672588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.675525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.676189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.676869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.677208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.678103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.680166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.685884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9abceb-df87-4ec7-a0b5-6c50c4b5db96 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.691708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.692922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.693433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.693945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.696528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.697880 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:03.697880 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:03.697880 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:03.699271 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2514/5063] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:03 2022] GET /identity/v3/roles => generated 2319 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:03.699664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.700228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.709716 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2550/5064] 10.222.0.26 () {58 vars in 1363 bytes} [Tue Sep 27 10:17:03 2022] HEAD /identity/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:03.714166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.717251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b270b-a4c3-41b6-a3b5-a8b4ec288f45 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.722917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.724929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.725274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.725726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] PATH_INFO: `/v3/projects/954717fecc8142178eb36ceb49b864ee/users/c7b28035ce8c46969624d1d390741807/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.729684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b68a7-d03e-4c5d-a42e-3690ec794e17 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.735924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.736892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.737374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.737887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.740461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=2d07ce47431146d4abbca7fbbf44b9b2, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.742221 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:03.742221 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:03.742221 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:03.743736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.744208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=954717fecc8142178eb36ceb49b864ee, user_id=c7b28035ce8c46969624d1d390741807, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.746905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.757313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2551/5065] 10.222.0.26 () {58 vars in 1365 bytes} [Tue Sep 27 10:17:03 2022] DELETE /identity/v3/system/groups/2d07ce47431146d4abbca7fbbf44b9b2/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:03.763034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.766270 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2515/5066] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:03 2022] PUT /identity/v3/projects/954717fecc8142178eb36ceb49b864ee/users/c7b28035ce8c46969624d1d390741807/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:03.770357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.775541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f04f1-de3e-4738-89ae-efd80e9e0727 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.830982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:03.862711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd943d-4a73-45c9-a2d2-7bb4cbb88800 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.871860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.872874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.873166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.873525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.874699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.875453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.875728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.876193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.876742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=faa766ce7d784ab3b5a3b882841e8f49, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.876952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.878846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.899039 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2516/5067] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:03 2022] GET /identity/v3/roles => generated 2319 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:03.900153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.903337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.917789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9b52-c6cd-482e-b368-7d6f6cd189cc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:03.923950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5380c-6752-4c0c-b2e1-d2f33db71b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:03.925297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5380c-6752-4c0c-b2e1-d2f33db71b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:03.925568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5380c-6752-4c0c-b2e1-d2f33db71b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:03.925843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5380c-6752-4c0c-b2e1-d2f33db71b6b admin admin] PATH_INFO: `/v3/projects/954717fecc8142178eb36ceb49b864ee/users/c7b28035ce8c46969624d1d390741807/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:03.929332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5380c-6752-4c0c-b2e1-d2f33db71b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=954717fecc8142178eb36ceb49b864ee, user_id=c7b28035ce8c46969624d1d390741807, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:03.932704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5380c-6752-4c0c-b2e1-d2f33db71b6b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:03.942694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2552/5068] 10.222.0.26 () {60 vars in 1480 bytes} [Tue Sep 27 10:17:03 2022] PUT /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:03.947459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:03.949476 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2517/5069] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:03 2022] PUT /identity/v3/projects/954717fecc8142178eb36ceb49b864ee/users/c7b28035ce8c46969624d1d390741807/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:04.071450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7b25-ae41-49b8-b75a-9e3ce9c7cb21 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.072241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bc094-8a38-4035-95e5-88fd289de559 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.072527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bc094-8a38-4035-95e5-88fd289de559 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.072839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bc094-8a38-4035-95e5-88fd289de559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.115691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8d6bc094-8a38-4035-95e5-88fd289de559 None None] MFA Rules not processed for user `c7b28035ce8c46969624d1d390741807`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:04.118435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8d6bc094-8a38-4035-95e5-88fd289de559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:04.120067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:04.159169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.160102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.160742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.161073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.163698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=faa766ce7d784ab3b5a3b882841e8f49, user_id=3c6740d580674798bbd1e2c8c79e1465)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.169308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.188361 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2553/5070] 10.222.0.26 () {58 vars in 1362 bytes} [Tue Sep 27 10:17:03 2022] GET /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles => generated 408 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:04.197826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.211223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3f906-9790-42e3-bc8a-4a277377cdd8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.221043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.221987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.222272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.222546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.228030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=faa766ce7d784ab3b5a3b882841e8f49, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.231250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.242873 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2518/5071] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:04.243051 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2554/5072] 10.222.0.26 () {58 vars in 1462 bytes} [Tue Sep 27 10:17:04 2022] HEAD /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:04.245292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bc094-8a38-4035-95e5-88fd289de559 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.261172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.262250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bc094-8a38-4035-95e5-88fd289de559 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.263359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:04.268707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.270501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.270947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.271640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.274648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=faa766ce7d784ab3b5a3b882841e8f49, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.276908 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:04.276908 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:04.276908 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:04.279689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.289719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role d66df6111bc44bf7b60729fa93422584 was removed from user 3c6740d580674798bbd1e2c8c79e1465 on domain faa766ce7d784ab3b5a3b882841e8f49. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:04.289961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:04.290921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:04.316528 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2519/5073] 10.222.0.26 () {58 vars in 1464 bytes} [Tue Sep 27 10:17:04 2022] DELETE /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:04.322695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.324966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:04.338818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf2cdd-e1d3-47cc-9318-5a2ed18edfd9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.392393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.393418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.393418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.393625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.395677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:04.466073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2308c9d-2215-462a-a3f5-9f86b5682cee tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.472474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.474655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.477395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.478963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.479461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.479995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.485300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.487343 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2555/5074] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:04.490929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.515194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2520/5075] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:04 2022] PUT /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:04.523228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.545162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684059fc-8e70-4b41-9047-1bc8204f6f0a service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.586588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:04.636647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.639170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.639705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.640367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.643277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_grants(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.649506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.670655 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2556/5076] 10.222.0.26 () {58 vars in 1365 bytes} [Tue Sep 27 10:17:04 2022] GET /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles => generated 409 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:04.674506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.686843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732111d4-8828-46b6-8de1-fbb473da4d07 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.694086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.695489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.695956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.696428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.700070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:check_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.702692 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:04.702692 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:04.702692 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:04.707568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.718182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2521/5077] 10.222.0.26 () {58 vars in 1465 bytes} [Tue Sep 27 10:17:04 2022] HEAD /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:04.724143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.737687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03fb70-3b3e-4cc0-ad4f-9706bc064e01 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.744122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.745565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.746055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.746502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.769894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=339c23e8cefe4ccdb98aa25939614bc0, user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.774202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.782253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role d66df6111bc44bf7b60729fa93422584 was removed from user 3c6740d580674798bbd1e2c8c79e1465 on project 339c23e8cefe4ccdb98aa25939614bc0. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:04.782668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:04.785180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:04.806759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2557/5078] 10.222.0.26 () {58 vars in 1467 bytes} [Tue Sep 27 10:17:04 2022] DELETE /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:04.812497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.814405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:04.828709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795213c-ccd9-4db1-875a-b634cfa0dc40 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.850806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:04.900271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.901011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.901618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.902020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.904091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.905401 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:04.905401 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:04.905401 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:04.906162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.915605 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2522/5079] 10.222.0.26 () {60 vars in 1378 bytes} [Tue Sep 27 10:17:04 2022] PUT /identity/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:04.919815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.931144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8d980-efd6-4c56-9517-49ed025cec20 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.936068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.936865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.937243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.937615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.939347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=3c6740d580674798bbd1e2c8c79e1465)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.940698 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:04.940698 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:04.940698 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:04.941619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:04.960811 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2558/5080] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:17:04 2022] GET /identity/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles => generated 374 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:04.963093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:04.981174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad004d4-fd02-47c3-99af-bb018fb5ec77 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:04.983913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:04.984742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:04.985059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:04.985872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:04.988802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:04.990062 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:04.990062 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:04.990062 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:04.991093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.003878 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2523/5081] 10.222.0.26 () {58 vars in 1360 bytes} [Tue Sep 27 10:17:04 2022] HEAD /identity/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:05.009532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.021969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0171-270a-41d0-9350-0e99dde9c38f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.027472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.028282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.028711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.029058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.031037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=3c6740d580674798bbd1e2c8c79e1465, role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.032782 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:05.032782 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:05.032782 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:05.033306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.043071 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2559/5082] 10.222.0.26 () {58 vars in 1362 bytes} [Tue Sep 27 10:17:05 2022] DELETE /identity/v3/system/users/3c6740d580674798bbd1e2c8c79e1465/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:05.048689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.061701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c6618-0b4f-40c9-aa7d-bf7cd0da32ea tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.067086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.067270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.067807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.067936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.069379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.070431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.092485 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2524/5083] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:17:05 2022] POST /identity/v3/roles => generated 270 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:05.096223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.107880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50509e-3e2d-47f0-af32-3e40b9d25be9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.114094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.114749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.115563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.115863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.116649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.118491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.164905 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2560/5084] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:17:05 2022] POST /identity/v3/roles => generated 268 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:05.166163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.184784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b4f0e-c22d-4df4-9ca2-1b3d18dea5c2 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.212450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.213203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.213487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.213758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/762a029b27864e07b81f72af07f5ff00` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.219822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=fb043d638e1444fbb146a7e97ff1f2fe, implied_role_id=762a029b27864e07b81f72af07f5ff00)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.221787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.278997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2525/5085] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:05 2022] PUT /identity/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/762a029b27864e07b81f72af07f5ff00 => generated 529 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:05.282420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.300126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad8b5-c81a-4aba-bad8-8893ecfd5e9f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.407851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.408497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.408817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.409101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.409797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.411513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.447640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.449857 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2561/5086] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:17:05 2022] POST /identity/v3/domains => generated 276 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:05.459640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3a7f5-698b-4673-8347-8e3291534c2c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.569496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.570461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.570825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.571108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.571812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.573614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.604156 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2526/5087] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:17:05 2022] POST /identity/v3/roles => generated 270 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:05.604790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.626773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48864d-b0d7-4c54-95dd-28915198bb73 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.634687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.635532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.635946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.636247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/e4e9d0f1a160460ab99d438ffab41438` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.638868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=fb043d638e1444fbb146a7e97ff1f2fe, implied_role_id=e4e9d0f1a160460ab99d438ffab41438)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.639901 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:05.639901 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:05.639901 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:05.640840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.694328 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2562/5088] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:05 2022] PUT /identity/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/e4e9d0f1a160460ab99d438ffab41438 => generated 531 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:05.698720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.711453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a339d5-c863-4d94-b6ad-16a3b9d6a0bb tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.843710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:05.845351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:05.845646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:05.845929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:05.848145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=fb043d638e1444fbb146a7e97ff1f2fe, implied_role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:05.849942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:05.864619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:05.905780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:05.907797 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2527/5089] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:05 2022] PUT /identity/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/d66df6111bc44bf7b60729fa93422584 => generated 525 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:05.922050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94cb12-5014-4858-9481-00e4bd8390be tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:05.968253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:06.012438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:06.013377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:06.013928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:06.014515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/d66df6111bc44bf7b60729fa93422584/implies/fb043d638e1444fbb146a7e97ff1f2fe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:06.016683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=d66df6111bc44bf7b60729fa93422584, implied_role_id=fb043d638e1444fbb146a7e97ff1f2fe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:06.020797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] fb043d638e1444fbb146a7e97ff1f2fe cannot be an implied roles.: keystone.exception.InvalidImpliedRole: fb043d638e1444fbb146a7e97ff1f2fe cannot be an implied roles. Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Sep 27 10:17:06.057207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Sep 27 10:17:06.058410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:17:06.058410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:17:06.058410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Sep 27 10:17:06.058410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Sep 27 10:17:06.058410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: fb043d638e1444fbb146a7e97ff1f2fe cannot be an implied roles. Sep 27 10:17:06.058410 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:06.062578 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2563/5090] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:05 2022] PUT /identity/v3/roles/d66df6111bc44bf7b60729fa93422584/implies/fb043d638e1444fbb146a7e97ff1f2fe => generated 116 bytes in 159 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:17:06.066125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:06.087711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c640b-b64a-45fc-9c51-722a2590a9e9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:06.095421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:06.096429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:06.096683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:06.097071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:06.106216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=fb043d638e1444fbb146a7e97ff1f2fe, implied_role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:06.108007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:06.121460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2528/5091] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:06 2022] DELETE /identity/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:06.125952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:06.143481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae73fa-98c1-423b-9642-4e5c2ac95ced tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:06.227691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:06.228616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:06.229288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:06.229564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/e4e9d0f1a160460ab99d438ffab41438` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:06.231215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=fb043d638e1444fbb146a7e97ff1f2fe, implied_role_id=e4e9d0f1a160460ab99d438ffab41438)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:06.232233 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:06.232233 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:06.232233 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:06.233785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:06.249574 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2564/5092] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:06 2022] DELETE /identity/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/e4e9d0f1a160460ab99d438ffab41438 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:06.253611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:06.267691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c1fca-38bc-40d2-ade6-32a39e5528a4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:06.362061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:06.362957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:06.363819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:06.364106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/e4e9d0f1a160460ab99d438ffab41438` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:06.365703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=e4e9d0f1a160460ab99d438ffab41438)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:06.368169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:06.437445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role e4e9d0f1a160460ab99d438ffab41438 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:06.438127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.439835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.443279 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2529/5093] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:06 2022] DELETE /identity/v3/roles/e4e9d0f1a160460ab99d438ffab41438 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:06.447431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:06.449618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:06.471464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d6154-bd90-4e82-b95e-3131f6dfcca8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:06.623239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:06.624200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:06.624862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:06.625139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/057641dce4c748b1ad752a91bc31b724` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:06.647190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=057641dce4c748b1ad752a91bc31b724)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:06.651252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:06.697113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '057641dce4c748b1ad752a91bc31b724'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.697914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '057641dce4c748b1ad752a91bc31b724'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.739813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '057641dce4c748b1ad752a91bc31b724'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.740775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '057641dce4c748b1ad752a91bc31b724'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.744490 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2565/5094] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:17:06 2022] PATCH /identity/v3/domains/057641dce4c748b1ad752a91bc31b724 => generated 277 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:06.748747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:06.770589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de4c62-3817-4ccd-bacc-66174426782e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:06.884562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:06.885562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:06.885999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:06.886486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/057641dce4c748b1ad752a91bc31b724` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:06.900958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=057641dce4c748b1ad752a91bc31b724)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:06.903178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:06.933324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '057641dce4c748b1ad752a91bc31b724'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.937788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '057641dce4c748b1ad752a91bc31b724'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:06.956953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:07.022833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057641dce4c748b1ad752a91bc31b724', 'request_id': 'req-92d09656-2c5f-4455-b9b1-33892817c5da'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.037092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057641dce4c748b1ad752a91bc31b724', 'request_id': 'req-92d09656-2c5f-4455-b9b1-33892817c5da'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.048792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057641dce4c748b1ad752a91bc31b724', 'request_id': 'req-92d09656-2c5f-4455-b9b1-33892817c5da'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.073380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] The token cache is being invalidate because project 057641dce4c748b1ad752a91bc31b724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:07.073960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.074832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.087264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '057641dce4c748b1ad752a91bc31b724', 'request_id': 'req-92d09656-2c5f-4455-b9b1-33892817c5da'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.088218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '057641dce4c748b1ad752a91bc31b724', 'request_id': 'req-92d09656-2c5f-4455-b9b1-33892817c5da'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.094554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '057641dce4c748b1ad752a91bc31b724', 'request_id': 'req-92d09656-2c5f-4455-b9b1-33892817c5da'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.103015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2530/5095] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:17:06 2022] DELETE /identity/v3/domains/057641dce4c748b1ad752a91bc31b724 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:07.107714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:07.109420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:07.126459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53a751-fa97-4db4-9b33-d0e87e6da3a1 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:07.147936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:07.241398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:07.242322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:07.243560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:07.243940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/762a029b27864e07b81f72af07f5ff00` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:07.247586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=fb043d638e1444fbb146a7e97ff1f2fe, implied_role_id=762a029b27864e07b81f72af07f5ff00)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:07.250651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:07.278636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:07.282449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:07.289612 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2566/5096] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:07 2022] DELETE /identity/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe/implies/762a029b27864e07b81f72af07f5ff00 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:07.301234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d09656-2c5f-4455-b9b1-33892817c5da tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:07.388134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:07.389577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:07.389577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:07.389577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/762a029b27864e07b81f72af07f5ff00` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:07.391165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=762a029b27864e07b81f72af07f5ff00)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:07.392770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:07.446825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role 762a029b27864e07b81f72af07f5ff00 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:07.447347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.448025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.450202 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2531/5097] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:07 2022] DELETE /identity/v3/roles/762a029b27864e07b81f72af07f5ff00 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:07.454580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:07.457001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:07.471166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33bfb-2d85-4c8d-832c-a2cbd4bf2e22 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:07.559853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:07.560924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:07.561273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:07.561609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:07.563535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=fb043d638e1444fbb146a7e97ff1f2fe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:07.571074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:07.660847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role fb043d638e1444fbb146a7e97ff1f2fe has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:07.662200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.663708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:07.672538 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2567/5098] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:07 2022] DELETE /identity/v3/roles/fb043d638e1444fbb146a7e97ff1f2fe => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:07.674091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:07.686722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e01d4-4d80-4c21-8faa-fd42bcf034e0 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:07.832799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:07.862655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:07.863263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:07.863549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:07.863829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:07.867121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:07.868316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:07.946695 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2532/5099] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:07 2022] PUT /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 519 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:07.947526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:07.949356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:07.985013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82e23f-fc09-499c-bafd-ef01dabaa63b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.103650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.104511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.104925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.105330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.107152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.109261 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:08.109261 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:08.109261 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:08.111804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.113113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.118030 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:08.118030 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:08.118030 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:08.120989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.132019 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2568/5100] 10.222.0.26 () {58 vars in 1345 bytes} [Tue Sep 27 10:17:07 2022] HEAD /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:08.135935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.148504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bfc4c-ee07-4736-a952-4e51087167b5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.154941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.155667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.155956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.156314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.157933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.158957 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:08.158957 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:08.158957 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:08.163597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.172206 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2533/5101] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:17:08 2022] GET /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 519 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:08.172864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.184168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ff349-4014-47ac-a874-bb979fca37aa tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.189087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.189839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.190967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.190967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.191991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.194058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.217439 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2569/5102] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:08 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:08.221366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.236818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cfb3d-9497-4105-93a5-e0c158bce938 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.320176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.320563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.320922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.321262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.324173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.337386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189.: keystone.exception.ImpliedRoleNotFound: 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189. Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = query.one() Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._iter().one() Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._only_one_row( Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:08.350722 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 27 10:17:08.351833 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 27 10:17:08.352915 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 27 10:17:08.352915 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189. Sep 27 10:17:08.352915 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:08.356905 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2534/5103] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:17:08 2022] GET /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 137 bytes in 136 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:17:08.358455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.369396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa1007-8a25-4d39-ad32-7548d1d6c02a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.375082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.375754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.376060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.376357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.377973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.379560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.381509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189.: keystone.exception.ImpliedRoleNotFound: 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189. Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = query.one() Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._iter().one() Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._only_one_row( Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:08.394117 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 27 10:17:08.395376 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 27 10:17:08.396426 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 27 10:17:08.396426 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 27 10:17:08.396426 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 27 10:17:08.396426 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 7722957c891e43498a31f4ce9abf17f0 does not imply f785ffeca3af431d975dbc0641c58189. Sep 27 10:17:08.396426 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:08.399542 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2570/5104] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:08 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 137 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:17:08.407422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.430704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01ceed-0b5f-4d5d-b3a3-41ecf29fef5c tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.462981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.463847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.464173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.466814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.471652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.476128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.514231 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2535/5105] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:08 2022] PUT /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 519 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:08.518082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.533077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056482b1-3178-4efd-8ca7-326c91df9777 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.606973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.607923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.608274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.608693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.610516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=1a993c0260984a1bb8f97c239c6ecfaf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.612214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.668433 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2571/5106] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:08 2022] PUT /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf => generated 518 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:08.669691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.686667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23159e7f-1f5e-4a04-8368-315a93a8cabc tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.756532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.757249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.757536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.757812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.759476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=1a993c0260984a1bb8f97c239c6ecfaf, implied_role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.761142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.792730 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2536/5107] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:08 2022] PUT /identity/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584 => generated 518 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:08.795293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.807630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0bd8b-6649-47f6-941e-fbebd7beea88 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.858076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.860268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:08.887320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:08.916063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.917630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.917930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.918536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.919385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.921428 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:08.921428 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:08.921428 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:08.922325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.952593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2572/5108] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:17:08 2022] GET /identity/v3/role_inferences => generated 1601 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:08.954072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:08.964746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154453a-cc97-467a-ad86-c51ec52ea0b4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.969568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9bfb0-4166-4f5e-95f1-8b573abd2cbd tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:08.970096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.971055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.971486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.971995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.974500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:08.976134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:08.976134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:08.976134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:08.977180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=1a993c0260984a1bb8f97c239c6ecfaf, implied_role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:08.978270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:08.979250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:08.996467 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2573/5109] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:08 2022] DELETE /identity/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:08.997636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.065834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.072990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.089244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab404a4e-0fe7-4dc6-a552-8fb48a5eba8b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.090474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2537/5110] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:09.107275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.108209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.108645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.109443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.111620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=1a993c0260984a1bb8f97c239c6ecfaf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.114356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.116198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:09.128155 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2574/5111] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:08 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:09.134941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.149039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33accb1-d4c0-40e3-b34e-d0a5c3880cfd service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.181463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.184984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:09.295328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849f0ef-efd5-43b6-87a6-0aa129204785 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.296921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.297815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.298363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.299108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.301456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.302467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.303231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.303536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.304528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.305096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] PATH_INFO: `/v3/users/095e814e7164440a9dddb2095a2912ef` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.307139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=095e814e7164440a9dddb2095a2912ef)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.309465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.325843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2538/5112] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:09 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:09.327035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.436627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:09.436807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095e814e7164440a9dddb2095a2912ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:09.438350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095e814e7164440a9dddb2095a2912ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:09.442419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f098f4d-ae5e-4370-891a-2fb568245f0a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.454214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.455453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.455733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.456322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.457282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.464694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.483024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095e814e7164440a9dddb2095a2912ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:09.483997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2539/5113] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:09 2022] GET /identity/v3/roles => generated 2319 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:09.485285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.487913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:09.498916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095e814e7164440a9dddb2095a2912ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:09.499843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f5554362-6b71-40e3-beb1-d72a0013ae30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095e814e7164440a9dddb2095a2912ef'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:09.506204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8bf3d-99e8-4066-9284-39f9fc5ea916 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.513057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.514747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.515125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.515694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.516798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.518951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.539704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2575/5114] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:09 2022] DELETE /identity/v3/users/095e814e7164440a9dddb2095a2912ef => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:09.544422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.544782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.545403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.548953 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2540/5115] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:17:09 2022] POST /identity/v3/roles => generated 234 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:09.552831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.564102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23d84a-21bb-49f8-85fc-b29c10f04506 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.574871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:09.576762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:09.670805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.671496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.671850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.675120 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2576/5116] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:09.675444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/f98d2e576a3d43a19712c7914697c4b9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.677352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:update_role(role_id=f98d2e576a3d43a19712c7914697c4b9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.678814 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:09.678814 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:09.678814 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:09.679780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.686767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.688326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:09.722932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:09.730968 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2541/5117] 10.222.0.26 () {60 vars in 1243 bytes} [Tue Sep 27 10:17:09 2022] PATCH /identity/v3/roles/f98d2e576a3d43a19712c7914697c4b9 => generated 237 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:09.735129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.770558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58916bc1-9b23-412c-b85f-69b80102004b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.784258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.784957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.788916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.789199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/f98d2e576a3d43a19712c7914697c4b9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.792946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0650e-036b-4be1-8db4-76e13c2740e9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.798747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.800481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.801273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.803823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.811066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:get_role(role_id=f98d2e576a3d43a19712c7914697c4b9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.812252 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:09.812252 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:09.812252 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:09.813420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.815029 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2542/5118] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:17:09 2022] GET /identity/v3/roles/f98d2e576a3d43a19712c7914697c4b9 => generated 237 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:09.823889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.826077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.827832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.840567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a60142-5c23-4067-bec2-327d975cfb87 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.844887 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2577/5119] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:09.845922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.846540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.846827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.847099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.847777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.849756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.873662 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2543/5120] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:09 2022] GET /identity/v3/roles => generated 2547 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:09.875012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.893999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efd5fc-751e-422e-b87b-1296a97bad95 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:09.913585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:09.914418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:09.915027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:09.915405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/f98d2e576a3d43a19712c7914697c4b9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:09.918787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f98d2e576a3d43a19712c7914697c4b9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:09.921848 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:09.921848 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:09.921848 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:09.923077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:09.986752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role f98d2e576a3d43a19712c7914697c4b9 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:09.987263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:09.988201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:09.995138 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2578/5121] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:09 2022] DELETE /identity/v3/roles/f98d2e576a3d43a19712c7914697c4b9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:09.996184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:09.997992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:10.013476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb87d1-29a7-4cdb-b11c-d6840e36416f tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:10.110002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:10.111465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.111598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.111913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.114005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:10.116451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:10.152928 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2544/5122] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:09 2022] PUT /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 519 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:10.158278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:10.171923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddc250-fcc5-4e33-b59a-d89eee25798a tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:10.259135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:10.259727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.260096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.260374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.262440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=1a993c0260984a1bb8f97c239c6ecfaf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:10.264432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:10.284607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:10.286826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:10.299514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2579/5123] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:10 2022] PUT /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf => generated 518 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:10.300813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:10.335585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:10.397136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:10.402072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9092035-67b7-4c20-bb32-0ebe477766b8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:10.408416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:10.409115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.409556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.410467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.412783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=1a993c0260984a1bb8f97c239c6ecfaf, implied_role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:10.413055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d52000-03e1-401d-bfa0-66a200892ea9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:10.414972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:10.450773 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2580/5124] 10.222.0.26 () {60 vars in 1363 bytes} [Tue Sep 27 10:17:10 2022] PUT /identity/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584 => generated 518 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:10.452393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:10.500795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:10.501452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.501732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.502002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] PATH_INFO: `/v3/projects/a6c1259e89284611bc708904b122eb7e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.504174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c1259e89284611bc708904b122eb7e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:10.507223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:10.550903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f5776-2b0c-43ae-9859-395d34ebbab5 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:10.557019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:10.557747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.558035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.558850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.559920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=1a993c0260984a1bb8f97c239c6ecfaf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:10.560983 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:10.560983 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:10.560983 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:10.561955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:10.597570 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2581/5125] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:17:10 2022] GET /identity/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies => generated 487 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:10.598681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:10.618754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c1259e89284611bc708904b122eb7e', 'request_id': 'req-c9f68b97-6321-4126-b353-7b9c8c9ec535'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:10.626127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375260e5-f123-4ac6-9869-5506ade3bf7b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:10.629716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c1259e89284611bc708904b122eb7e', 'request_id': 'req-c9f68b97-6321-4126-b353-7b9c8c9ec535'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:10.644371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c1259e89284611bc708904b122eb7e', 'request_id': 'req-c9f68b97-6321-4126-b353-7b9c8c9ec535'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:10.675419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] The token cache is being invalidate because project a6c1259e89284611bc708904b122eb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:10.678712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:10.679808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c9f68b97-6321-4126-b353-7b9c8c9ec535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:10.689583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2545/5126] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:10 2022] DELETE /identity/v3/projects/a6c1259e89284611bc708904b122eb7e => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:10.692645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3438-2e41-42d4-972c-394e806ff02d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.695371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3438-2e41-42d4-972c-394e806ff02d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.695659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3438-2e41-42d4-972c-394e806ff02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.731014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-abef3438-2e41-42d4-972c-394e806ff02d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:10.734394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-abef3438-2e41-42d4-972c-394e806ff02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:10.789466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:10.791582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.793142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.793431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.796742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=7722957c891e43498a31f4ce9abf17f0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:10.799353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:10.816551 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2582/5127] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:17:10 2022] GET /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies => generated 657 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:10.824251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:10.831641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:10.877748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91dfb4-af8c-4ba3-8fb6-2dd05e604dba tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:10.884080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:10.898274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:10.898993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.899277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.901947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.907412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=1a993c0260984a1bb8f97c239c6ecfaf, implied_role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:10.912983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:10.916016 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2546/5128] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:10.926809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428a5c4-d218-47f0-8830-83a6edeaca0f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:10.926809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428a5c4-d218-47f0-8830-83a6edeaca0f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:10.927033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428a5c4-d218-47f0-8830-83a6edeaca0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:10.949599 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2583/5129] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:10 2022] DELETE /identity/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf/implies/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:10.951267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:10.995785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2428a5c4-d218-47f0-8830-83a6edeaca0f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:11.001625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2428a5c4-d218-47f0-8830-83a6edeaca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:11.156545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db024d-fbb8-4841-8f2a-e7836bcc7c0e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:11.169662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:11.170554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:11.170842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:11.172702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:11.175777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=1a993c0260984a1bb8f97c239c6ecfaf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:11.177520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:11.185425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:11.197697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2547/5130] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:11.203866 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2584/5131] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:10 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/1a993c0260984a1bb8f97c239c6ecfaf => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:11.205250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428a5c4-d218-47f0-8830-83a6edeaca0f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:11.208114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:11.246568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428a5c4-d218-47f0-8830-83a6edeaca0f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:11.459811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a0ec-28be-4de7-9832-4b081b46d8d8 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:11.486887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:11.487610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:11.487900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:11.488734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:11.492315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=7722957c891e43498a31f4ce9abf17f0, implied_role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:11.500783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:11.506839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:11.507762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:11.509495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:11.518671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:11.519996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:11.522118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:11.526759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:11.537952 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2585/5132] 10.222.0.26 () {58 vars in 1347 bytes} [Tue Sep 27 10:17:11 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0/implies/f785ffeca3af431d975dbc0641c58189 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:11.538939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:11.557656 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2548/5133] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:11.562335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:11.582843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50a7dd-f14f-4c2c-b83d-ec2158484620 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:11.664902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29acf62-ee25-4379-9d77-906893efad0b tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:11.664902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:11.664902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:11.665523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:11.665523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/057641dce4c748b1ad752a91bc31b724` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:11.675647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Could not find domain: 057641dce4c748b1ad752a91bc31b724.: keystone.exception.DomainNotFound: Could not find domain: 057641dce4c748b1ad752a91bc31b724. Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 057641dce4c748b1ad752a91bc31b724. Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:11.682695 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:17:11.683860 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 057641dce4c748b1ad752a91bc31b724. Sep 27 10:17:11.685059 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:11.687917 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2586/5134] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:17:11 2022] PATCH /identity/v3/domains/057641dce4c748b1ad752a91bc31b724 => generated 112 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:17:11.697342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:11.717406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d7bae-38b8-4da7-ab86-67842accc141 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:11.725011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:11.726561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:11.726561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:11.726561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/users/3c6740d580674798bbd1e2c8c79e1465` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:11.729507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3c6740d580674798bbd1e2c8c79e1465)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:11.731854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:11.770586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:11.775813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:11.776537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:11.777676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:11.779250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:11.784191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:11.816147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6740d580674798bbd1e2c8c79e1465'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:11.817010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6740d580674798bbd1e2c8c79e1465'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:11.828050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6740d580674798bbd1e2c8c79e1465'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:11.846977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6740d580674798bbd1e2c8c79e1465'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:11.847912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6740d580674798bbd1e2c8c79e1465'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:11.863708 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2549/5135] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:11 2022] POST /identity/v3/projects => generated 371 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:11.867582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:11.873199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:11.876571 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2587/5136] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:11 2022] DELETE /identity/v3/users/3c6740d580674798bbd1e2c8c79e1465 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:11.878418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:11.881644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:11.894225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:11.930479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:11.982303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5829f0-483f-491b-a8ab-44c703820716 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:11.991340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:11.991918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:11.992350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:11.993351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/d66df6111bc44bf7b60729fa93422584` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:11.995316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d66df6111bc44bf7b60729fa93422584)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:11.998354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:12.062635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role d66df6111bc44bf7b60729fa93422584 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:12.063088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:12.064983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:12.071890 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2588/5137] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:11 2022] DELETE /identity/v3/roles/d66df6111bc44bf7b60729fa93422584 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:12.071890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:12.073525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:12.087123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fefa99-4744-43fe-ab3c-c623ac44b098 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:12.247227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba445d-9178-40a4-903f-e5a0c20658d4 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:12.261901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:12.262596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.262877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.263154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/groups/2d07ce47431146d4abbca7fbbf44b9b2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.269305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_group(group_id=2d07ce47431146d4abbca7fbbf44b9b2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:12.270354 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:12.270354 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:17:12.270354 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:17:12.271634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:12.275757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:12.276393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.276904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.277306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.285057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:12.287270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:12.335804 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2550/5138] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:11 2022] POST /identity/v3/users => generated 286 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:12.342970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:12.347371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:12.422563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5643488-aa62-4205-b329-618403ec2c38 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:12.428815 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2589/5139] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:17:12 2022] DELETE /identity/v3/groups/2d07ce47431146d4abbca7fbbf44b9b2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:12.432122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:12.441268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:12.441504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.441788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.442066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.442736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:12.445156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:12.478692 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2551/5140] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:12 2022] GET /identity/v3/roles => generated 2094 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:12.483573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:12.502914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3148d8-3e7d-481a-a3e9-369012f20cbf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:12.558631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:12.627184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:12.627411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.627635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.627831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.628484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:12.630327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:12.643448 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2552/5141] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:12 2022] GET /identity/v3/roles => generated 2094 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:12.647789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:12.658734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326ae0-d42b-4042-8f58-3d8f8944fdc9 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:12.665946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236a3f-6486-468f-9bf9-72b12defaf29 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:12.675846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:12.676592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:12.679951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.679951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.679951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/projects/339c23e8cefe4ccdb98aa25939614bc0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.681382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.684716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_project(project_id=339c23e8cefe4ccdb98aa25939614bc0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:12.696782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:12.698092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.698580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] PATH_INFO: `/v3/projects/ae1ddda2488c4d6183b3318b4ed0400a/users/b66e47ddd4c34bd99634d3ca1eb83b33/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.728138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1ddda2488c4d6183b3318b4ed0400a, user_id=b66e47ddd4c34bd99634d3ca1eb83b33, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:12.739354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:12.781075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '339c23e8cefe4ccdb98aa25939614bc0', 'request_id': 'req-99711eab-6e77-46fe-9c6f-5b1d29be1269'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:12.797141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '339c23e8cefe4ccdb98aa25939614bc0', 'request_id': 'req-99711eab-6e77-46fe-9c6f-5b1d29be1269'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:12.803519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2553/5142] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:12 2022] PUT /identity/v3/projects/ae1ddda2488c4d6183b3318b4ed0400a/users/b66e47ddd4c34bd99634d3ca1eb83b33/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:12.806713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:12.812890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:12.819629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '339c23e8cefe4ccdb98aa25939614bc0', 'request_id': 'req-99711eab-6e77-46fe-9c6f-5b1d29be1269'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:12.846445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:12.874941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] The token cache is being invalidate because project 339c23e8cefe4ccdb98aa25939614bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:12.875234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:12.876164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99711eab-6e77-46fe-9c6f-5b1d29be1269 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:12.891035 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2590/5143] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:12 2022] DELETE /identity/v3/projects/339c23e8cefe4ccdb98aa25939614bc0 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:12.894632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.897137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.898128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.957974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] MFA Rules not processed for user `b66e47ddd4c34bd99634d3ca1eb83b33`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:12.959329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:12.970045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb15b7f-d069-4e0c-a781-553230712dc3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:12.978962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:12.979938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:12.980444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:12.980953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:12.983533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:13.004022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:13.006128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:13.030365 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2554/5144] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:13.031298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:13.033054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:13.084176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2591/5145] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:13.092056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:13.094383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:13.153431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0fd62-b5e2-4513-9b7a-655d3a423e77 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:13.161082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:13.162945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:13.163452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:13.163963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:13.166010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=faa766ce7d784ab3b5a3b882841e8f49)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:13.169017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:13.212268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.213704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.254785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:13.264524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.266053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.269513 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2555/5146] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:17:13 2022] PATCH /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49 => generated 278 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:13.274418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:13.276839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:13.293324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d2e6a-950e-4755-9322-88eb9c9cce99 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:13.400468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80dbaf-b5e8-4e1e-8842-aabd1128a074 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:13.415669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:13.416441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:13.416879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:13.419415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/domains/faa766ce7d784ab3b5a3b882841e8f49` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:13.425362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:13.425995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:13.426266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:13.426536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:13.427171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:13.430170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:13.454271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=faa766ce7d784ab3b5a3b882841e8f49)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:13.456434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:13.459363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:13.522139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.524581 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2592/5147] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:13 2022] POST /identity/v3/projects => generated 373 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:13.529081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:13.533358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:13.556609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.647049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49', 'request_id': 'req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.665872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49', 'request_id': 'req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.695036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49', 'request_id': 'req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.718324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c2896-3068-4a8f-aedb-ca176cb3db4a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:13.728516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] The token cache is being invalidate because project faa766ce7d784ab3b5a3b882841e8f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:13.728847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.731126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.747460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49', 'request_id': 'req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.750748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49', 'request_id': 'req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.763074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'faa766ce7d784ab3b5a3b882841e8f49', 'request_id': 'req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:13.776155 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2556/5148] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:17:13 2022] DELETE /identity/v3/domains/faa766ce7d784ab3b5a3b882841e8f49 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:13.790783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:13.794376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:13.853959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:13.857898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:13.858208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:13.858487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:13.859139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:13.861320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:13.910177 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2593/5149] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:13 2022] POST /identity/v3/users => generated 293 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:13.914227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:13.917286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:13.941324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63bc94-d71a-45b0-a239-845853e06ca6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:13.955702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:13.956348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:13.956626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:13.956940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:13.959019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:13.962393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:13.968992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf87b9d-2cc7-4f53-a5f0-0879c91974b6 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:13.984648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:13.985531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:13.985818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:13.986099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:13.989436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1a993c0260984a1bb8f97c239c6ecfaf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:13.991414 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2594/5150] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:13 2022] GET /identity/v3/roles => generated 2094 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:13.991738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:13.993126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:14.013624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316140e-fc97-4c62-a9e4-a910c2cc820e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:14.021322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:14.022330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:14.022638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:14.022918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] PATH_INFO: `/v3/projects/3d1b83e7de244f15a1221937208469ca/users/460df909df784f609876223418ba76e7/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:14.026632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:14.045574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1b83e7de244f15a1221937208469ca, user_id=460df909df784f609876223418ba76e7, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:14.052477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:14.091272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role 1a993c0260984a1bb8f97c239c6ecfaf has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:14.092144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:14.094182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:14.103637 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2557/5151] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:13 2022] DELETE /identity/v3/roles/1a993c0260984a1bb8f97c239c6ecfaf => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:14.106389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:14.109275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:14.117097 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2595/5152] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:13 2022] PUT /identity/v3/projects/3d1b83e7de244f15a1221937208469ca/users/460df909df784f609876223418ba76e7/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:14.121758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:14.124635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:14.145590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:14.288644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cac2d-6661-4df0-8640-6c7f1e7d9140 tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:14.362956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:14.363828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:14.364223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:14.372806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66be1ab-13cb-4642-b982-224a11cd8e66 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:14.373216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/f785ffeca3af431d975dbc0641c58189` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:14.375570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f785ffeca3af431d975dbc0641c58189)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:14.378561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:14.503338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role f785ffeca3af431d975dbc0641c58189 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:14.503864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:14.504637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-368308bf-c707-4a9d-b7fd-851ff4e6598e tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:14.507533 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2558/5153] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:14 2022] DELETE /identity/v3/roles/f785ffeca3af431d975dbc0641c58189 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:14.509263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:14.509820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:14.510219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:14.566243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:14.566913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:14.567198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:14.572098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:14.574215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:14.576255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:14.614548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:14.621206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:14.626048 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2596/5154] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:14 2022] GET /identity/v3/roles => generated 1645 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:14.635077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:14.648925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:14.685829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60005003-3160-4881-8412-4a683cfa7ffc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:14.781177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2559/5155] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:14.785569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:14.786739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:14.848461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6beefa908daf4834afa1c476bc26267b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ceead42ebe69410a87378451144494d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:14.851411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:14.857334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:14.857844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] PATH_INFO: `/v3/roles/7722957c891e43498a31f4ce9abf17f0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:14.862520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7722957c891e43498a31f4ce9abf17f0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:14.864967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:14.901652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:14.923121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f4123-fdb7-42ea-885d-7971f1d89ddb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:14.928459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345709b8-313d-4c41-8ce4-7e77303e85e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:14.929181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345709b8-313d-4c41-8ce4-7e77303e85e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:14.929464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345709b8-313d-4c41-8ce4-7e77303e85e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:14.930608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345709b8-313d-4c41-8ce4-7e77303e85e5 admin admin] PATH_INFO: `/v3/projects/3d1b83e7de244f15a1221937208469ca/users/460df909df784f609876223418ba76e7/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:14.932822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invalidating the token cache because role 7722957c891e43498a31f4ce9abf17f0 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:14.933235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:14.934113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:14.934969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345709b8-313d-4c41-8ce4-7e77303e85e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1b83e7de244f15a1221937208469ca, user_id=460df909df784f609876223418ba76e7, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:14.936184 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2597/5156] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:14 2022] DELETE /identity/v3/roles/7722957c891e43498a31f4ce9abf17f0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:14.938517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345709b8-313d-4c41-8ce4-7e77303e85e5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:14.940106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:14.945897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:14.966924 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2560/5157] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:14 2022] PUT /identity/v3/projects/3d1b83e7de244f15a1221937208469ca/users/460df909df784f609876223418ba76e7/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:14.969554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:14.969849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:14.970177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:15.035898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] MFA Rules not processed for user `460df909df784f609876223418ba76e7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:15.039814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:15.049041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:15.170745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad9c3-281c-4e5f-a013-c210113c446d tempest-RolesV3TestJSON-389777146 tempest-RolesV3TestJSON-389777146-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:15.240028 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2561/5158] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:15.241783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:15.248209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:15.352996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:15.354015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:15.354468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:15.354844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] PATH_INFO: `/v3/users/5567d7d2026742b9b46bbfda36c4702a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:15.356511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5567d7d2026742b9b46bbfda36c4702a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:15.358424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:15.431023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e8bb0-901f-4205-8222-fc6c7c38672c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:15.444556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbb2d7-6f8d-4ba6-a951-ce233610da9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:15.445235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb2d7-6f8d-4ba6-a951-ce233610da9d service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:15.445515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb2d7-6f8d-4ba6-a951-ce233610da9d service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:15.445797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb2d7-6f8d-4ba6-a951-ce233610da9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:15.458810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f7bbb2d7-6f8d-4ba6-a951-ce233610da9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:15.486037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5567d7d2026742b9b46bbfda36c4702a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:15.489072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5567d7d2026742b9b46bbfda36c4702a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:15.510040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5567d7d2026742b9b46bbfda36c4702a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:15.528482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5567d7d2026742b9b46bbfda36c4702a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:15.531413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5567d7d2026742b9b46bbfda36c4702a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:15.581819 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2598/5159] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:14 2022] DELETE /identity/v3/users/5567d7d2026742b9b46bbfda36c4702a => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:15.586872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:15.590767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:15.647364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb2d7-6f8d-4ba6-a951-ce233610da9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:15.653586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb2d7-6f8d-4ba6-a951-ce233610da9d service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:15.662911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:15.694030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2562/5160] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:15.695272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:15.695563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:15.695850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:15.755847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:15.759070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:15.784133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9775a-3a99-4c99-b8e4-d21c83809966 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:15.788568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:15.789270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:15.790074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:15.790074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:15.791734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:15.842168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:15.869173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:15.871441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:15.882511 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2563/5161] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:15.884598 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2599/5162] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:15.888571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:15.890724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:16.046299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:16.049440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:16.130335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7ee49-3cf5-4155-97f3-7b90cb0c9d96 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:16.139420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:16.140245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:16.140694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:16.141019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:16.170475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:16.172221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:16.203421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2564/5163] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:16.208196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7b63d-fbf9-4647-90d9-24fd260d2dae service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:16.216872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:16.217671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:16.218020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:16.218373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:16.220416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:16.239287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:16.241184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:16.254368 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2600/5164] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:16.682781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:16.684598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:16.708829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c750d7f-9849-4f17-a051-27b93e0f3688 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:16.726043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:16.726812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:16.727192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:16.727537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] PATH_INFO: `/v3/users/b66e47ddd4c34bd99634d3ca1eb83b33` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:16.729343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66e47ddd4c34bd99634d3ca1eb83b33)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:16.731366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:16.866888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66e47ddd4c34bd99634d3ca1eb83b33'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:16.869945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66e47ddd4c34bd99634d3ca1eb83b33'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:16.873582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:16.875120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:16.889554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66e47ddd4c34bd99634d3ca1eb83b33'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:16.905696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a100-84a9-4f48-8460-3694fb417f10 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:16.908731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66e47ddd4c34bd99634d3ca1eb83b33'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:16.916175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:16.930443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:16.930560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:16.930765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] PATH_INFO: `/v3/projects/31d5b9b67fbe41079c7cba8ae39c161e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:16.932179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d5b9b67fbe41079c7cba8ae39c161e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:16.934844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:16.972444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3ca007ff-ab9c-4030-b54a-e9aef2e13417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66e47ddd4c34bd99634d3ca1eb83b33'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.008900 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2565/5165] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:16 2022] DELETE /identity/v3/users/b66e47ddd4c34bd99634d3ca1eb83b33 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:17.012153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:17.019873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:17.021384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:17.021759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:17.029509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d5b9b67fbe41079c7cba8ae39c161e', 'request_id': 'req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.052865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d5b9b67fbe41079c7cba8ae39c161e', 'request_id': 'req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.062357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d5b9b67fbe41079c7cba8ae39c161e', 'request_id': 'req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.076507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:17.078259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:17.088556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] The token cache is being invalidate because project 31d5b9b67fbe41079c7cba8ae39c161e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:17.088901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.089880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.094604 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2601/5166] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:16 2022] DELETE /identity/v3/projects/31d5b9b67fbe41079c7cba8ae39c161e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:17.095594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:17.100879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:17.202607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:17.233273 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2566/5167] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:17.257435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:17.259084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:17.262197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c37c3e-b5dd-4d99-bdc9-22e4b969731e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:17.270128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:17.271403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:17.272488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:17.272488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] PATH_INFO: `/v3/users/6beefa908daf4834afa1c476bc26267b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:17.273910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6beefa908daf4834afa1c476bc26267b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:17.276389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:17.382766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5911afe-4e44-48a6-b108-00af38069af4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:17.391075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:17.391719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:17.392003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:17.392277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:17.413024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6beefa908daf4834afa1c476bc26267b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.413969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6beefa908daf4834afa1c476bc26267b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.418912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:17.420293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:17.434233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6beefa908daf4834afa1c476bc26267b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.442428 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2567/5168] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:17.449354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6beefa908daf4834afa1c476bc26267b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.451703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6beefa908daf4834afa1c476bc26267b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:17.496380 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2602/5169] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:17 2022] DELETE /identity/v3/users/6beefa908daf4834afa1c476bc26267b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:18.687174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:18.689543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:18.714628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d688b04-27dc-4f97-b638-4b9e209bc27e service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:18.792448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:18.794088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:18.860066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:18.861766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:18.862173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:18.862730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] PATH_INFO: `/v3/projects/ae1ddda2488c4d6183b3318b4ed0400a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:18.864181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1ddda2488c4d6183b3318b4ed0400a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:18.869148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:18.918309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:18.957346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f6ef-85e8-4ee7-9738-0788a51fed90 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:19.002861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1ddda2488c4d6183b3318b4ed0400a', 'request_id': 'req-2df6e141-e31d-4f81-acb2-0803d43ca541'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.014748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1ddda2488c4d6183b3318b4ed0400a', 'request_id': 'req-2df6e141-e31d-4f81-acb2-0803d43ca541'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.029596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1ddda2488c4d6183b3318b4ed0400a', 'request_id': 'req-2df6e141-e31d-4f81-acb2-0803d43ca541'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.056542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] The token cache is being invalidate because project ae1ddda2488c4d6183b3318b4ed0400a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:19.057178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.058037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.063845 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2568/5170] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:18 2022] DELETE /identity/v3/projects/ae1ddda2488c4d6183b3318b4ed0400a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:19.065147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:19.066845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:19.106877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:19.108832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.109131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.110132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] PATH_INFO: `/v3/projects/ceead42ebe69410a87378451144494d2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.111487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceead42ebe69410a87378451144494d2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:19.114642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:19.139582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:19.191854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6e141-e31d-4f81-acb2-0803d43ca541 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:19.233679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceead42ebe69410a87378451144494d2', 'request_id': 'req-e56e0ba8-1c38-4357-879c-d9878db97b79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.258192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceead42ebe69410a87378451144494d2', 'request_id': 'req-e56e0ba8-1c38-4357-879c-d9878db97b79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.266912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceead42ebe69410a87378451144494d2', 'request_id': 'req-e56e0ba8-1c38-4357-879c-d9878db97b79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.284042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] The token cache is being invalidate because project ceead42ebe69410a87378451144494d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:19.284494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.285713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e56e0ba8-1c38-4357-879c-d9878db97b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.288912 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2603/5171] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:18 2022] DELETE /identity/v3/projects/ceead42ebe69410a87378451144494d2 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:19.294238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717fc0b-a8af-45aa-a78f-5decc547cfbe None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.294529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717fc0b-a8af-45aa-a78f-5decc547cfbe None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.297409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717fc0b-a8af-45aa-a78f-5decc547cfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.303248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:19.303924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.304283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.304491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] PATH_INFO: `/v3/users/460df909df784f609876223418ba76e7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.306208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=460df909df784f609876223418ba76e7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:19.308135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:19.344285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0717fc0b-a8af-45aa-a78f-5decc547cfbe None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:19.345549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0717fc0b-a8af-45aa-a78f-5decc547cfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:19.361398 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2604/5172] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:19.370503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f33c8c-0aa8-40ed-b7dc-8be02667e958 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.371037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f33c8c-0aa8-40ed-b7dc-8be02667e958 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.371452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f33c8c-0aa8-40ed-b7dc-8be02667e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.419206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-08f33c8c-0aa8-40ed-b7dc-8be02667e958 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:19.420803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08f33c8c-0aa8-40ed-b7dc-8be02667e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:19.434046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '460df909df784f609876223418ba76e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.435138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '460df909df784f609876223418ba76e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.455462 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2605/5173] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:19.456408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f33c8c-0aa8-40ed-b7dc-8be02667e958 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:19.457522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '460df909df784f609876223418ba76e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.475984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f33c8c-0aa8-40ed-b7dc-8be02667e958 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:19.477215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '460df909df784f609876223418ba76e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.483550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:19.484262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.484534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.484832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.485484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:19.485769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:19.487358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:19.513900 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2606/5174] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:19.515495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '460df909df784f609876223418ba76e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:19.516304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:19.524118 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2569/5175] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:19 2022] DELETE /identity/v3/users/460df909df784f609876223418ba76e7 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:19.528843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:19.548633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c57f2-da1e-497b-91e6-19057629cb5f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:19.633463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:19.635735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.636172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.636546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.639369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:19.643753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:19.655104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:19.719148 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2607/5176] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:19 2022] POST /identity/v3/projects => generated 367 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:19.720231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:19.735320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b1dc7-af02-45d2-bd0f-f2fe97e0c0d0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:19.825520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:19.841615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:19.842466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.847899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.848597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.850953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:19.855111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:19.898704 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2570/5177] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:19 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:19.903100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:19.914042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e373b-e1f2-48b5-9a08-4a748293a46c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:19.919906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:19.920555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.920944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.921226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.923106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:19.925288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:19.945968 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2608/5178] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:19.946826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:19.958352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51394b-4be3-4949-ad66-0025a093ce0d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:19.964105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba784171-95e1-4190-b60f-79dd33de1212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:19.965018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba784171-95e1-4190-b60f-79dd33de1212 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:19.965644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba784171-95e1-4190-b60f-79dd33de1212 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:19.965644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba784171-95e1-4190-b60f-79dd33de1212 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:19.966468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba784171-95e1-4190-b60f-79dd33de1212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:19.968269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba784171-95e1-4190-b60f-79dd33de1212 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:19.983361 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2571/5179] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:19.984610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:19.996212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a85a57-88aa-44cb-b076-58ba5cd7171f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.002285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.003118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.003500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.003833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] PATH_INFO: `/v3/projects/c8f0327362e649028455459af40f5f12/users/172b384e97154a63a6e9fe153ec8bc0f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.021569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f0327362e649028455459af40f5f12, user_id=172b384e97154a63a6e9fe153ec8bc0f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:20.024426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:20.047354 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2609/5180] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:19 2022] PUT /identity/v3/projects/c8f0327362e649028455459af40f5f12/users/172b384e97154a63a6e9fe153ec8bc0f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:20.058622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79f3a-bee2-40d0-84cb-0d70716dfad4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.058903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79f3a-bee2-40d0-84cb-0d70716dfad4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.059181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79f3a-bee2-40d0-84cb-0d70716dfad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.093501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d7e79f3a-bee2-40d0-84cb-0d70716dfad4 None None] MFA Rules not processed for user `172b384e97154a63a6e9fe153ec8bc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:20.095224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d7e79f3a-bee2-40d0-84cb-0d70716dfad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.196553 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2572/5181] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:20.197997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.212287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87bc4-569e-4f5b-91c4-35dd4be18987 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.216594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21eeb3-83e6-4256-95e3-e908da18389d tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '172b384e97154a63a6e9fe153ec8bc0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8f0327362e649028455459af40f5f12', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.217187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21eeb3-83e6-4256-95e3-e908da18389d tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.217455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21eeb3-83e6-4256-95e3-e908da18389d tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.217903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21eeb3-83e6-4256-95e3-e908da18389d tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] PATH_INFO: `/v3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.219277 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2610/5182] 10.222.0.26 () {58 vars in 1104 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/v3 => generated 255 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:20.226611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79f3a-bee2-40d0-84cb-0d70716dfad4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.238626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79f3a-bee2-40d0-84cb-0d70716dfad4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.243528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef9986-2c16-4e71-b0d2-99715d46c138 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '172b384e97154a63a6e9fe153ec8bc0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8f0327362e649028455459af40f5f12', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.244158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9986-2c16-4e71-b0d2-99715d46c138 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.244445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9986-2c16-4e71-b0d2-99715d46c138 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.244828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9986-2c16-4e71-b0d2-99715d46c138 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] PATH_INFO: `/v3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.250783 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2573/5183] 10.222.0.26 () {58 vars in 1104 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/v3 => generated 255 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:20.252139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21eeb3-83e6-4256-95e3-e908da18389d tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.266421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21eeb3-83e6-4256-95e3-e908da18389d tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.271856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f6a4b-1cd3-41fc-a9df-5e03b523dc07 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '172b384e97154a63a6e9fe153ec8bc0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8f0327362e649028455459af40f5f12', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.272477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f6a4b-1cd3-41fc-a9df-5e03b523dc07 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.272790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f6a4b-1cd3-41fc-a9df-5e03b523dc07 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.273246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f6a4b-1cd3-41fc-a9df-5e03b523dc07 tempest-TestApiDiscovery-686998921 tempest-TestApiDiscovery-686998921-project] PATH_INFO: `/v3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.284552 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2611/5184] 10.222.0.26 () {58 vars in 1104 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/v3 => generated 255 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:20.284679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500853e8-019e-42ee-9ec1-19dacec919ad None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.284679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500853e8-019e-42ee-9ec1-19dacec919ad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.285006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500853e8-019e-42ee-9ec1-19dacec919ad None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.285841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.286125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.286400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.286580 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2574/5185] 10.222.0.26 () {56 vars in 895 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/ => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:17:20.292341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128dec0-5f19-4db8-bd04-28fa7b29bfc4 None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.293032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128dec0-5f19-4db8-bd04-28fa7b29bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.293601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128dec0-5f19-4db8-bd04-28fa7b29bfc4 None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.296094 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2575/5186] 10.222.0.26 () {56 vars in 895 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/ => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 10:17:20.303033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128dec0-5f19-4db8-bd04-28fa7b29bfc4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.319344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] MFA Rules not processed for user `1d92f7d2ee2d47beaa4f5d43ac40cd82`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:20.320609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.405403 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2612/5187] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:20.410061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.412586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.421302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128dec0-5f19-4db8-bd04-28fa7b29bfc4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.431369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.432977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.433360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.434958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] PATH_INFO: `/v3/users/172b384e97154a63a6e9fe153ec8bc0f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.436066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821ba22-7eb6-4c75-98b6-341b8d54b550 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.436645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] RBAC: Authorizing `identity:delete_user(user_id=172b384e97154a63a6e9fe153ec8bc0f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:20.439157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:20.442597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.443425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.443700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.443973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] PATH_INFO: `/v3/projects/3d1b83e7de244f15a1221937208469ca` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.445592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1b83e7de244f15a1221937208469ca)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:20.449991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:20.523017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172b384e97154a63a6e9fe153ec8bc0f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.524269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172b384e97154a63a6e9fe153ec8bc0f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.528639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1b83e7de244f15a1221937208469ca', 'request_id': 'req-cd70a7d6-35c3-4a07-9abb-917a118076a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.535511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172b384e97154a63a6e9fe153ec8bc0f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.539584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:20.542465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1b83e7de244f15a1221937208469ca', 'request_id': 'req-cd70a7d6-35c3-4a07-9abb-917a118076a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.548406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172b384e97154a63a6e9fe153ec8bc0f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.557620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1b83e7de244f15a1221937208469ca', 'request_id': 'req-cd70a7d6-35c3-4a07-9abb-917a118076a7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.576136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] The token cache is being invalidate because project 3d1b83e7de244f15a1221937208469ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:20.577056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.578089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cd70a7d6-35c3-4a07-9abb-917a118076a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.578991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b8c0b7d-249f-45fb-82ac-526496e5776c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172b384e97154a63a6e9fe153ec8bc0f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:20.583540 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2613/5188] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:20 2022] DELETE /identity/v3/projects/3d1b83e7de244f15a1221937208469ca => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:20.586403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.587234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.587514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.597772 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2576/5189] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:20 2022] DELETE /identity/v3/users/172b384e97154a63a6e9fe153ec8bc0f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:20.600445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.600796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.601120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.623752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:20.625084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.642666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:20.644383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.682048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:20.713918 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2577/5190] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:20.715287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.716948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.751750 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2614/5191] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:20.751750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.751750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.834713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946af847-dc11-4261-9c1d-377ece313e7e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.842781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.842953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc69824-a05d-48b0-9ff3-565b34be9342 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.844249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.845016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.845739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.846948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3479147-5390-4e39-8546-56659aa27755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.847559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3479147-5390-4e39-8546-56659aa27755 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.847857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3479147-5390-4e39-8546-56659aa27755 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.848094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3479147-5390-4e39-8546-56659aa27755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.849409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.869461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:20.870881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3479147-5390-4e39-8546-56659aa27755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:20.872552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3479147-5390-4e39-8546-56659aa27755 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:20.881291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:20.884841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2615/5192] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:20.887064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fe2f5-6b99-42f7-b392-04d923425ec8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.887620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fe2f5-6b99-42f7-b392-04d923425ec8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.889382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fe2f5-6b99-42f7-b392-04d923425ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.896365 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2578/5193] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:20.924014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-154fe2f5-6b99-42f7-b392-04d923425ec8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:20.925989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-154fe2f5-6b99-42f7-b392-04d923425ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:20.937315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2616/5194] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:20.943604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:20.971225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65cb00-5988-46e6-99e0-cb8c2fa0b6c9 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:20.980109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:20.980807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:20.981273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:20.981555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:20.982340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:20.982593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:20.984349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:21.025849 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2579/5195] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:21.027571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fe2f5-6b99-42f7-b392-04d923425ec8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.038697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fe2f5-6b99-42f7-b392-04d923425ec8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:21.060554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:21.061041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.061320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.062809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.065031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:21.068315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:21.128440 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2617/5196] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:21 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:21.133106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.148635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3c77-b31d-4ccb-9834-d438386a07b4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:21.259869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:21.260297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.260698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.261078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.263392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:21.266265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:21.284828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:21.316496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.329566 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2580/5197] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:21 2022] POST /identity/v3/users => generated 278 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:21.332185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf0ff1-76e4-42ec-8d76-69ac466c4501 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:21.340383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:21.341212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.341594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.341969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.342725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:21.353407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:21.373710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2618/5198] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:21.375081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.389748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf1845-82f8-4814-8dab-4cdd980b9ead admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:21.404048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:21.406363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.406664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.407017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.407653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:21.409432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:21.427885 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2581/5199] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:21.435400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.447517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.453544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:21.481696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e370-f8a7-492c-b9f0-7f71738fa6b7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:21.491973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:21.492558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.493137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.493969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] PATH_INFO: `/v3/projects/c8f0327362e649028455459af40f5f12` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.495384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f0327362e649028455459af40f5f12)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:21.498865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:21.515652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557261b3-68f4-4012-9b0e-c44fb179d998 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:21.533667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38743a4-3e67-46a5-b8a1-f11ec70c3b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:21.534362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38743a4-3e67-46a5-b8a1-f11ec70c3b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.534632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38743a4-3e67-46a5-b8a1-f11ec70c3b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.535210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38743a4-3e67-46a5-b8a1-f11ec70c3b64 admin admin] PATH_INFO: `/v3/projects/64dd9a15966148b88e42e312d29f0e55/users/eea2a6c4a4f049ebba996287df6ef7a5/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.595132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38743a4-3e67-46a5-b8a1-f11ec70c3b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64dd9a15966148b88e42e312d29f0e55, user_id=eea2a6c4a4f049ebba996287df6ef7a5, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:21.596492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f0327362e649028455459af40f5f12', 'request_id': 'req-69e5c682-0191-4f86-88b4-ff8844ef24d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:21.597884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38743a4-3e67-46a5-b8a1-f11ec70c3b64 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:21.612411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f0327362e649028455459af40f5f12', 'request_id': 'req-69e5c682-0191-4f86-88b4-ff8844ef24d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:21.628209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f0327362e649028455459af40f5f12', 'request_id': 'req-69e5c682-0191-4f86-88b4-ff8844ef24d6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:21.657255 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2619/5200] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:21 2022] PUT /identity/v3/projects/64dd9a15966148b88e42e312d29f0e55/users/eea2a6c4a4f049ebba996287df6ef7a5/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:21.659031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.659369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.659738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.667648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] The token cache is being invalidate because project c8f0327362e649028455459af40f5f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:21.668517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:21.669938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69e5c682-0191-4f86-88b4-ff8844ef24d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:21.678021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2582/5201] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:21 2022] DELETE /identity/v3/projects/c8f0327362e649028455459af40f5f12 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:21.680581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb249d9-f6a1-4d57-93ef-40d43bd4fe35 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.681229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb249d9-f6a1-4d57-93ef-40d43bd4fe35 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.681866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb249d9-f6a1-4d57-93ef-40d43bd4fe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.715269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8fb249d9-f6a1-4d57-93ef-40d43bd4fe35 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:21.718163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8fb249d9-f6a1-4d57-93ef-40d43bd4fe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:21.740052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] MFA Rules not processed for user `eea2a6c4a4f049ebba996287df6ef7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:21.742264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:21.927356 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2583/5202] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:21.930469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7aa0-2278-4ff0-bcaf-a507293303ee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.932032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7aa0-2278-4ff0-bcaf-a507293303ee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.933207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7aa0-2278-4ff0-bcaf-a507293303ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.965873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9dcd7aa0-2278-4ff0-bcaf-a507293303ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:21.966891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9dcd7aa0-2278-4ff0-bcaf-a507293303ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:21.971492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:21.977035 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2584/5203] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:21.982046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd7aa0-2278-4ff0-bcaf-a507293303ee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.982228 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2620/5204] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:21.990557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:21.991984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:21.992748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd7aa0-2278-4ff0-bcaf-a507293303ee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:21.997581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:21.998308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:21.998677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:21.999041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:21.999784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:22.000175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.002401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.020175 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2585/5205] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:22.021662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.031402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b46fa-a92a-4786-a884-816f67b70c04 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.033787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f6daa-0c06-4a48-8cdf-6c27ef3eb32e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.040481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.041257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.042046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.042449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.043184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.045361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.069854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.070484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.070766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.071039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.076248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.086304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.102745 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2586/5206] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:22 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:22.104345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.132846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2621/5207] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:21 2022] POST /identity/v3/projects => generated 355 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:22.136609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.162429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:22.254352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71eee6-e55e-4159-9f8d-18fdcef2360e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.275777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.277680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.279033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727a3af-19c3-4ec3-9407-68243179803c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.279639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.281637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.283761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.286867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.287964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.288562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.290704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.300720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.303822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.312091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.347320 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2622/5208] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:22 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:22.351246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.372763 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2587/5209] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:22 2022] POST /identity/v3/users => generated 288 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:22.374108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.386066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530140-bbb6-4f51-8d28-8d62ab398346 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.386615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe921c-d74a-45f6-b29e-dfe4d3cf5016 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.405680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.406327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.406604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.407048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.408200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.410977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.411825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.412353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.413419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.413587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.414695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.417070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.438696 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2588/5210] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:22.440054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.447125 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2623/5211] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:22.454147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.459590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567f9dc-c63c-4eaf-be1e-29e85e598021 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.471244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac8570-270d-4785-bddc-d49daec13afb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.476722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.477749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.477964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.478240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] PATH_INFO: `/v3/projects/942313e3fc534b8996fecaa6f48a80a9/users/3ade97345d214139aa8408fc712a617f/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.480460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.484177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.484507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.484826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.486854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.493669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.527962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942313e3fc534b8996fecaa6f48a80a9, user_id=3ade97345d214139aa8408fc712a617f, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.533721 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2589/5212] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:22.533933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.548321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.551365 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2624/5213] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:22 2022] PUT /identity/v3/projects/942313e3fc534b8996fecaa6f48a80a9/users/3ade97345d214139aa8408fc712a617f/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:22.567005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976867a-72da-4305-8a53-17255fd628d9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.571089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.688437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f9efc-3c9f-4a4b-b5cc-f13c61eace55 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.700397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.701508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.702537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.702825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.703557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.705276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.719558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fe2f5-25e1-4a7d-956d-9884981b7c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.720473 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2625/5214] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:22.725537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe2f5-25e1-4a7d-956d-9884981b7c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.725999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe2f5-25e1-4a7d-956d-9884981b7c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.726106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:22.726339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe2f5-25e1-4a7d-956d-9884981b7c0a admin admin] PATH_INFO: `/v3/projects/4de3065ed9d34742a95f0cbb5299cc41/users/e11a8973fc4443c9a4054e2dda80fae7/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.750645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b51b47-3c06-4ab2-b463-f5e3536a9d12 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:22.763036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fe2f5-25e1-4a7d-956d-9884981b7c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de3065ed9d34742a95f0cbb5299cc41, user_id=e11a8973fc4443c9a4054e2dda80fae7, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.766283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fe2f5-25e1-4a7d-956d-9884981b7c0a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.772145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1941ffc-576c-4d26-8646-3158bc57fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:22.774585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1941ffc-576c-4d26-8646-3158bc57fa13 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.774859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1941ffc-576c-4d26-8646-3158bc57fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.776963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1941ffc-576c-4d26-8646-3158bc57fa13 admin admin] PATH_INFO: `/v3/projects/942313e3fc534b8996fecaa6f48a80a9/users/3ade97345d214139aa8408fc712a617f/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.781507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1941ffc-576c-4d26-8646-3158bc57fa13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942313e3fc534b8996fecaa6f48a80a9, user_id=3ade97345d214139aa8408fc712a617f, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:22.792642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1941ffc-576c-4d26-8646-3158bc57fa13 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:22.803085 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2590/5215] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:22 2022] PUT /identity/v3/projects/4de3065ed9d34742a95f0cbb5299cc41/users/e11a8973fc4443c9a4054e2dda80fae7/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:22.814179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181063d-62b1-4d38-9dd2-4ec1649993a9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.814507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181063d-62b1-4d38-9dd2-4ec1649993a9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.815923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181063d-62b1-4d38-9dd2-4ec1649993a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.829477 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2626/5216] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:22 2022] PUT /identity/v3/projects/942313e3fc534b8996fecaa6f48a80a9/users/3ade97345d214139aa8408fc712a617f/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:22.833573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:22.835870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:22.835870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:22.873915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] MFA Rules not processed for user `3ade97345d214139aa8408fc712a617f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:22.875435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:22.880335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:22.896789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e181063d-62b1-4d38-9dd2-4ec1649993a9 None None] MFA Rules not processed for user `e11a8973fc4443c9a4054e2dda80fae7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:22.899910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e181063d-62b1-4d38-9dd2-4ec1649993a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:23.006674 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2627/5217] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:23.022246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:23.024176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:23.080507 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2591/5218] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:23.082306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181063d-62b1-4d38-9dd2-4ec1649993a9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:23.094099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181063d-62b1-4d38-9dd2-4ec1649993a9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:23.115009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e11a8973fc4443c9a4054e2dda80fae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4de3065ed9d34742a95f0cbb5299cc41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:23.128215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:23.128506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:23.128815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] PATH_INFO: `/v3/auth/catalog` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:23.129433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:23.136929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:23.139411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1782515-d6c5-47eb-a2c2-f1876174db83 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:23.141910 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2592/5219] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:17:23 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:23.147715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:23.151049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:23.152963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:23.153384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:23.153761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:23.158815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:23.174079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:23.175889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:23.193270 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2628/5220] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:23.255439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a4ed8-1ff4-4b74-8ea6-3b93c7959d17 tempest-IdentityCatalogTest-1772609047 tempest-IdentityCatalogTest-1772609047-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:23.261469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:23.262207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:23.262644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:23.262921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] PATH_INFO: `/v3/users/e11a8973fc4443c9a4054e2dda80fae7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:23.264891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11a8973fc4443c9a4054e2dda80fae7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:23.267508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:23.299892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:23.312351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81fad9-97bc-43a2-aaa6-4e0eb7b8c2a2 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:23.317254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:23.318272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:23.318719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:23.319195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:23.331784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11a8973fc4443c9a4054e2dda80fae7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:23.332320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11a8973fc4443c9a4054e2dda80fae7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:23.340990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11a8973fc4443c9a4054e2dda80fae7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:23.370763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11a8973fc4443c9a4054e2dda80fae7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:23.398397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6bec2d0f-6e32-41ae-9074-649be89488e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11a8973fc4443c9a4054e2dda80fae7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:23.409386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:23.412938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:23.413303 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2593/5221] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:23 2022] DELETE /identity/v3/users/e11a8973fc4443c9a4054e2dda80fae7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:23.414893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:23.420686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:23.420686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:23.446638 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2629/5222] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:23.448933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:23.450364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:23.541010 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2594/5223] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:23.609856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:23.613244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:23.801059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:23.803440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:23.824606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:23.860469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11429195-60be-40bc-919d-ae49308c49d3 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:23.867710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:23.977560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:23.978186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:23.978458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:23.978726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:23.990260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5f864-948b-4734-ae0e-6f7226a5528c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:23.991157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:23.992928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:24.006290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66b4e5-5d28-416d-a02e-828fd5a99af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:24.007064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66b4e5-5d28-416d-a02e-828fd5a99af8 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:24.007305 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2630/5224] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:24.007805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66b4e5-5d28-416d-a02e-828fd5a99af8 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:24.007805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66b4e5-5d28-416d-a02e-828fd5a99af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:24.009758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5b66b4e5-5d28-416d-a02e-828fd5a99af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:24.029362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66b4e5-5d28-416d-a02e-828fd5a99af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:24.031042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66b4e5-5d28-416d-a02e-828fd5a99af8 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:24.045460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2595/5225] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:25.310170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:25.312503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:25.362699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df9c1b-3a7c-455d-acfb-d8cfef58127b service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:25.378855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:25.380841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:25.381916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:25.383175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] PATH_INFO: `/v3/projects/4de3065ed9d34742a95f0cbb5299cc41` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:25.387218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de3065ed9d34742a95f0cbb5299cc41)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:25.392468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:25.529125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de3065ed9d34742a95f0cbb5299cc41', 'request_id': 'req-048707c5-744a-4ad6-a55c-a08b9e63aeec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.540010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de3065ed9d34742a95f0cbb5299cc41', 'request_id': 'req-048707c5-744a-4ad6-a55c-a08b9e63aeec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.549195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de3065ed9d34742a95f0cbb5299cc41', 'request_id': 'req-048707c5-744a-4ad6-a55c-a08b9e63aeec'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.579132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] The token cache is being invalidate because project 4de3065ed9d34742a95f0cbb5299cc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:25.579424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.580079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-048707c5-744a-4ad6-a55c-a08b9e63aeec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.584981 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2631/5226] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:25 2022] DELETE /identity/v3/projects/4de3065ed9d34742a95f0cbb5299cc41 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:25.586896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba8c0-3b37-42d8-a1f3-e36698630c5e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:25.587375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba8c0-3b37-42d8-a1f3-e36698630c5e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:25.587465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ba8c0-3b37-42d8-a1f3-e36698630c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:25.649774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-892ba8c0-3b37-42d8-a1f3-e36698630c5e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:25.652542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-892ba8c0-3b37-42d8-a1f3-e36698630c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:25.761232 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2596/5227] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:25.769132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:25.769414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:25.769686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:25.793673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:25.796823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:25.807412 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2632/5228] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:25.813227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ba8c0-3b37-42d8-a1f3-e36698630c5e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:25.829901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ba8c0-3b37-42d8-a1f3-e36698630c5e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:25.833844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:25.835109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:25.836300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:25.836300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:25.836300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:25.837111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:25.837419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:25.839028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:25.843293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:25.857853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77df9-5757-4259-a4d9-6c596899edad None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:25.860717 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2597/5229] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:25.862974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:25.864100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:25.864789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:25.865209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:25.865433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] PATH_INFO: `/v3/users/04440378e21043b2a287e9b4c800080d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:25.866680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04440378e21043b2a287e9b4c800080d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:25.868682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:25.876696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349b21c-f7a1-43cb-8d98-37aec09033d6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:25.883442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:25.884399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:25.884710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:25.884991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:25.885638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:25.887594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:25.924679 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2598/5230] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:25 2022] POST /identity/v3/projects => generated 395 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:25.926336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:25.930967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04440378e21043b2a287e9b4c800080d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.931672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04440378e21043b2a287e9b4c800080d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.954280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04440378e21043b2a287e9b4c800080d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.973589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04440378e21043b2a287e9b4c800080d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.974747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04440378e21043b2a287e9b4c800080d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:25.987250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:26.006562 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2633/5231] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:25 2022] DELETE /identity/v3/users/04440378e21043b2a287e9b4c800080d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:26.026754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade394-11c4-443b-ba89-0e512243a366 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:26.060151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:26.160016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:26.160854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.161067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.161343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.162018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:26.164764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:26.200430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2599/5232] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:25 2022] POST /identity/v3/users => generated 304 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:26.203061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:26.205098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:26.254057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2a9d7-4050-40bb-99d0-d980e5f06833 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:26.277718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:26.278408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.278669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.278941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.279589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:26.281240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:26.301374 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2634/5233] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:26.306523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:26.322220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef56cf-9dad-462f-8dc1-abe525b39715 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:26.340128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:26.343377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.343759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.346736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] PATH_INFO: `/v3/projects/7aed84307e4d4a5ab5bd481e4752e5c4/users/39321956af7646a8830f749c794274aa/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.389374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aed84307e4d4a5ab5bd481e4752e5c4, user_id=39321956af7646a8830f749c794274aa, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:26.393554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:26.400468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:26.402612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:26.434099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0a0fb-f1fb-4621-9ce3-dcfe39bff7b1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:26.456724 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2600/5234] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:26 2022] PUT /identity/v3/projects/7aed84307e4d4a5ab5bd481e4752e5c4/users/39321956af7646a8830f749c794274aa/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:26.471270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:26.524135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:26.524767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.525052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.525323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] PATH_INFO: `/v3/projects/898c5ffacd6247c6a8bff965ebaf0591` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.526631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=898c5ffacd6247c6a8bff965ebaf0591)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:26.530460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:26.559266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e9c94-f3e3-489a-9a8f-8b607f8e4c67 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:26.565776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:26.566617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.567112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.567796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.569658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:26.572261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:26.592780 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2601/5235] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:26.594562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:26.623661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fe786-c411-444a-8ffb-2506ccb7a03d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:26.640973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '898c5ffacd6247c6a8bff965ebaf0591', 'request_id': 'req-43d6790d-3261-4c04-8980-49d4b048f8d9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:26.653409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '898c5ffacd6247c6a8bff965ebaf0591', 'request_id': 'req-43d6790d-3261-4c04-8980-49d4b048f8d9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:26.675149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '898c5ffacd6247c6a8bff965ebaf0591', 'request_id': 'req-43d6790d-3261-4c04-8980-49d4b048f8d9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:26.695519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] The token cache is being invalidate because project 898c5ffacd6247c6a8bff965ebaf0591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:26.696237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:26.697758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-43d6790d-3261-4c04-8980-49d4b048f8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:26.699917 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2635/5236] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:26 2022] DELETE /identity/v3/projects/898c5ffacd6247c6a8bff965ebaf0591 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:26.713894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee958-586f-418c-8bff-8042e5645abd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.714373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee958-586f-418c-8bff-8042e5645abd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.714651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee958-586f-418c-8bff-8042e5645abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.740044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd4699-dc53-4568-9e07-81643f09dabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:26.741336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd4699-dc53-4568-9e07-81643f09dabe admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.741681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd4699-dc53-4568-9e07-81643f09dabe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.741991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd4699-dc53-4568-9e07-81643f09dabe admin admin] PATH_INFO: `/v3/projects/7aed84307e4d4a5ab5bd481e4752e5c4/users/39321956af7646a8830f749c794274aa/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.752253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-146ee958-586f-418c-8bff-8042e5645abd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:26.753726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-146ee958-586f-418c-8bff-8042e5645abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:26.760900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd4699-dc53-4568-9e07-81643f09dabe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aed84307e4d4a5ab5bd481e4752e5c4, user_id=39321956af7646a8830f749c794274aa, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:26.766107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd4699-dc53-4568-9e07-81643f09dabe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:26.785249 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2636/5237] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:26.787071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c2b41-0ec5-4e7f-8a00-8fed6d912720 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.787516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c2b41-0ec5-4e7f-8a00-8fed6d912720 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.796562 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2602/5238] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:26 2022] PUT /identity/v3/projects/7aed84307e4d4a5ab5bd481e4752e5c4/users/39321956af7646a8830f749c794274aa/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:26.797818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c2b41-0ec5-4e7f-8a00-8fed6d912720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.798290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.798658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.798851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.843987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] MFA Rules not processed for user `39321956af7646a8830f749c794274aa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:26.845127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ff9c2b41-0ec5-4e7f-8a00-8fed6d912720 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:26.845782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:26.851436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ff9c2b41-0ec5-4e7f-8a00-8fed6d912720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:26.853299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:26.972167 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2637/5239] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:26.974133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c2b41-0ec5-4e7f-8a00-8fed6d912720 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:26.979790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:26.987279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c2b41-0ec5-4e7f-8a00-8fed6d912720 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:26.995834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:26.996558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:26.997801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:26.998163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:26.999308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:26.999689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.001957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.020954 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2603/5240] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:27.033156 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2638/5241] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:27.034201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.036190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.038975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:27.052009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4194dc5-c86a-4c62-97b1-fccc26e94520 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.052806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03110a-7c2f-4c18-8f2e-63fb79815d1a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.058539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.059039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.059263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.059846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.059846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.060044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.060155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.060387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.060507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.061201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.062575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.063371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.107583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2604/5242] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/projects => generated 395 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:27.108745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2639/5243] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:27.109006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.110539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.257824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3686c-54c3-442a-a3f9-39e552345ad2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.263502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.264215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.264563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.264876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.265942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.267715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.270995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a71eb-1a14-48e2-9e5b-3f39b09f5514 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.278674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.279555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.279982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.280399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.281620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.284047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.322840 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2640/5244] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:27.325387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.329893 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2605/5245] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:27.337310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.351226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cc5b3-02a7-4c97-a157-256130cfc506 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.354278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e110b-3d8f-4e20-a954-1347529e857e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.364374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.364374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.364374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.364374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.365427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.365427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.365609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.366796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.368087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.368380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.369156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.375141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.377459 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2641/5246] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:27.381377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.389565 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2606/5247] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:27.396145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae243-bec4-4d62-9cb2-20c146048466 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.401729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.402006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.402806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.403175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.403675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.404550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.406484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.423002 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2642/5248] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:27.440242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68eaf8-bd1c-4ef7-9516-8fecbe2f321c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.442421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.452858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.453693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.454223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.454659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.455755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.457596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.470927 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2607/5249] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:27.478632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec4959-8aa9-49a4-84ea-1f8ef34159d7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.485284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.496077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5affb10-6e31-46e3-93c3-c371b72cc572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.497943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5affb10-6e31-46e3-93c3-c371b72cc572 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.497943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5affb10-6e31-46e3-93c3-c371b72cc572 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.497943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5affb10-6e31-46e3-93c3-c371b72cc572 admin admin] PATH_INFO: `/v3/projects/2d92e150d7e84937bb27f31b946301c0/users/0b1b834836eb4048b9c0e9812dcf71a3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.508754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c458ac-d788-4b1e-bd58-175b93bd95aa admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.515073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3fcd-ea1c-4ce7-9ca9-4d61dcf5ee6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.516403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3fcd-ea1c-4ce7-9ca9-4d61dcf5ee6e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.516889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3fcd-ea1c-4ce7-9ca9-4d61dcf5ee6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.517332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3fcd-ea1c-4ce7-9ca9-4d61dcf5ee6e admin admin] PATH_INFO: `/v3/projects/6f682e305e81438ba07d9898afbc7715/users/7bd9af81d21b4fd0a8dbc1df693a9f34/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.539251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e3fcd-ea1c-4ce7-9ca9-4d61dcf5ee6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f682e305e81438ba07d9898afbc7715, user_id=7bd9af81d21b4fd0a8dbc1df693a9f34, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.542871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5affb10-6e31-46e3-93c3-c371b72cc572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d92e150d7e84937bb27f31b946301c0, user_id=0b1b834836eb4048b9c0e9812dcf71a3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.543622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e3fcd-ea1c-4ce7-9ca9-4d61dcf5ee6e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.545865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5affb10-6e31-46e3-93c3-c371b72cc572 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.577001 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2608/5250] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:27 2022] PUT /identity/v3/projects/6f682e305e81438ba07d9898afbc7715/users/7bd9af81d21b4fd0a8dbc1df693a9f34/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:27.578764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.579053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.579331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.585628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2643/5251] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:27 2022] PUT /identity/v3/projects/2d92e150d7e84937bb27f31b946301c0/users/0b1b834836eb4048b9c0e9812dcf71a3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:27.587383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18017e-310d-40a5-8c7f-55689184040f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.587828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18017e-310d-40a5-8c7f-55689184040f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.588105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18017e-310d-40a5-8c7f-55689184040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.624694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] MFA Rules not processed for user `7bd9af81d21b4fd0a8dbc1df693a9f34`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:27.626926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:27.634932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8a18017e-310d-40a5-8c7f-55689184040f None None] MFA Rules not processed for user `0b1b834836eb4048b9c0e9812dcf71a3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:27.636540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8a18017e-310d-40a5-8c7f-55689184040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:27.641558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:27.715732 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2609/5252] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:27.730328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.741359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:27.746665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:27.754257 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2644/5253] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:27.760629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18017e-310d-40a5-8c7f-55689184040f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.845381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18017e-310d-40a5-8c7f-55689184040f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.850365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.851209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.851460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.851730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.852582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.854607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.865229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d6fcb-545b-4984-92aa-069b50299ee8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.869163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.869846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.870131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.870405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.892261 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2645/5254] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:27.896819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:27.961847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.963849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:27.974484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4fdf9-6819-4825-a32c-322596857f66 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:27.980177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2610/5255] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:27 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:27.985278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:27.987311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:27.987597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:27.987876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:27.988932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:27.990954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:28.030457 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2646/5256] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:27 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:28.035056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:28.051121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452a61-4b8b-4b67-9959-0cb0123a1738 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:28.062865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:28.063746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.064147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.064549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.065500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:28.068750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:28.084749 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2611/5257] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:28.086650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:28.098035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97d3df-8c6f-4e1e-be0c-ebe6d774ccd6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:28.104798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:28.105593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.105875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.106222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] PATH_INFO: `/v3/projects/ac9734b83e72476a9bd7fcd8dde4b8ee/users/79fc352d2c8e46e9ae582dbedfcfc4a2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.153964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9734b83e72476a9bd7fcd8dde4b8ee, user_id=79fc352d2c8e46e9ae582dbedfcfc4a2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:28.160774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:28.195078 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2647/5258] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:28 2022] PUT /identity/v3/projects/ac9734b83e72476a9bd7fcd8dde4b8ee/users/79fc352d2c8e46e9ae582dbedfcfc4a2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:28.200114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:28.218669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8153dd-0eb3-4657-9f8e-fc1589ca9650 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:28.284245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:28.357057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690076ec-9681-41b2-a1b5-46de522367e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:28.357765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690076ec-9681-41b2-a1b5-46de522367e2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.358225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690076ec-9681-41b2-a1b5-46de522367e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.358602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690076ec-9681-41b2-a1b5-46de522367e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.359619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690076ec-9681-41b2-a1b5-46de522367e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:28.361340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690076ec-9681-41b2-a1b5-46de522367e2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:28.374915 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2612/5259] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:28.379710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:28.391627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cc488-e993-47c1-9632-0d1fc69664af admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:28.420331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:28.422991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.423461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.424003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] PATH_INFO: `/v3/projects/ac9734b83e72476a9bd7fcd8dde4b8ee/users/79fc352d2c8e46e9ae582dbedfcfc4a2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.429864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9734b83e72476a9bd7fcd8dde4b8ee, user_id=79fc352d2c8e46e9ae582dbedfcfc4a2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:28.436235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:28.486654 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2648/5260] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:28 2022] PUT /identity/v3/projects/ac9734b83e72476a9bd7fcd8dde4b8ee/users/79fc352d2c8e46e9ae582dbedfcfc4a2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:28.490279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ecb95-20a2-4d13-8f18-f7a33468b5e0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.491953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ecb95-20a2-4d13-8f18-f7a33468b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.492835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ecb95-20a2-4d13-8f18-f7a33468b5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.580875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c67ecb95-20a2-4d13-8f18-f7a33468b5e0 None None] MFA Rules not processed for user `79fc352d2c8e46e9ae582dbedfcfc4a2`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:28.582011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c67ecb95-20a2-4d13-8f18-f7a33468b5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:28.754818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:28.766730 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2613/5261] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:28.772402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ecb95-20a2-4d13-8f18-f7a33468b5e0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:28.800992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ecb95-20a2-4d13-8f18-f7a33468b5e0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:28.807557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:28.808466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.808856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.809200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.809918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:28.812017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:28.840176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:28.852362 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2614/5262] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:28 2022] POST /identity/v3/projects => generated 334 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:28.856088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:28.868188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c5ebb-7116-4f45-8d08-c49c018c3b48 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:28.979708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:28.980617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cde67-8321-4e73-b18c-7d49a62b4939 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:28.981201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.981835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.982323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] PATH_INFO: `/v3/users/39321956af7646a8830f749c794274aa` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.984143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] RBAC: Authorizing `identity:delete_user(user_id=39321956af7646a8830f749c794274aa)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:28.986422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:28.987246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:28.987364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:28.987938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:28.988118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:28.988861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:28.991222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.085364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39321956af7646a8830f749c794274aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.087049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39321956af7646a8830f749c794274aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.092715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2615/5263] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:17:28 2022] POST /identity/v3/projects => generated 332 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:29.092715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.094104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:29.104516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39321956af7646a8830f749c794274aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.115917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39321956af7646a8830f749c794274aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.116878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39321956af7646a8830f749c794274aa'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.145662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:29.145867 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2649/5264] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:28 2022] DELETE /identity/v3/users/39321956af7646a8830f749c794274aa => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:29.148244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.150008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:29.198743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc0636-59ec-4f54-b71d-7d79b81810a1 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:29.261590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c68511-34e5-4b8b-8751-44363c2f22db admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:29.269233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabfd37-fd27-4e18-82ef-af50a2210aeb tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:29.269960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabfd37-fd27-4e18-82ef-af50a2210aeb tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.270272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabfd37-fd27-4e18-82ef-af50a2210aeb tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.270573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabfd37-fd27-4e18-82ef-af50a2210aeb tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.271242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabfd37-fd27-4e18-82ef-af50a2210aeb tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:29.274129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabfd37-fd27-4e18-82ef-af50a2210aeb tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.292841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:29.324145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:29.324910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.325189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.325467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.328565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:29.344036 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2650/5265] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:29 2022] POST /identity/v3/projects => generated 357 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:29.345553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:29.346016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.346372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.346708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.347888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.371231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:29.373185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:29.387824 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2616/5266] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:29.389238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.421385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710366cd-1045-48c1-b31b-e3c77bd75be3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:29.500242 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2651/5267] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:29.515081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.516808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:29.592544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:29.593203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.593482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.593769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.594456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:29.596466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.636281 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2617/5268] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:17:29 2022] GET /identity/v3/projects => generated 8267 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:29.641469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.654031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03824423-de0a-4c4e-ae6a-be1938d25d4f tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:29.659979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:29.660448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.660855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.661196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.662323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:29.662858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:29.665024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.671143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7dbc3-aaf7-45ba-afb5-e139d078c9f0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:29.679545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:29.680591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.681139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.681807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.691083 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2618/5269] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:17:29 2022] GET /identity/v3/projects?enabled=False => generated 445 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:29.692331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.696500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:29.699022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.705647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e8a1a-6d55-46ab-84d9-0e0f0890fa54 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:29.711239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:29.712078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.712357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.712630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.713371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Adding query filter params (parent_id=e7de1fdf61fb4266bfc3dcc3de49354d) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:29.713648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:29.715514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.723841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2652/5270] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:29.739302 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2619/5271] 10.222.0.26 () {58 vars in 1216 bytes} [Tue Sep 27 10:17:29 2022] GET /identity/v3/projects?parent_id=e7de1fdf61fb4266bfc3dcc3de49354d => generated 497 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:29.744421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.759559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c18744-04a5-4c11-bb22-6d2c21873b91 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:29.772840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:29.774039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:29.774397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:29.774882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects/e333c23e00d248c0b8a7f798f438bd14` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:29.777105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e333c23e00d248c0b8a7f798f438bd14)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:29.779702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:29.862257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e333c23e00d248c0b8a7f798f438bd14', 'request_id': 'req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.888840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e333c23e00d248c0b8a7f798f438bd14', 'request_id': 'req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.898745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e333c23e00d248c0b8a7f798f438bd14', 'request_id': 'req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.938638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] The token cache is being invalidate because project e333c23e00d248c0b8a7f798f438bd14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:29.938949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.941745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:29.948383 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2653/5272] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:29 2022] DELETE /identity/v3/projects/e333c23e00d248c0b8a7f798f438bd14 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:29.952290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:29.954643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:29.975183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc75e-f434-45b1-a604-b0ced70c2437 tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:30.085845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:30.104250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:30.105461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:30.105877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:30.106278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects/e7de1fdf61fb4266bfc3dcc3de49354d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:30.108938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e7de1fdf61fb4266bfc3dcc3de49354d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:30.112033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:30.241466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7de1fdf61fb4266bfc3dcc3de49354d', 'request_id': 'req-83a159be-acc5-4ad4-9a62-8e18d7bf347b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.267045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7de1fdf61fb4266bfc3dcc3de49354d', 'request_id': 'req-83a159be-acc5-4ad4-9a62-8e18d7bf347b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.288760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7de1fdf61fb4266bfc3dcc3de49354d', 'request_id': 'req-83a159be-acc5-4ad4-9a62-8e18d7bf347b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.296719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:30.298768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:30.323118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:30.340466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] The token cache is being invalidate because project e7de1fdf61fb4266bfc3dcc3de49354d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:30.341463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.342897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.345833 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2620/5273] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:29 2022] DELETE /identity/v3/projects/e7de1fdf61fb4266bfc3dcc3de49354d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:30.350255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:30.355311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:30.453388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69a03d-9aa5-4cf9-96e1-9a652abf166a tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:30.473619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a159be-acc5-4ad4-9a62-8e18d7bf347b tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:30.495429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79fc352d2c8e46e9ae582dbedfcfc4a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:30.498387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:30.498884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:30.499425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] PATH_INFO: `/v3/projects/de92fc9c5a374105a39a22874ab555cd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:30.506760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorizing `identity:delete_project(project_id=de92fc9c5a374105a39a22874ab555cd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:30.509229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:30.560580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:30.561370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:30.561659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:30.561928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] PATH_INFO: `/v3/projects/7aed84307e4d4a5ab5bd481e4752e5c4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:30.563147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aed84307e4d4a5ab5bd481e4752e5c4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:30.565366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:30.636721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de92fc9c5a374105a39a22874ab555cd', 'request_id': 'req-e99919dc-f111-4807-8636-b25a5fb5522e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.649638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de92fc9c5a374105a39a22874ab555cd', 'request_id': 'req-e99919dc-f111-4807-8636-b25a5fb5522e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.665790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de92fc9c5a374105a39a22874ab555cd', 'request_id': 'req-e99919dc-f111-4807-8636-b25a5fb5522e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.680694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aed84307e4d4a5ab5bd481e4752e5c4', 'request_id': 'req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.694058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] The token cache is being invalidate because project de92fc9c5a374105a39a22874ab555cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:30.694327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.695268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.697812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2621/5274] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:30 2022] DELETE /identity/v3/projects/de92fc9c5a374105a39a22874ab555cd => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:30.701822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aed84307e4d4a5ab5bd481e4752e5c4', 'request_id': 'req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.715814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aed84307e4d4a5ab5bd481e4752e5c4', 'request_id': 'req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.717300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:30.726094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:30.757421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] The token cache is being invalidate because project 7aed84307e4d4a5ab5bd481e4752e5c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:30.760273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.767054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:30.781872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:30.786210 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2654/5275] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:30 2022] DELETE /identity/v3/projects/7aed84307e4d4a5ab5bd481e4752e5c4 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:30.789622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:30.796716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:30.920534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99919dc-f111-4807-8636-b25a5fb5522e tempest-ListProjectsTestJSON-704943942 tempest-ListProjectsTestJSON-704943942-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:30.948683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:30.969165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181fa12-155e-43a2-9f4f-eefc4dc5a8d8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:30.975497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:30.976366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:30.979081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:30.979081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] PATH_INFO: `/v3/users/7bd9af81d21b4fd0a8dbc1df693a9f34` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:30.982452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd9af81d21b4fd0a8dbc1df693a9f34)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:30.984603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:31.002076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:31.002831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:31.003188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:31.003539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] PATH_INFO: `/v3/users/0b1b834836eb4048b9c0e9812dcf71a3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:31.004857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b1b834836eb4048b9c0e9812dcf71a3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:31.007262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:31.097354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b1b834836eb4048b9c0e9812dcf71a3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.098180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b1b834836eb4048b9c0e9812dcf71a3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.111286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b1b834836eb4048b9c0e9812dcf71a3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.114505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd9af81d21b4fd0a8dbc1df693a9f34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.117988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd9af81d21b4fd0a8dbc1df693a9f34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.128589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b1b834836eb4048b9c0e9812dcf71a3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.137131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd9af81d21b4fd0a8dbc1df693a9f34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.158788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd9af81d21b4fd0a8dbc1df693a9f34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.160066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd9af81d21b4fd0a8dbc1df693a9f34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.169611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-04d893df-4782-45d4-bced-8d52cd245833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b1b834836eb4048b9c0e9812dcf71a3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:31.188204 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2622/5276] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:30 2022] DELETE /identity/v3/users/0b1b834836eb4048b9c0e9812dcf71a3 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:31.192693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:31.192981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:31.194032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:31.211313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2655/5277] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:30 2022] DELETE /identity/v3/users/7bd9af81d21b4fd0a8dbc1df693a9f34 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:31.228020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:31.229381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:31.301775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:31.391045 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2623/5278] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:31.401629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:31.403834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:31.504604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584529a-7518-4165-88e0-2ea7a054a5d2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:31.509164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:31.509971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:31.510310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:31.511197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:31.530506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:31.532293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:31.549112 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2656/5279] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:31.969225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:31.970803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:31.988415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa61ecb-6e96-40fc-b814-d9566e22fe2e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:32.027586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:32.028399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:32.028703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:32.029073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] PATH_INFO: `/v3/projects/2d92e150d7e84937bb27f31b946301c0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:32.034566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d92e150d7e84937bb27f31b946301c0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:32.035567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:32.104227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d92e150d7e84937bb27f31b946301c0', 'request_id': 'req-f69c4b5e-a0eb-4793-9eb3-974c02245614'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.113453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d92e150d7e84937bb27f31b946301c0', 'request_id': 'req-f69c4b5e-a0eb-4793-9eb3-974c02245614'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.129091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d92e150d7e84937bb27f31b946301c0', 'request_id': 'req-f69c4b5e-a0eb-4793-9eb3-974c02245614'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.145538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] The token cache is being invalidate because project 2d92e150d7e84937bb27f31b946301c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:32.146117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.148238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.149985 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2624/5280] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:31 2022] DELETE /identity/v3/projects/2d92e150d7e84937bb27f31b946301c0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:32.154417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:32.166803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb7fd0-68d5-4130-8a2c-d8cafb90db07 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:32.243321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:32.244038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:32.244462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:32.244826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] PATH_INFO: `/v3/users/79fc352d2c8e46e9ae582dbedfcfc4a2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:32.246219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79fc352d2c8e46e9ae582dbedfcfc4a2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:32.249090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:32.269042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:32.326763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79fc352d2c8e46e9ae582dbedfcfc4a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.327644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79fc352d2c8e46e9ae582dbedfcfc4a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.349576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79fc352d2c8e46e9ae582dbedfcfc4a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.382842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79fc352d2c8e46e9ae582dbedfcfc4a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.383566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f176eed5-ab19-4122-8665-97d27007682b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79fc352d2c8e46e9ae582dbedfcfc4a2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.414149 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2657/5281] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:32 2022] DELETE /identity/v3/users/79fc352d2c8e46e9ae582dbedfcfc4a2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:32.446603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:32.455144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:32.489492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c4b5e-a0eb-4793-9eb3-974c02245614 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:32.589740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:32.591019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:32.591539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:32.591996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] PATH_INFO: `/v3/projects/6f682e305e81438ba07d9898afbc7715` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:32.593445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f682e305e81438ba07d9898afbc7715)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:32.595721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:32.686717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f682e305e81438ba07d9898afbc7715', 'request_id': 'req-9028cac2-1667-493f-a2d8-918e7d837586'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.708004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f682e305e81438ba07d9898afbc7715', 'request_id': 'req-9028cac2-1667-493f-a2d8-918e7d837586'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.728376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f682e305e81438ba07d9898afbc7715', 'request_id': 'req-9028cac2-1667-493f-a2d8-918e7d837586'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.737031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:32.756782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] The token cache is being invalidate because project 6f682e305e81438ba07d9898afbc7715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:32.757233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.758967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9028cac2-1667-493f-a2d8-918e7d837586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:32.761228 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2625/5282] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:32 2022] DELETE /identity/v3/projects/6f682e305e81438ba07d9898afbc7715 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:32.767946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e99a9-88e0-4abc-aaca-93612d243212 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:32.768298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e99a9-88e0-4abc-aaca-93612d243212 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:32.769326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e99a9-88e0-4abc-aaca-93612d243212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:32.815752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c33e99a9-88e0-4abc-aaca-93612d243212 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:32.818648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c33e99a9-88e0-4abc-aaca-93612d243212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:32.941030 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2658/5283] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:32.949350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12766c-59cf-4bce-b41a-3671637cecbd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:32.949918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12766c-59cf-4bce-b41a-3671637cecbd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:32.954163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12766c-59cf-4bce-b41a-3671637cecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:32.982862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6c12766c-59cf-4bce-b41a-3671637cecbd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:32.984389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6c12766c-59cf-4bce-b41a-3671637cecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:32.993609 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2626/5284] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:32.997813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e99a9-88e0-4abc-aaca-93612d243212 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.011046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e99a9-88e0-4abc-aaca-93612d243212 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:33.016749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:33.017865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.019129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.019560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.020529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:33.021003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:33.022979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:33.038082 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2659/5285] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:33.042301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12766c-59cf-4bce-b41a-3671637cecbd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.053173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12766c-59cf-4bce-b41a-3671637cecbd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:33.059280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:33.059912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.060379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.060504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.061189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:33.063934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:33.113760 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2627/5286] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:33 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:33.116528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.127942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb334-bf07-4e7a-903f-832c993aa657 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:33.177481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.178539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:33.249282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:33.250053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.250435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.250810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.251780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:33.254504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:33.303374 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2660/5287] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:33 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:33.303612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed49c-dd31-45c7-99ac-10ceb60280c8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:33.305358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.316873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:33.320050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:33.321147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.321538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.321907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] PATH_INFO: `/v3/projects/ac9734b83e72476a9bd7fcd8dde4b8ee` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.323331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9734b83e72476a9bd7fcd8dde4b8ee)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:33.327959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:33.333462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95346fda-6950-4e2c-9497-56a7638c8fa9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:33.348885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:33.350886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.353281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.355433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.356706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:33.358628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:33.377908 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2661/5288] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:33.379031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.394349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f569578-b23f-4218-9f5b-f0d16a1c0cc0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:33.402044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:33.402748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.403093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.403433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.404207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:33.405914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:33.420388 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2662/5289] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:33.426909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.441369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e318b-a57a-4b80-9f1f-d3da7a4f7b91 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:33.443828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'request_id': 'req-cf820f47-c712-48e7-a47c-72aa712e0080'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:33.462113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'request_id': 'req-cf820f47-c712-48e7-a47c-72aa712e0080'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:33.491251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9734b83e72476a9bd7fcd8dde4b8ee', 'request_id': 'req-cf820f47-c712-48e7-a47c-72aa712e0080'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:33.512369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] The token cache is being invalidate because project ac9734b83e72476a9bd7fcd8dde4b8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:33.512933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:33.514002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf820f47-c712-48e7-a47c-72aa712e0080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:33.516983 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2628/5290] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:33 2022] DELETE /identity/v3/projects/ac9734b83e72476a9bd7fcd8dde4b8ee => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:33.522352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315461-2b61-4701-8366-f51c76febc30 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.522643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315461-2b61-4701-8366-f51c76febc30 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.522975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315461-2b61-4701-8366-f51c76febc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.552752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-66315461-2b61-4701-8366-f51c76febc30 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:33.554069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66315461-2b61-4701-8366-f51c76febc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:33.625183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772a4e0-a76b-464d-8661-99b4a818178a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:33.626242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772a4e0-a76b-464d-8661-99b4a818178a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.626852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772a4e0-a76b-464d-8661-99b4a818178a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.626852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772a4e0-a76b-464d-8661-99b4a818178a admin admin] PATH_INFO: `/v3/projects/0c15672d950a46e8beeb0dbb915e9fc1/users/b8b49b3d4df545cca6a456707513f02c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.663406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772a4e0-a76b-464d-8661-99b4a818178a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c15672d950a46e8beeb0dbb915e9fc1, user_id=b8b49b3d4df545cca6a456707513f02c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:33.668525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772a4e0-a76b-464d-8661-99b4a818178a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:33.676876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:33.719209 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2663/5291] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:33 2022] PUT /identity/v3/projects/0c15672d950a46e8beeb0dbb915e9fc1/users/b8b49b3d4df545cca6a456707513f02c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:33.720900 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2629/5292] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:33.723933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb772136-184c-4eae-a645-859c2bee123c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.724367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb772136-184c-4eae-a645-859c2bee123c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.724628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb772136-184c-4eae-a645-859c2bee123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.730195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bdcc4-14e8-4379-8225-9ef99786a4bd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:33.730674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bdcc4-14e8-4379-8225-9ef99786a4bd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:33.731028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bdcc4-14e8-4379-8225-9ef99786a4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:33.778257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bb772136-184c-4eae-a645-859c2bee123c None None] MFA Rules not processed for user `b8b49b3d4df545cca6a456707513f02c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:33.779831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb772136-184c-4eae-a645-859c2bee123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:33.781066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8d6bdcc4-14e8-4379-8225-9ef99786a4bd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:33.787452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8d6bdcc4-14e8-4379-8225-9ef99786a4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:33.949818 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2664/5293] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:33.951409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb772136-184c-4eae-a645-859c2bee123c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:33.954262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bb772136-184c-4eae-a645-859c2bee123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:33.993315 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2630/5294] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:34.001055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bdcc4-14e8-4379-8225-9ef99786a4bd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.016567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bdcc4-14e8-4379-8225-9ef99786a4bd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.021765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.021765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.021765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.022247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.023941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:34.023941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.030988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.058665 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2631/5295] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:34.064550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.080891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8379872-d4b3-41f8-aa56-0ed3237c95f3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.090550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.091774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.092238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.092621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.093951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.098003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.147747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb772136-184c-4eae-a645-859c2bee123c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.160506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.165711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.166186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.166519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.181084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.182985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.188290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2632/5296] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:34 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:34.193669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:34.195301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.200752 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2665/5297] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:34.354679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68b85-aa3c-4594-8397-fd1eda30068a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.366228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.367727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:34.376581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.378352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.380338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.381354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.385111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.385111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.389865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858b68-dc6f-4d9a-abe4-4da6ef0bc5d6 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.406931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.407696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.408280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.408668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] PATH_INFO: `/v3/users/b8b49b3d4df545cca6a456707513f02c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.410229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b49b3d4df545cca6a456707513f02c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.414705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.457060 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2633/5298] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:34 2022] POST /identity/v3/users => generated 284 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:34.457060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.480316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3877e1-5ae7-486b-8319-5efb3c769d4c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.492562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.493385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.493762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.494179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.495228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.497100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.511015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2634/5299] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:34.514211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b49b3d4df545cca6a456707513f02c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:34.515109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.515234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b49b3d4df545cca6a456707513f02c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:34.517534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:34.526835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b49b3d4df545cca6a456707513f02c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:34.535812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64f66e-4a3c-42fc-8f90-109155155e0d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.536542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b49b3d4df545cca6a456707513f02c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:34.537419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3bc86c1-d39e-46a5-9f81-cb09c1aa6fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b49b3d4df545cca6a456707513f02c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:34.542401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.542824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.543126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.543383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.544039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.551837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.564514 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2635/5300] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:34.570610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2666/5301] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:34 2022] DELETE /identity/v3/users/b8b49b3d4df545cca6a456707513f02c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:34.572832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.577160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.577445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.577713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.585248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc59b5-c4bd-452f-b773-8c17ba926bac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.604923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:34.607800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:34.615306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:34.674419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d195c8e-4ab5-4e74-a4bd-44280693ddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.675377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d195c8e-4ab5-4e74-a4bd-44280693ddb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.675809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d195c8e-4ab5-4e74-a4bd-44280693ddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.676081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d195c8e-4ab5-4e74-a4bd-44280693ddb1 admin admin] PATH_INFO: `/v3/projects/1436e33748d643aaba576b88c464ce70/users/e9ac333809634f3e808a98d6708bdcc0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.692035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d195c8e-4ab5-4e74-a4bd-44280693ddb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1436e33748d643aaba576b88c464ce70, user_id=e9ac333809634f3e808a98d6708bdcc0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:34.695103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d195c8e-4ab5-4e74-a4bd-44280693ddb1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:34.701814 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2667/5302] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:34.714305 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2636/5303] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:34 2022] PUT /identity/v3/projects/1436e33748d643aaba576b88c464ce70/users/e9ac333809634f3e808a98d6708bdcc0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:34.714538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.715770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:34.725348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552244b-370f-46bf-bedb-7dd164a61734 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.725646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552244b-370f-46bf-bedb-7dd164a61734 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.726280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552244b-370f-46bf-bedb-7dd164a61734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:34.758066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c552244b-370f-46bf-bedb-7dd164a61734 None None] MFA Rules not processed for user `e9ac333809634f3e808a98d6708bdcc0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:34.759699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c552244b-370f-46bf-bedb-7dd164a61734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:34.825745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce187eb-bda4-4afe-8640-0931bcb29f6a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:34.864125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:34.906975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:34.907183 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2637/5304] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:34.907553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:34.907860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:34.908075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552244b-370f-46bf-bedb-7dd164a61734 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:34.908162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.002405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.004304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.004836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552244b-370f-46bf-bedb-7dd164a61734 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.014499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.015840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.016637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.016965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.017610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.020071 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2668/5305] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:35.029042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.079817 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2638/5306] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:34 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:35.081937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.094246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ac945-a0ea-41cd-a048-965ce4c6a1bf service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.181476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.182608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.183154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.183567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.185068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.188044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.228715 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2669/5307] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:35 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:35.233988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.252560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bfeb-e517-4406-a6e1-700d6bd4ac85 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.263152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.263895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.264095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.264341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.265046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.266813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.328001 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2639/5308] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:35.332361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.347119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192802e-ca22-48db-aa87-57da6fef2ebd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.354010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.355043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.355427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.356131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] PATH_INFO: `/v3/projects/b17c3ba2d9a14979a424f3c8306376ab/users/afda9a4e4b454518b3715ba2623fa43e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.377566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17c3ba2d9a14979a424f3c8306376ab, user_id=afda9a4e4b454518b3715ba2623fa43e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.382162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.406317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2670/5309] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:35 2022] PUT /identity/v3/projects/b17c3ba2d9a14979a424f3c8306376ab/users/afda9a4e4b454518b3715ba2623fa43e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:35.410814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.424131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e39dc2-5030-4179-9c59-1f9486b677d7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.438806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:35.509104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee681f-3936-4d76-9447-72b50993957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.510561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee681f-3936-4d76-9447-72b50993957f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.511294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee681f-3936-4d76-9447-72b50993957f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.512056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee681f-3936-4d76-9447-72b50993957f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.513009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee681f-3936-4d76-9447-72b50993957f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.515230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee681f-3936-4d76-9447-72b50993957f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.535218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2640/5310] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:35.540151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.554698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcc182-2d68-40a9-9cc9-eedb327a8563 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.571477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.572639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.573260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.573711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] PATH_INFO: `/v3/projects/b17c3ba2d9a14979a424f3c8306376ab/users/afda9a4e4b454518b3715ba2623fa43e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.577603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17c3ba2d9a14979a424f3c8306376ab, user_id=afda9a4e4b454518b3715ba2623fa43e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.582390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.611683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2671/5311] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:35 2022] PUT /identity/v3/projects/b17c3ba2d9a14979a424f3c8306376ab/users/afda9a4e4b454518b3715ba2623fa43e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:35.614284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93ed01-4bdc-4384-932c-6c86d2171d17 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.615009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93ed01-4bdc-4384-932c-6c86d2171d17 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.615685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93ed01-4bdc-4384-932c-6c86d2171d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.683463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6e93ed01-4bdc-4384-932c-6c86d2171d17 None None] MFA Rules not processed for user `afda9a4e4b454518b3715ba2623fa43e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:35.684730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6e93ed01-4bdc-4384-932c-6c86d2171d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:35.730663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.733096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:35.803358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:35.817155 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2641/5312] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:35.819694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93ed01-4bdc-4384-932c-6c86d2171d17 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.834993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93ed01-4bdc-4384-932c-6c86d2171d17 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.840905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.841620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.841978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.842328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.843720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.846342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.855055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45f83e-65d9-4e94-997d-f58800254d97 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.861937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:35.862590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:35.863333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:35.863333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] PATH_INFO: `/v3/projects/0c15672d950a46e8beeb0dbb915e9fc1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:35.864651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c15672d950a46e8beeb0dbb915e9fc1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:35.869153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:35.872750 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2642/5313] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:17:35 2022] POST /identity/v3/domains => generated 275 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:35.876466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:35.953365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085a0b4-f2fb-4ca6-8653-d0ebbdc0d623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:35.959613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c15672d950a46e8beeb0dbb915e9fc1', 'request_id': 'req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:35.969446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c15672d950a46e8beeb0dbb915e9fc1', 'request_id': 'req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:35.980762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c15672d950a46e8beeb0dbb915e9fc1', 'request_id': 'req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:35.992870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:35.993796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] The token cache is being invalidate because project 0c15672d950a46e8beeb0dbb915e9fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:35.993946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:35.994750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50ae0ff1-10bb-43b3-8ae5-79ff21c2f6db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:35.996715 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2672/5314] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:35 2022] DELETE /identity/v3/projects/0c15672d950a46e8beeb0dbb915e9fc1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:36.002329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdbb89-80ad-4afa-88ae-f48c8e76adcd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.002605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdbb89-80ad-4afa-88ae-f48c8e76adcd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.002879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdbb89-80ad-4afa-88ae-f48c8e76adcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.029521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-34cdbb89-80ad-4afa-88ae-f48c8e76adcd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:36.031320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34cdbb89-80ad-4afa-88ae-f48c8e76adcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:36.051971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.052846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.053402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.053780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.054862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.057394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.097331 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2643/5315] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:35 2022] POST /identity/v3/users => generated 345 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:36.101351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.103043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:36.118532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafd5ac-5d5e-48df-959b-54fe5df7a500 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.124804 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2673/5316] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:36.125705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.126335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.126617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.126891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.127718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.129583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b9e2-a140-467a-9bdb-a0e1b9d8c5e3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.129712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.130125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b9e2-a140-467a-9bdb-a0e1b9d8c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.130652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b9e2-a140-467a-9bdb-a0e1b9d8c5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.153591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b1b3b9e2-a140-467a-9bdb-a0e1b9d8c5e3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:36.154994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b1b3b9e2-a140-467a-9bdb-a0e1b9d8c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:36.159592 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2644/5317] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:36 2022] POST /identity/v3/users => generated 321 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:36.163651 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2674/5318] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:36.164133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.167551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b9e2-a140-467a-9bdb-a0e1b9d8c5e3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.176245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc366a5-866b-4753-a40b-c3b82481219f tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.177734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b9e2-a140-467a-9bdb-a0e1b9d8c5e3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.181434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.182087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.182347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.182619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users/6936a7f308064398bf721a71781cfdf3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.182914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.183642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.184047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.184417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.185206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:36.185605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.187316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.200565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2675/5319] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:36.202877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:get_user(user_id=6936a7f308064398bf721a71781cfdf3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.205191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.205470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.207224 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2645/5320] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/users/6936a7f308064398bf721a71781cfdf3 => generated 345 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:36.212786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.217272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b767d6-9a6d-4273-8eda-d9768973c493 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.222101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.222820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.223199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.223627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.224025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f65b4-cdc8-4f66-95b9-15945d450ba8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.224503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.226556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.228791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.229782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.229782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.230094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.231341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Adding query filter params (domain_id=f15aa512faff46e89d9f2f98a9fc230d) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:36.231341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.232197 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:17:36.232197 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:17:36.232197 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:17:36.233957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.249806 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2646/5321] 10.222.0.26 () {58 vars in 1207 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/users?domain_id=f15aa512faff46e89d9f2f98a9fc230d => generated 482 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:36.255072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.266240 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2676/5322] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:36 2022] POST /identity/v3/projects => generated 353 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:36.267494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee040-bb9a-410d-b8cf-fd3dced6dc55 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.270131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.302925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:36.360459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.361141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.361508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.361821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.362626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.364422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.408722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084a665-f7f4-45d4-95eb-784edfa6e964 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.409291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.410046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.410531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.410891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.411646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.415001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.448708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2677/5323] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:36 2022] POST /identity/v3/users => generated 277 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:36.452618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.465443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3c999-8baf-4cf3-b1a3-efcb4da32f58 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.472140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.473228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.473731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.474119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.475720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.477707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.493069 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2647/5324] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/users => generated 8833 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:36.498288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.504067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2678/5325] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:36.508523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.513457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721973f-69f9-4734-9f54-448124fdd623 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.520352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.521016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.521581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.521945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.522653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Adding query filter params (name=tempest-test_user-739128488) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:36.523061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.525304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.541195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d0635-1ddc-4fca-bd19-7de876be4a42 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.561644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.562540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.563232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.573200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2648/5326] 10.222.0.26 () {58 vars in 1187 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/users?name=tempest-test_user-739128488 => generated 472 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:36.579141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.583993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.584839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.592005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.598073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a502216-b886-422e-849c-68523bf340b2 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.617142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2679/5327] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:36.625560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.637345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018830-4ef9-4b22-87c8-7bcdcb5e5ad8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.644225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e8cb9-7d0d-4b10-9c11-8842ded5d10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.645100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e8cb9-7d0d-4b10-9c11-8842ded5d10f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.645379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e8cb9-7d0d-4b10-9c11-8842ded5d10f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.645654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e8cb9-7d0d-4b10-9c11-8842ded5d10f admin admin] PATH_INFO: `/v3/projects/84395a87c3ca4a0bb99fcf668eeeb388/users/d71888b49e004e019002ac8d6686d233/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.659689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.660342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.660629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.660937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.666015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e8cb9-7d0d-4b10-9c11-8842ded5d10f admin admin] RBAC: Authorizing `identity:create_grant(project_id=84395a87c3ca4a0bb99fcf668eeeb388, user_id=d71888b49e004e019002ac8d6686d233, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.668147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:36.668676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.670619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e8cb9-7d0d-4b10-9c11-8842ded5d10f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.682383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:36.713362 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2680/5328] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:36 2022] PUT /identity/v3/projects/84395a87c3ca4a0bb99fcf668eeeb388/users/d71888b49e004e019002ac8d6686d233/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:36.725399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad2e5a-af29-477d-b5eb-122888d43030 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.725922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad2e5a-af29-477d-b5eb-122888d43030 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.726520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad2e5a-af29-477d-b5eb-122888d43030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.746301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2649/5329] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 10:17:36 2022] GET /identity/v3/users?enabled=False => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:36.752469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.775543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dcad2e5a-af29-477d-b5eb-122888d43030 None None] MFA Rules not processed for user `d71888b49e004e019002ac8d6686d233`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:36.777642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dcad2e5a-af29-477d-b5eb-122888d43030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:36.789412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81617f-0831-45ed-b21c-1c95816c3d8e tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:36.831878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:36.901338 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2681/5330] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:36.905722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad2e5a-af29-477d-b5eb-122888d43030 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:36.919359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:36.920401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:36.921655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:36.922089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users/cd7db49951e042b38707b2413e74bf58` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:36.951728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:delete_user(user_id=cd7db49951e042b38707b2413e74bf58)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:36.954341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.020120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:37.038940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad2e5a-af29-477d-b5eb-122888d43030 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.039250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7db49951e042b38707b2413e74bf58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.040220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7db49951e042b38707b2413e74bf58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.044692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.044692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.044984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.045343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.046811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.048703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.055296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7db49951e042b38707b2413e74bf58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.071361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7db49951e042b38707b2413e74bf58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.089391 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2682/5331] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:36 2022] POST /identity/v3/projects => generated 357 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:37.091656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7db49951e042b38707b2413e74bf58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.093243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.096435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.106170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2650/5332] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:36 2022] DELETE /identity/v3/users/cd7db49951e042b38707b2413e74bf58 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:37.107565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.109966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.180943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46a9c4-830f-427d-af49-03a52fbc09fc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.211222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ff15-ba61-4dc5-9fcb-8e3f3b535a9b tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.219426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.220322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.220796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.221216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/users/6936a7f308064398bf721a71781cfdf3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.222826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6936a7f308064398bf721a71781cfdf3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.235192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.289650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.290298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.290577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.293540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.294405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.296627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.318274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:37.323651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6936a7f308064398bf721a71781cfdf3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.325285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6936a7f308064398bf721a71781cfdf3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.331209 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2683/5333] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:37 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:37.335889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.340188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.344557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6936a7f308064398bf721a71781cfdf3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.367265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6936a7f308064398bf721a71781cfdf3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.385801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6936a7f308064398bf721a71781cfdf3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.389955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9920b-0810-4b54-96c4-3e9430b8fe22 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.400734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.403537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.404512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.404907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.406721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.412544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.414175 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2651/5334] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:37 2022] DELETE /identity/v3/users/6936a7f308064398bf721a71781cfdf3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:37.418097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.426156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.446410 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2684/5335] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:37.447559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.463372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ccb24-4d14-4781-bd09-b9f329ab3057 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.518597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa53f-3a19-46b6-bce7-22ee845effe4 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.526221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.527176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.527509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.527855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/domains/f15aa512faff46e89d9f2f98a9fc230d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.529332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f15aa512faff46e89d9f2f98a9fc230d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.531712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.538909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.539750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.541793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.541793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] PATH_INFO: `/v3/projects/f81d84c4379b4d9eb8c12ae5caec790f/users/c2f9956cd42d4d3aaa01f875df754e22/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.544822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:37.560046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81d84c4379b4d9eb8c12ae5caec790f, user_id=c2f9956cd42d4d3aaa01f875df754e22, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.563011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.563254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.564159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.582328 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2685/5336] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:37 2022] PUT /identity/v3/projects/f81d84c4379b4d9eb8c12ae5caec790f/users/c2f9956cd42d4d3aaa01f875df754e22/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:37.586617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.588996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.599091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.600695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.606160 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2652/5337] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:17:37 2022] PATCH /identity/v3/domains/f15aa512faff46e89d9f2f98a9fc230d => generated 276 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:37.607350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.609113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.666125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d882ac-1e03-402c-b938-4756918b6eaa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.671592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.672217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.672534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.673062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.673721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.675376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.695006 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2686/5338] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:37.699697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:37.702677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.718252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222746c-1ec2-4452-af4b-9e4c5a5147d1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.723841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5afea-8887-49e8-b796-2830fcf85753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.724398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58184819-cc8a-4fc3-8383-e9af02663ca8 tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:37.724963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5afea-8887-49e8-b796-2830fcf85753 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.725521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5afea-8887-49e8-b796-2830fcf85753 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.725966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5afea-8887-49e8-b796-2830fcf85753 admin admin] PATH_INFO: `/v3/projects/f81d84c4379b4d9eb8c12ae5caec790f/users/c2f9956cd42d4d3aaa01f875df754e22/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.728524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5afea-8887-49e8-b796-2830fcf85753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81d84c4379b4d9eb8c12ae5caec790f, user_id=c2f9956cd42d4d3aaa01f875df754e22, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.733287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5afea-8887-49e8-b796-2830fcf85753 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.757872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2687/5339] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:37 2022] PUT /identity/v3/projects/f81d84c4379b4d9eb8c12ae5caec790f/users/c2f9956cd42d4d3aaa01f875df754e22/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:37.771386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb073e-ca83-4693-8e7c-1de8077202ae None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.771386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb073e-ca83-4693-8e7c-1de8077202ae None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.771386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb073e-ca83-4693-8e7c-1de8077202ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.779793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:37.824879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1fdb073e-ca83-4693-8e7c-1de8077202ae None None] MFA Rules not processed for user `c2f9956cd42d4d3aaa01f875df754e22`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:37.826633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1fdb073e-ca83-4693-8e7c-1de8077202ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:37.863104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afda9a4e4b454518b3715ba2623fa43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b17c3ba2d9a14979a424f3c8306376ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:37.865120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:37.874437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:37.875226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] PATH_INFO: `/v3/domains/f15aa512faff46e89d9f2f98a9fc230d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:37.889632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f15aa512faff46e89d9f2f98a9fc230d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:37.891052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:37.943601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.951874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:37.969983 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2688/5340] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:38.064167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d', 'request_id': 'req-69bf0c1d-132f-4f41-a635-a628d6fda1de'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.078182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb073e-ca83-4693-8e7c-1de8077202ae None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:38.091264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb073e-ca83-4693-8e7c-1de8077202ae None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:38.112157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d', 'request_id': 'req-69bf0c1d-132f-4f41-a635-a628d6fda1de'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.136502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d', 'request_id': 'req-69bf0c1d-132f-4f41-a635-a628d6fda1de'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.155427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] The token cache is being invalidate because project f15aa512faff46e89d9f2f98a9fc230d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:38.156491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.162098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.181050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d', 'request_id': 'req-69bf0c1d-132f-4f41-a635-a628d6fda1de'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.181318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:38.182240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d', 'request_id': 'req-69bf0c1d-132f-4f41-a635-a628d6fda1de'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.189450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f15aa512faff46e89d9f2f98a9fc230d', 'request_id': 'req-69bf0c1d-132f-4f41-a635-a628d6fda1de'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.199201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f9956cd42d4d3aaa01f875df754e22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f81d84c4379b4d9eb8c12ae5caec790f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:38.200855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:38.202045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:38.202937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:38.203757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:38.205289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:38.207852 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2653/5341] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:17:37 2022] DELETE /identity/v3/domains/f15aa512faff46e89d9f2f98a9fc230d => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:38.212216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:38.214368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:38.217566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:38.264292 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2689/5342] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:38.266021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:38.270315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:38.288541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a57c4-5a3d-4fce-b8ac-547374b12466 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:38.295116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f9956cd42d4d3aaa01f875df754e22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f81d84c4379b4d9eb8c12ae5caec790f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:38.296358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:38.296921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:38.298038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:38.298931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:38.303054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:38.370985 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2690/5343] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:38 2022] POST /identity/v3/projects => generated 348 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:38.372260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:38.390186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf0c1d-132f-4f41-a635-a628d6fda1de tempest-UsersV3TestJSON-839981125 tempest-UsersV3TestJSON-839981125-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:38.426245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:38.475789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4b3ae-24a0-4039-9d41-e028c656fec1 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:38.482649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f9956cd42d4d3aaa01f875df754e22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f81d84c4379b4d9eb8c12ae5caec790f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:38.483403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:38.483687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:38.483959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:38.484823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:38.485130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:38.486968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:38.505255 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2691/5344] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:38.512568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:38.514229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:38.515193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:38.515610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] PATH_INFO: `/v3/users/e9ac333809634f3e808a98d6708bdcc0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:38.517660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ac333809634f3e808a98d6708bdcc0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:38.520447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:38.533517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:38.535229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:38.617817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ac333809634f3e808a98d6708bdcc0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.618508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ac333809634f3e808a98d6708bdcc0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.626018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ac333809634f3e808a98d6708bdcc0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.639532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ac333809634f3e808a98d6708bdcc0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.656373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2af47401-2156-49e6-b9e6-f7ddde153ec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ac333809634f3e808a98d6708bdcc0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:38.664911 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2654/5345] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:38 2022] DELETE /identity/v3/users/e9ac333809634f3e808a98d6708bdcc0 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:38.669767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:38.670168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:38.670559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:38.670863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca353e46-9a17-4088-8435-6edb9b8b68bc tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:38.696762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:38.697789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:38.750225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:38.750939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:38.751214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:38.751914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:38.754432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:38.775550 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2655/5346] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:38.786671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:38.789727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:38.798207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:38.818057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c096991-093d-42ca-a9ff-387b7f818969 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:38.821971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:38.822588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:38.822866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:38.823134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:38.834804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:38.836582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:38.845747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:38.847510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:38.859400 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2656/5348] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:38.859666 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2692/5348] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:39.048543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:39.060744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a82cd-a599-457a-a2bc-30ee1fa1e3f8 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:39.065908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f9956cd42d4d3aaa01f875df754e22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f81d84c4379b4d9eb8c12ae5caec790f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:39.066068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:39.066352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:39.066630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] PATH_INFO: `/v3/projects/7a190d2b0f0b43259215786920684725` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:39.067753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7a190d2b0f0b43259215786920684725)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:39.069785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:39.096939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:39.124413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a190d2b0f0b43259215786920684725', 'request_id': 'req-9faaff50-bb86-4b10-919a-3cbc87083d44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.131133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a190d2b0f0b43259215786920684725', 'request_id': 'req-9faaff50-bb86-4b10-919a-3cbc87083d44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.140820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a190d2b0f0b43259215786920684725', 'request_id': 'req-9faaff50-bb86-4b10-919a-3cbc87083d44'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.170180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:39.171840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:39.178057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] The token cache is being invalidate because project 7a190d2b0f0b43259215786920684725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:39.178760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.179504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.187015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8eecf-275f-4648-b9a1-88638562aaac service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:39.187715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2657/5349] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:39 2022] DELETE /identity/v3/projects/7a190d2b0f0b43259215786920684725 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:39.283654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:39.285273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:39.285637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:39.285992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] PATH_INFO: `/v3/projects/1436e33748d643aaba576b88c464ce70` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:39.287388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1436e33748d643aaba576b88c464ce70)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:39.289615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:39.300028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:39.301598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:39.317526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaff50-bb86-4b10-919a-3cbc87083d44 tempest-QuotasTest-1875515434 tempest-QuotasTest-1875515434-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:39.323552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:39.325941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:39.326821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:39.327542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] PATH_INFO: `/v3/users/d71888b49e004e019002ac8d6686d233` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:39.328618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d71888b49e004e019002ac8d6686d233)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:39.331464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:39.413614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1436e33748d643aaba576b88c464ce70', 'request_id': 'req-537977d7-cd18-45ae-900b-d732eef23b13'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.431209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1436e33748d643aaba576b88c464ce70', 'request_id': 'req-537977d7-cd18-45ae-900b-d732eef23b13'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.439342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1436e33748d643aaba576b88c464ce70', 'request_id': 'req-537977d7-cd18-45ae-900b-d732eef23b13'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.461879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd71888b49e004e019002ac8d6686d233'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.462714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd71888b49e004e019002ac8d6686d233'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.476400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] The token cache is being invalidate because project 1436e33748d643aaba576b88c464ce70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:39.476833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.479333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.481332 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2693/5350] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:39 2022] DELETE /identity/v3/projects/1436e33748d643aaba576b88c464ce70 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:39.482495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd71888b49e004e019002ac8d6686d233'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.485348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:39.492018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:39.509918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd71888b49e004e019002ac8d6686d233'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.533025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6cf0e9ec-dc25-454b-9328-007efb6a7c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd71888b49e004e019002ac8d6686d233'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.545380 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2658/5351] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:39 2022] DELETE /identity/v3/users/d71888b49e004e019002ac8d6686d233 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:39.550249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:39.550622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:39.550950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:39.616312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:39.620060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:39.628954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:39.637025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537977d7-cd18-45ae-900b-d732eef23b13 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:39.756847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2659/5352] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:39.758704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:39.759581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:39.759869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:39.760200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] PATH_INFO: `/v3/users/afda9a4e4b454518b3715ba2623fa43e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:39.761385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afda9a4e4b454518b3715ba2623fa43e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:39.768607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:39.772837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:39.774368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:39.834830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:39.914269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afda9a4e4b454518b3715ba2623fa43e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.915040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afda9a4e4b454518b3715ba2623fa43e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.919592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135e43a-7e65-4c3a-a9b8-9e8a27afa12f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:39.929455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:39.930923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:39.931293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:39.931698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:39.933165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afda9a4e4b454518b3715ba2623fa43e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.949514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:39.961729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afda9a4e4b454518b3715ba2623fa43e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.965527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afda9a4e4b454518b3715ba2623fa43e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:39.984763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:39.986423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:40.026862 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2660/5353] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:40.038905 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2694/5354] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:39 2022] DELETE /identity/v3/users/afda9a4e4b454518b3715ba2623fa43e => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:40.629300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:40.631305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:40.656438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff702c24-6b46-448f-8674-dabbbb758d3d service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:40.763704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:40.766147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:40.781616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:40.782824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:40.782973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:40.783408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] PATH_INFO: `/v3/projects/b17c3ba2d9a14979a424f3c8306376ab` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:40.785129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17c3ba2d9a14979a424f3c8306376ab)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:40.787712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:40.894865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17c3ba2d9a14979a424f3c8306376ab', 'request_id': 'req-605c2505-7493-4dcb-af59-6a7a3ca3648e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:40.906212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17c3ba2d9a14979a424f3c8306376ab', 'request_id': 'req-605c2505-7493-4dcb-af59-6a7a3ca3648e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:40.926650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17c3ba2d9a14979a424f3c8306376ab', 'request_id': 'req-605c2505-7493-4dcb-af59-6a7a3ca3648e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:40.940559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] The token cache is being invalidate because project b17c3ba2d9a14979a424f3c8306376ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:40.940857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:40.941560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-605c2505-7493-4dcb-af59-6a7a3ca3648e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:40.942420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963ac5a-0b7b-426b-bc35-2fc57bcacad3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:40.948071 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2661/5355] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:40 2022] DELETE /identity/v3/projects/b17c3ba2d9a14979a424f3c8306376ab => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:40.953562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138be4-7bd8-43f1-88b3-35d717c02f88 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:40.953845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138be4-7bd8-43f1-88b3-35d717c02f88 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:40.954124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138be4-7bd8-43f1-88b3-35d717c02f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:40.991116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-21138be4-7bd8-43f1-88b3-35d717c02f88 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:40.993216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-21138be4-7bd8-43f1-88b3-35d717c02f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:41.017926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:41.090067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:41.101209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.101813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.102093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.102378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] PATH_INFO: `/v3/projects/84395a87c3ca4a0bb99fcf668eeeb388` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.103502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84395a87c3ca4a0bb99fcf668eeeb388)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.105729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:41.113374 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2662/5356] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:41.122238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dbbc0-205b-4b24-8769-743f0e5dd63b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.122605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dbbc0-205b-4b24-8769-743f0e5dd63b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.122952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dbbc0-205b-4b24-8769-743f0e5dd63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.147306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e94dbbc0-205b-4b24-8769-743f0e5dd63b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:41.149604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e94dbbc0-205b-4b24-8769-743f0e5dd63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:41.173943 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2663/5357] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:41.179102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dbbc0-205b-4b24-8769-743f0e5dd63b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:41.199653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dbbc0-205b-4b24-8769-743f0e5dd63b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:41.204119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84395a87c3ca4a0bb99fcf668eeeb388', 'request_id': 'req-12ea5b79-8f72-4488-b73f-f439a6f16123'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.223910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84395a87c3ca4a0bb99fcf668eeeb388', 'request_id': 'req-12ea5b79-8f72-4488-b73f-f439a6f16123'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.237082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84395a87c3ca4a0bb99fcf668eeeb388', 'request_id': 'req-12ea5b79-8f72-4488-b73f-f439a6f16123'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.258193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] The token cache is being invalidate because project 84395a87c3ca4a0bb99fcf668eeeb388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:41.258602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.259613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.265799 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2695/5358] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:40 2022] DELETE /identity/v3/projects/84395a87c3ca4a0bb99fcf668eeeb388 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:41.268905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:41.271412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:41.354916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.355547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.355821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.356090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.356782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:41.357111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.359224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:41.406828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2664/5359] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:41.413138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:41.415513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:41.428737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea5b79-8f72-4488-b73f-f439a6f16123 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:41.441624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.443563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d529bee-d3b9-44be-a65c-0662ba153a22 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:41.443764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.459645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.460038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] PATH_INFO: `/v3/users/c2f9956cd42d4d3aaa01f875df754e22` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.462232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f9956cd42d4d3aaa01f875df754e22)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.464619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:41.465170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.465928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.466215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.466492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.467176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.476499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:41.541212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2665/5360] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:41 2022] POST /identity/v3/projects => generated 397 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:41.542361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:41.555208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f9956cd42d4d3aaa01f875df754e22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.556357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f9956cd42d4d3aaa01f875df754e22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.569912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f9956cd42d4d3aaa01f875df754e22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.594756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:41.597439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f9956cd42d4d3aaa01f875df754e22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.599694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b846054b-9f54-4173-96f9-aebf00339c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f9956cd42d4d3aaa01f875df754e22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:41.647085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635775-8e75-4dba-b7ba-acd80283ad21 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:41.675087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.675920 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2696/5361] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:41 2022] DELETE /identity/v3/users/c2f9956cd42d4d3aaa01f875df754e22 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:41.676034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.676034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.676034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.676034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.679297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:41.715581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4025-b79e-45e1-8ec5-c38826502253 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.715862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4025-b79e-45e1-8ec5-c38826502253 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.716136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4025-b79e-45e1-8ec5-c38826502253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.722307 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2666/5362] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:41 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:41.731084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:41.735878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:41.757358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74419f1-768d-4f0d-bfe6-95ca90063cea admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:41.765268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7a9e4025-b79e-45e1-8ec5-c38826502253 None None] MFA Rules not processed for user `dbf876e1c23642578363f9c5995f3d86`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:41.766618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7a9e4025-b79e-45e1-8ec5-c38826502253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:41.834578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:41.854955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.856202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.856555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.857720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.858126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.861916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:41.881406 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2667/5363] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:41.881620 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2697/5364] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:41.883621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:41.895066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10500242-72a1-4aa0-989e-916883451558 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:41.900815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2dd76-7dcc-4c2e-bc1f-bda1c4065335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.901839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dd76-7dcc-4c2e-bc1f-bda1c4065335 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.902178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dd76-7dcc-4c2e-bc1f-bda1c4065335 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.902428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dd76-7dcc-4c2e-bc1f-bda1c4065335 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.903171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2dd76-7dcc-4c2e-bc1f-bda1c4065335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.905054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2dd76-7dcc-4c2e-bc1f-bda1c4065335 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:41.921228 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2668/5365] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:41.922869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e4025-b79e-45e1-8ec5-c38826502253 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:41.950620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e4025-b79e-45e1-8ec5-c38826502253 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:41.957522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:41.961723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:41.962090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:41.962433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] PATH_INFO: `/v3/projects/26e722c6139c432cabec29fc90f098c1/users/6a2cb64197e244849cac6ea29000d216/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:41.983447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e722c6139c432cabec29fc90f098c1, user_id=6a2cb64197e244849cac6ea29000d216, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:41.987548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.011594 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2698/5366] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:41 2022] PUT /identity/v3/projects/26e722c6139c432cabec29fc90f098c1/users/6a2cb64197e244849cac6ea29000d216/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:42.023587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb256b-c35c-4c57-9d0d-dc59e7fa885a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.024423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb256b-c35c-4c57-9d0d-dc59e7fa885a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.024423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb256b-c35c-4c57-9d0d-dc59e7fa885a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.184923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.187113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:42.249608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-aecb256b-c35c-4c57-9d0d-dc59e7fa885a None None] MFA Rules not processed for user `6a2cb64197e244849cac6ea29000d216`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:42.251450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aecb256b-c35c-4c57-9d0d-dc59e7fa885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:42.297236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1f1fe-0613-4653-bd2a-45f06e05d695 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.304492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.305545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.305545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.305773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] PATH_INFO: `/v3/projects/f81d84c4379b4d9eb8c12ae5caec790f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.306835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81d84c4379b4d9eb8c12ae5caec790f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.309364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.337280 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2669/5367] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:42.341400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb256b-c35c-4c57-9d0d-dc59e7fa885a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.354794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb256b-c35c-4c57-9d0d-dc59e7fa885a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.389330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81d84c4379b4d9eb8c12ae5caec790f', 'request_id': 'req-04862a8c-894a-4028-bbef-f5631bdcfb60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:42.400019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81d84c4379b4d9eb8c12ae5caec790f', 'request_id': 'req-04862a8c-894a-4028-bbef-f5631bdcfb60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:42.406208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81d84c4379b4d9eb8c12ae5caec790f', 'request_id': 'req-04862a8c-894a-4028-bbef-f5631bdcfb60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:42.408203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:42.421030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] The token cache is being invalidate because project f81d84c4379b4d9eb8c12ae5caec790f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:42.421473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:42.421957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-04862a8c-894a-4028-bbef-f5631bdcfb60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:42.423563 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2699/5368] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:42 2022] DELETE /identity/v3/projects/f81d84c4379b4d9eb8c12ae5caec790f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:42.429930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9336635-0572-4916-81e3-65bacafa02fe None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.431245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9336635-0572-4916-81e3-65bacafa02fe None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.431533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9336635-0572-4916-81e3-65bacafa02fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.432030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.432793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.433170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.433541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.434296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.436597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.453129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a9336635-0572-4916-81e3-65bacafa02fe None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:42.455307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a9336635-0572-4916-81e3-65bacafa02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:42.465855 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2700/5369] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:42.474121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2670/5370] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:42 2022] POST /identity/v3/projects => generated 397 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:42.475550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92eb82-67c2-487f-a0de-8817e8b4acec None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.475838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92eb82-67c2-487f-a0de-8817e8b4acec None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.476339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92eb82-67c2-487f-a0de-8817e8b4acec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.477125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.479463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:42.500271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bf92eb82-67c2-487f-a0de-8817e8b4acec None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:42.501519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bf92eb82-67c2-487f-a0de-8817e8b4acec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:42.561847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:42.584710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2644e99-9f43-44e5-97f6-7710e7ec8155 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.607567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.608373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.609026 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2701/5371] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:42.609150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.609339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.613365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.613621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92eb82-67c2-487f-a0de-8817e8b4acec None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.617901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.633811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92eb82-67c2-487f-a0de-8817e8b4acec None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.640149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.641094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.641518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.641969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.648113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:42.648113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.648113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.656027 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2671/5372] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:42 2022] POST /identity/v3/users => generated 305 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:42.657472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.676070 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2702/5373] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:42.676495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ddd2b-e7f3-4d20-a901-aeb86c074db0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.677667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.689218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59f08f-cf5f-4f49-848d-5f55df133294 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.690429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.695327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.695792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.696063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.696581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.697638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.700209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.704161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.705529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.706916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.712868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.720397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.746446 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2703/5374] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:42 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:42.747000 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2672/5375] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:42.750341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.752939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:42.776135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ea5d8-4937-4fb1-ac45-eecf5a6dd897 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.899974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.901894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.912773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.913222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] PATH_INFO: `/v3/projects/0dd5452984f7495e8584095ce7003402/users/12b8c5793dc948e8bf53a8dac867f6f4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.916050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e8315-663e-4c6a-8537-004751bda556 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:42.928282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:42.929049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:42.933744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:42.934061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:42.934732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.942661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:42.972754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd5452984f7495e8584095ce7003402, user_id=12b8c5793dc948e8bf53a8dac867f6f4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:42.977395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:43.002312 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2704/5376] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:42 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:43.002502 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2673/5377] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:42 2022] PUT /identity/v3/projects/0dd5452984f7495e8584095ce7003402/users/12b8c5793dc948e8bf53a8dac867f6f4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:43.002627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.004055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.006680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:43.058625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc5b22-bafc-44c7-8c84-852acf196af5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:43.159820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:43.195299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419d59b-fec0-4d76-aa7b-ea770dc93b45 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:43.201825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:43.203459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.203583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.204121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.205176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:43.207027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:43.221388 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2674/5378] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:43.222341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.229283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:43.232273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.232995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.236459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] PATH_INFO: `/v3/users/eea2a6c4a4f049ebba996287df6ef7a5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.239212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea2a6c4a4f049ebba996287df6ef7a5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:43.240165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebbb7c-a979-4d13-b739-80a8c7aab5f4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:43.244455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:43.246489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:43.247514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.247639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.247941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.248706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:43.251551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:43.294805 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2675/5379] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:43.294805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.308134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78d1d-ab13-48b2-ad41-fb7a40c3b228 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:43.322971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:43.326478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.326906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.329088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] PATH_INFO: `/v3/projects/0dd5452984f7495e8584095ce7003402/users/12b8c5793dc948e8bf53a8dac867f6f4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.341723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd5452984f7495e8584095ce7003402, user_id=12b8c5793dc948e8bf53a8dac867f6f4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:43.346691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:43.361197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea2a6c4a4f049ebba996287df6ef7a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:43.365513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea2a6c4a4f049ebba996287df6ef7a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:43.376218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2676/5380] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:43 2022] PUT /identity/v3/projects/0dd5452984f7495e8584095ce7003402/users/12b8c5793dc948e8bf53a8dac867f6f4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:43.377879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.379874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:43.384714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea2a6c4a4f049ebba996287df6ef7a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:43.397295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea2a6c4a4f049ebba996287df6ef7a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:43.398093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-61aa2f84-957d-4d54-9afd-388fce3833c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea2a6c4a4f049ebba996287df6ef7a5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:43.446871 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2705/5381] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:42 2022] DELETE /identity/v3/users/eea2a6c4a4f049ebba996287df6ef7a5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:43.449841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f91d-9186-4d11-ac90-a44d2253b39f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.450139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f91d-9186-4d11-ac90-a44d2253b39f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.450426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f91d-9186-4d11-ac90-a44d2253b39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.459728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:43.527126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7a26f91d-9186-4d11-ac90-a44d2253b39f None None] MFA Rules not processed for user `12b8c5793dc948e8bf53a8dac867f6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:43.529656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7a26f91d-9186-4d11-ac90-a44d2253b39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:43.570444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb406c-fbd9-44fc-83da-81400f6a844b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:43.704353 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2706/5382] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:43.707595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa973ec0-5196-4c05-ab0a-7261c53150e2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.719822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa973ec0-5196-4c05-ab0a-7261c53150e2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.721042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa973ec0-5196-4c05-ab0a-7261c53150e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.771868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:43.772545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.772834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.773183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.773905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:43.775508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:43.788415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2677/5383] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:43.789358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.790790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:43.811807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aa973ec0-5196-4c05-ab0a-7261c53150e2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:43.812827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa973ec0-5196-4c05-ab0a-7261c53150e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:43.843322 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2707/5384] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:43.844188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa973ec0-5196-4c05-ab0a-7261c53150e2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.857327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa973ec0-5196-4c05-ab0a-7261c53150e2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:43.871021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:43.871747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.872035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.872291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] PATH_INFO: `/v3/projects/642d881cee9343deb91c1cffb84568ba/users/ed44be35b33a4a7a83ced9086d21cb02/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.887977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279aa92-b770-4839-ae77-22fb0ee5823a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:43.898569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2cb64197e244849cac6ea29000d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e722c6139c432cabec29fc90f098c1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:43.899395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:43.899515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:43.899795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:43.900636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:43.904853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642d881cee9343deb91c1cffb84568ba, user_id=ed44be35b33a4a7a83ced9086d21cb02, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:43.907600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Sep 27 10:17:43.912550 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 27 10:17:43.913595 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:43.914173 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2678/5385] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:17:43 2022] POST /identity/v3/projects => generated 136 bytes in 125 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:17:43.920179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.922754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:43.944764 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2708/5386] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:43 2022] PUT /identity/v3/projects/642d881cee9343deb91c1cffb84568ba/users/ed44be35b33a4a7a83ced9086d21cb02/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:43.944764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:43.969706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:44.060648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77e249-c4a3-4b90-adfe-04cc6110c67a tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:44.121590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd21f-0621-4371-81b3-d8f56bc03133 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:44.129287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:44.135703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37329f-b993-4aa3-9e47-01540d8f66d1 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12b8c5793dc948e8bf53a8dac867f6f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dd5452984f7495e8584095ce7003402', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:44.136484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37329f-b993-4aa3-9e47-01540d8f66d1 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.136803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37329f-b993-4aa3-9e47-01540d8f66d1 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.137191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37329f-b993-4aa3-9e47-01540d8f66d1 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.137832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37329f-b993-4aa3-9e47-01540d8f66d1 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:44.142721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37329f-b993-4aa3-9e47-01540d8f66d1 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-1f37329f-b993-4aa3-9e47-01540d8f66d1 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Invalid input for field 'name': '' is too short Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: On instance['name']: Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: On instance['name']: Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: '' Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise error Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application '' Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:44.160494 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:17:44.162823 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application '' Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:17:44.164056 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2709/5387] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:17:43 2022] POST /identity/v3/projects => generated 283 bytes in 225 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:17:44.168780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99a2a4-f7c6-453c-a387-48266f49c62e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.170047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99a2a4-f7c6-453c-a387-48266f49c62e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.170297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99a2a4-f7c6-453c-a387-48266f49c62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.192205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:44.196416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.196718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.197172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.310586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ae99a2a4-f7c6-453c-a387-48266f49c62e None None] MFA Rules not processed for user `ed44be35b33a4a7a83ced9086d21cb02`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:44.314343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ae99a2a4-f7c6-453c-a387-48266f49c62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:44.442607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:44.445266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:44.473195 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2679/5388] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:44.475899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:44.513249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f195b8b-258f-41b0-8e7d-2e8bdb8a5365 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:44.520222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12b8c5793dc948e8bf53a8dac867f6f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dd5452984f7495e8584095ce7003402', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:44.526982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.527663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.528552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.542539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:44.544175 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2710/5389] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:44.551182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99a2a4-f7c6-453c-a387-48266f49c62e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:44.555683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:44.582676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99a2a4-f7c6-453c-a387-48266f49c62e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:44.589700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:44.590818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.591374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.591961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.592814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:44.595304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: On instance['name']: Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: On instance['name']: Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise error Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:17:44.605949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:44.607425 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:17:44.608709 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.615190 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2680/5390] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:17:44 2022] POST /identity/v3/projects => generated 412 bytes in 139 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:17:44.616330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:44.651488 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2711/5391] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:44 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:44.658824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:44.659828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9418e-611e-4de2-a6e6-f252f54881aa tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:44.822640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12b8c5793dc948e8bf53a8dac867f6f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dd5452984f7495e8584095ce7003402', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:44.826751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.831376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.832008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130fd6d-a517-43c9-9b11-da3925aa531d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:44.832482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] PATH_INFO: `/v3/projects/32f518d658484aa69fdb8167baf66476` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.850987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:44.861875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.862804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.863203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorizing `identity:delete_project(project_id=32f518d658484aa69fdb8167baf66476)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:44.865690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:44.867471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.870165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:44.879053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Could not find project: 32f518d658484aa69fdb8167baf66476.: keystone.exception.ProjectNotFound: Could not find project: 32f518d658484aa69fdb8167baf66476. Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 27 10:17:44.888328 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 32f518d658484aa69fdb8167baf66476. Sep 27 10:17:44.889949 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:44.901230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2681/5392] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:44 2022] DELETE /identity/v3/projects/32f518d658484aa69fdb8167baf66476 => generated 113 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:17:44.903538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:44.943908 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2712/5393] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:44 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:44.947524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:44.970309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d787f-80d6-43d5-8bed-a423d144403d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:44.978287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:44.979663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:44.979663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:44.979876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:44.980742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:44.982678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:44.992297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:45.024407 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2713/5394] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:45.026270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.040100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff50f3-2785-4ec3-8559-7c8921cd6554 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.047152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7cf30-1adb-42e2-a59f-7bf3812fab92 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.050055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2cb64197e244849cac6ea29000d216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26e722c6139c432cabec29fc90f098c1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.050931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.051216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.051495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.052153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.057245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.058058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.058680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.058966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] PATH_INFO: `/v3/projects/98a7351f5f4344e29eacf1c98e642f93/users/877a76fa20df40ceb8f9855b11b6d976/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Sep 27 10:17:45.066349 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._list_projects() Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.068341 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2682/5395] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:17:44 2022] GET /identity/v3/projects => generated 135 bytes in 166 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:17:45.073785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.103121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b53c7-b2fd-445a-8013-bf332e7c2610 tempest-ProjectsNegativeStaticTestJSON-1667127307 tempest-ProjectsNegativeStaticTestJSON-1667127307-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.114072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12b8c5793dc948e8bf53a8dac867f6f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dd5452984f7495e8584095ce7003402', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.114236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.114421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.114692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.115666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.116944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a7351f5f4344e29eacf1c98e642f93, user_id=877a76fa20df40ceb8f9855b11b6d976, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.117247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:45.121880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:45.149130 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2714/5396] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:45 2022] PUT /identity/v3/projects/98a7351f5f4344e29eacf1c98e642f93/users/877a76fa20df40ceb8f9855b11b6d976/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:45.153441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.157232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:45.204081 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2683/5397] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:45 2022] POST /identity/v3/projects => generated 357 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:45.205522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.310296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df3300-8434-47ff-98b0-f50f6f16862f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.408402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9014b2f-cd38-4c94-a920-88c94f327291 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.426034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12b8c5793dc948e8bf53a8dac867f6f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dd5452984f7495e8584095ce7003402', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.427077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.427571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.427909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.429257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.431492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:45.452040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.453971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.454335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.454709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.456539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.478450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.sql.core [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-345126658' for key 'project.ixu_project_name_domain_id'") Sep 27 10:17:45.478450 ncdv-master-1 devstack@keystone.service[104131]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 10:17:45.478450 ncdv-master-1 devstack@keystone.service[104131]: [parameters: {'id': '31232ddd03294bc19678c14577fd2a27', 'name': 'tempest-project-dup-345126658', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 27 10:17:45.478450 ncdv-master-1 devstack@keystone.service[104131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 10:17:45.487898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:45.517274 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2715/5398] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:45.524939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-345126658, project id 31232ddd03294bc19678c14577fd2a27.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-345126658, project id 31232ddd03294bc19678c14577fd2a27. Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application conn.query(q) Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 10:17:45.529490 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result.read() Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-345126658' for key 'project.ixu_project_name_domain_id'") Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 10:17:45.530697 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application yield resource Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise self.value Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 10:17:45.532047 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application session.commit() Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.flush() Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application compat.raise_( Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 10:17:45.534331 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 10:17:45.535655 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application conn.query(q) Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result.read() Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 10:17:45.537145 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-345126658' for key 'project.ixu_project_name_domain_id'") Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [parameters: {'id': '31232ddd03294bc19678c14577fd2a27', 'name': 'tempest-project-dup-345126658', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:45.538170 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-345126658 at domain ID default. Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:17:45.539389 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-345126658, project id 31232ddd03294bc19678c14577fd2a27. Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2684/5399] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:17:45 2022] POST /identity/v3/projects => generated 284 bytes in 331 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 27 10:17:45.541699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.558298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc4663-06ca-446e-8f56-4b41a6f6a159 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.566172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d161e2d-1422-467f-8d49-a400405992de tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.574428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4ce4a-8d24-474a-af80-fc925abba3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.575525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4ce4a-8d24-474a-af80-fc925abba3d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.575846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4ce4a-8d24-474a-af80-fc925abba3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.578789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4ce4a-8d24-474a-af80-fc925abba3d3 admin admin] PATH_INFO: `/v3/projects/98a7351f5f4344e29eacf1c98e642f93/users/877a76fa20df40ceb8f9855b11b6d976/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.583253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12b8c5793dc948e8bf53a8dac867f6f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dd5452984f7495e8584095ce7003402', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.583784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4ce4a-8d24-474a-af80-fc925abba3d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a7351f5f4344e29eacf1c98e642f93, user_id=877a76fa20df40ceb8f9855b11b6d976, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.586558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.586960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.587726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] PATH_INFO: `/v3/projects/e64602c941124b569195a60246d3f0e9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.590417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e64602c941124b569195a60246d3f0e9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.593462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4ce4a-8d24-474a-af80-fc925abba3d3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:45.595684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:45.645375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2716/5400] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:45 2022] PUT /identity/v3/projects/98a7351f5f4344e29eacf1c98e642f93/users/877a76fa20df40ceb8f9855b11b6d976/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:45.662213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.667326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.667571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.725324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] MFA Rules not processed for user `877a76fa20df40ceb8f9855b11b6d976`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:45.733341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64602c941124b569195a60246d3f0e9', 'request_id': 'req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:45.742918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64602c941124b569195a60246d3f0e9', 'request_id': 'req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:45.745408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:45.762837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64602c941124b569195a60246d3f0e9', 'request_id': 'req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:45.786495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] The token cache is being invalidate because project e64602c941124b569195a60246d3f0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:45.787892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:45.788734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:45.790677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2685/5401] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:45 2022] DELETE /identity/v3/projects/e64602c941124b569195a60246d3f0e9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:45.792186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.796468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:45.866438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:45.898402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaad7f-8fd4-47f7-b3a6-33e06ea2e310 tempest-ProjectsNegativeStaticTestJSON-1378647133 tempest-ProjectsNegativeStaticTestJSON-1378647133-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.903886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.904796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.905115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.905466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] PATH_INFO: `/v3/projects/64dd9a15966148b88e42e312d29f0e55` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.906842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64dd9a15966148b88e42e312d29f0e55)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.908018 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2717/5402] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:45.909254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:45.912532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:45.913922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:45.923078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:45.927763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078916-c569-4cd3-a228-cfdc61336154 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:45.933663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:45.934329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:45.934614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:45.934913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] PATH_INFO: `/v3/users/6a2cb64197e244849cac6ea29000d216` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:45.936141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a2cb64197e244849cac6ea29000d216)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:45.938126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:46.002134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64dd9a15966148b88e42e312d29f0e55', 'request_id': 'req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.010633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a2cb64197e244849cac6ea29000d216'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.011882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a2cb64197e244849cac6ea29000d216'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.014819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64dd9a15966148b88e42e312d29f0e55', 'request_id': 'req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.026460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64dd9a15966148b88e42e312d29f0e55', 'request_id': 'req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.030922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a2cb64197e244849cac6ea29000d216'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.043326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a2cb64197e244849cac6ea29000d216'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.044087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a2cb64197e244849cac6ea29000d216'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.044723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] The token cache is being invalidate because project 64dd9a15966148b88e42e312d29f0e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:46.045026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.046041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.048217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2686/5403] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:45 2022] DELETE /identity/v3/projects/64dd9a15966148b88e42e312d29f0e55 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:46.049814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:46.053027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:46.080809 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2718/5404] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:45 2022] DELETE /identity/v3/users/6a2cb64197e244849cac6ea29000d216 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:46.082201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:46.086497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:46.159423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fa4e4-dc9d-4e2c-aac0-2c1e048a7ea4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:46.184242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09978c-8aa7-4a68-a85d-855642825869 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:46.190508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:46.191223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:46.191504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:46.191828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] PATH_INFO: `/v3/users/3ade97345d214139aa8408fc712a617f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:46.193762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ade97345d214139aa8408fc712a617f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:46.196101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:46.245944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:46.246574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:46.246852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:46.247155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:46.249450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:46.270989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ade97345d214139aa8408fc712a617f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.272177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ade97345d214139aa8408fc712a617f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.285408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ade97345d214139aa8408fc712a617f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.297661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ade97345d214139aa8408fc712a617f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.298789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e9ce263-c3ee-4fdb-948a-096198b552de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ade97345d214139aa8408fc712a617f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:46.302694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:46.337032 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2719/5405] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:46 2022] DELETE /identity/v3/users/3ade97345d214139aa8408fc712a617f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:46.340973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:46.341471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:46.342799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:46.357296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:46.359338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:46.371750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:46.373966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:46.376392 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2687/5406] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:46.463412 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2720/5407] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:46.490684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:46.499928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:46.509498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bb895-09b1-4bca-ad40-8956a7d9187a service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:46.621323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e1bb7-faca-4ea0-81dc-5e14749b801f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:46.621968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e1bb7-faca-4ea0-81dc-5e14749b801f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:46.622357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e1bb7-faca-4ea0-81dc-5e14749b801f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:46.622869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e1bb7-faca-4ea0-81dc-5e14749b801f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:46.649009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e1bb7-faca-4ea0-81dc-5e14749b801f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:46.650466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e1bb7-faca-4ea0-81dc-5e14749b801f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:46.664177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2688/5408] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:47.780267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:47.781991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:47.806651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac1ef1-ed35-41d9-ae25-2e31961dc2ec None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:47.818788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:47.819957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:47.819957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:47.820142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] PATH_INFO: `/v3/projects/942313e3fc534b8996fecaa6f48a80a9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:47.830950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942313e3fc534b8996fecaa6f48a80a9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:47.834123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:47.919726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942313e3fc534b8996fecaa6f48a80a9', 'request_id': 'req-9482aa3f-c36e-42a8-ad73-f2a70e066655'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:47.932580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942313e3fc534b8996fecaa6f48a80a9', 'request_id': 'req-9482aa3f-c36e-42a8-ad73-f2a70e066655'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:47.944253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942313e3fc534b8996fecaa6f48a80a9', 'request_id': 'req-9482aa3f-c36e-42a8-ad73-f2a70e066655'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:47.967439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] The token cache is being invalidate because project 942313e3fc534b8996fecaa6f48a80a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:47.970225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:47.988999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:47.997016 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2721/5409] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:47 2022] DELETE /identity/v3/projects/942313e3fc534b8996fecaa6f48a80a9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:47.998966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e6c1-f0e8-4f65-a838-e185ce3abe0e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:47.999462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e6c1-f0e8-4f65-a838-e185ce3abe0e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:47.999550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e6c1-f0e8-4f65-a838-e185ce3abe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.054299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9d63e6c1-f0e8-4f65-a838-e185ce3abe0e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:48.055543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9d63e6c1-f0e8-4f65-a838-e185ce3abe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:48.060623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.069978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:48.190568 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2689/5410] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:48.198980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428c0f-fd10-44b1-ac2e-6e9aec7d239e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.199294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428c0f-fd10-44b1-ac2e-6e9aec7d239e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.199494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428c0f-fd10-44b1-ac2e-6e9aec7d239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.235629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e1428c0f-fd10-44b1-ac2e-6e9aec7d239e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:48.239049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e1428c0f-fd10-44b1-ac2e-6e9aec7d239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:48.258401 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2690/5411] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:48.262627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428c0f-fd10-44b1-ac2e-6e9aec7d239e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.278430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428c0f-fd10-44b1-ac2e-6e9aec7d239e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.293016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.293650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.293937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.294203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.294865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:48.295162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.297005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.309998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482aa3f-c36e-42a8-ad73-f2a70e066655 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.332051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.334225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.335220 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2691/5412] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:48.338376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.338516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] PATH_INFO: `/v3/projects/26e722c6139c432cabec29fc90f098c1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.340369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.344826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e722c6139c432cabec29fc90f098c1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.348817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.370269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37fa8a-c3da-4e46-ae6f-58d2333ff84e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.380909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:48.392755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.399583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.400032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.403910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.403910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.408786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.471706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e722c6139c432cabec29fc90f098c1', 'request_id': 'req-b7430500-4848-42bf-a656-8fcc147bcf4a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.471909 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2692/5413] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:48 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:48.477973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.497969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e722c6139c432cabec29fc90f098c1', 'request_id': 'req-b7430500-4848-42bf-a656-8fcc147bcf4a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.515662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e722c6139c432cabec29fc90f098c1', 'request_id': 'req-b7430500-4848-42bf-a656-8fcc147bcf4a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.543416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] The token cache is being invalidate because project 26e722c6139c432cabec29fc90f098c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:48.543915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.556389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.565645 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2722/5414] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:48 2022] DELETE /identity/v3/projects/26e722c6139c432cabec29fc90f098c1 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:48.570025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.572800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:48.606630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cd4b0-cb8e-4659-9e73-e28ecaa68cbb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.628389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.628850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.629397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.629983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.631281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.633584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.666626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:48.699454 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2693/5415] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:48 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:48.700532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.706518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:48.739185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8f26c-af1a-4365-8d31-1745d6076426 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.753292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.754109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.754265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.754788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.756817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.758926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.764757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7430500-4848-42bf-a656-8fcc147bcf4a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.773665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.774326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.774918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.775197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] PATH_INFO: `/v3/users/12b8c5793dc948e8bf53a8dac867f6f4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.776368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b8c5793dc948e8bf53a8dac867f6f4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.777769 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2694/5416] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:48.778854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.781550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.793633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f1c08-5a74-407e-a37c-2f3c20426753 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.799876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.801216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.801599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.801937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.802983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.805265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.843396 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2695/5417] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:48.844848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:48.862696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3f11b-ecd7-43e7-9829-0690fc9bc14d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:48.869820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013491e-ab80-40e1-a22b-b29ce45a7a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:48.870654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013491e-ab80-40e1-a22b-b29ce45a7a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.871037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013491e-ab80-40e1-a22b-b29ce45a7a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.871857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013491e-ab80-40e1-a22b-b29ce45a7a2d admin admin] PATH_INFO: `/v3/projects/db124c1623c148a5be49241f606003e1/users/26d236859f77471a8e17f3a5462756de/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:48.901836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b8c5793dc948e8bf53a8dac867f6f4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.903103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b8c5793dc948e8bf53a8dac867f6f4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.929851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013491e-ab80-40e1-a22b-b29ce45a7a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db124c1623c148a5be49241f606003e1, user_id=26d236859f77471a8e17f3a5462756de, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:48.935493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b8c5793dc948e8bf53a8dac867f6f4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.942130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013491e-ab80-40e1-a22b-b29ce45a7a2d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:48.948804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b8c5793dc948e8bf53a8dac867f6f4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.951891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b8c5793dc948e8bf53a8dac867f6f4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:48.976240 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2696/5418] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:48 2022] PUT /identity/v3/projects/db124c1623c148a5be49241f606003e1/users/26d236859f77471a8e17f3a5462756de/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:48.990562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14159-12cc-4bdc-95c3-446b92a6cbba None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:48.992821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14159-12cc-4bdc-95c3-446b92a6cbba None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:48.994124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14159-12cc-4bdc-95c3-446b92a6cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:49.004034 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2723/5419] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:48 2022] DELETE /identity/v3/users/12b8c5793dc948e8bf53a8dac867f6f4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:49.044245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7bb14159-12cc-4bdc-95c3-446b92a6cbba None None] MFA Rules not processed for user `26d236859f77471a8e17f3a5462756de`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:49.045862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bb14159-12cc-4bdc-95c3-446b92a6cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:49.182030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2697/5420] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:49.196489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:49.198734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:49.285567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:49.355921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ca035-4b68-4d15-95dc-ada2028a7461 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:49.362494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:49.363581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:49.364630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:49.365158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:49.372913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05139f31-c409-4fa7-bfee-3f978415d2ab None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:49.373196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05139f31-c409-4fa7-bfee-3f978415d2ab None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:49.373473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05139f31-c409-4fa7-bfee-3f978415d2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:49.378744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:49.380436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:49.393690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2724/5421] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:49.421174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:49.422557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-05139f31-c409-4fa7-bfee-3f978415d2ab None None] MFA Rules not processed for user `a7b8599d0b40424087e52d8e35b71ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:49.422682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:49.424285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-05139f31-c409-4fa7-bfee-3f978415d2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:49.517968 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2698/5422] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:49.563246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892d26-4692-4c4d-8383-26e14df94f99 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:49.573024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:49.574034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:49.574695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:49.575224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] PATH_INFO: `/v3/users/26d236859f77471a8e17f3a5462756de` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:49.577881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26d236859f77471a8e17f3a5462756de)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:49.580551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:49.678917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26d236859f77471a8e17f3a5462756de'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:49.680035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26d236859f77471a8e17f3a5462756de'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:49.692447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26d236859f77471a8e17f3a5462756de'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:49.703144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26d236859f77471a8e17f3a5462756de'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:49.705086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26d236859f77471a8e17f3a5462756de'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:49.734686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:49.735101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:49.735486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:49.740803 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2725/5423] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:49 2022] DELETE /identity/v3/users/26d236859f77471a8e17f3a5462756de => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:49.764152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:49.765453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:49.851225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:49.899735 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2699/5424] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:49.947334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:49.952718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.042822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:50.078598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556ecb-2f69-4aed-9577-33037f8ba936 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:50.086023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:50.087571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:50.088007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:50.089031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:50.118284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:50.123887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:50.147083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:50.149865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.155775 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2726/5425] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:50.275373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414fa69-e7cf-4c61-a4bf-b33f84e31912 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:50.283374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:50.285239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:50.285627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:50.286934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:50.288306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.312646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:50.315778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:50.336821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2700/5426] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:50.371679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:50.374486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.397169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8fd4b-9428-4c39-bc44-6bf2e48d27f2 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:50.404289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098acfa-cd94-4f61-b8d7-a44df73f6d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:50.405069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098acfa-cd94-4f61-b8d7-a44df73f6d97 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:50.405386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:50.405512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098acfa-cd94-4f61-b8d7-a44df73f6d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:50.405993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098acfa-cd94-4f61-b8d7-a44df73f6d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:50.406898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.409459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8098acfa-cd94-4f61-b8d7-a44df73f6d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.429617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d346-d633-4d22-8252-9efde36318f0 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:50.433587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098acfa-cd94-4f61-b8d7-a44df73f6d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:50.443995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:50.444189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:50.444412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:50.444628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] PATH_INFO: `/v3/projects/0dd5452984f7495e8584095ce7003402` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:50.446564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd5452984f7495e8584095ce7003402)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:50.448624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:50.457482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098acfa-cd94-4f61-b8d7-a44df73f6d97 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:50.494414 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2727/5427] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:50.568154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd5452984f7495e8584095ce7003402', 'request_id': 'req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:50.587513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd5452984f7495e8584095ce7003402', 'request_id': 'req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:50.612465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd5452984f7495e8584095ce7003402', 'request_id': 'req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:50.641976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] The token cache is being invalidate because project 0dd5452984f7495e8584095ce7003402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:50.642992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:50.644413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e539d9d-ee0e-4e6a-99d2-716c9be792f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:50.649599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2701/5428] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:50 2022] DELETE /identity/v3/projects/0dd5452984f7495e8584095ce7003402 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:50.656072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998632b-bd6e-4bd1-8806-e916bcc3ec98 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:50.661106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998632b-bd6e-4bd1-8806-e916bcc3ec98 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:50.663977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998632b-bd6e-4bd1-8806-e916bcc3ec98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:50.727216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3998632b-bd6e-4bd1-8806-e916bcc3ec98 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:50.730333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3998632b-bd6e-4bd1-8806-e916bcc3ec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.867947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2728/5429] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:50.877775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91736b0e-84ce-4291-a296-7b63158e61d6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:50.878091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91736b0e-84ce-4291-a296-7b63158e61d6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:50.880272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91736b0e-84ce-4291-a296-7b63158e61d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:50.917863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-91736b0e-84ce-4291-a296-7b63158e61d6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:50.919596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-91736b0e-84ce-4291-a296-7b63158e61d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:50.931879 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2702/5430] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:50.933128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998632b-bd6e-4bd1-8806-e916bcc3ec98 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:50.943314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998632b-bd6e-4bd1-8806-e916bcc3ec98 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:50.971140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:50.972035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:50.972400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:50.972929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:50.974198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:50.975824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:50.977578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.001938 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2729/5431] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:51.007190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91736b0e-84ce-4291-a296-7b63158e61d6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:51.019820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91736b0e-84ce-4291-a296-7b63158e61d6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:51.043248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:51.043879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.044183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.044448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.045702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:51.047771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.103029 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2703/5432] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:51 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:51.107184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:51.127235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4231df4-fbd8-45f9-a694-ff4cb51fa64c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:51.235512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:51.236171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.236436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.236738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.238296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:51.241066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.269308 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2730/5433] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:51 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:51.273409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:51.294545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff864f44-4a4a-4e26-9368-9d6156b2eb60 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:51.302675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:51.303228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.303506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.304059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.305113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:51.307338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.326836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2704/5434] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:51.329493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:51.355313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3c946-98fc-4238-982f-66900343e920 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:51.375627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a13603-a238-43f9-bea9-2499d31f5137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:51.376443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a13603-a238-43f9-bea9-2499d31f5137 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.376849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a13603-a238-43f9-bea9-2499d31f5137 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.377943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a13603-a238-43f9-bea9-2499d31f5137 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.377943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a13603-a238-43f9-bea9-2499d31f5137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:51.394121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a13603-a238-43f9-bea9-2499d31f5137 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.420602 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2731/5435] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:51.425488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:51.443803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2be4d-39ac-4dc7-819d-3f36321d920e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:51.453915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:51.454693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.454974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.455352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] PATH_INFO: `/v3/projects/f4dc4659695d4fd5993df98f13c714d1/users/3b2f9e5dfcba4aec9da33fe39f643695/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.469687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:51.478262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4dc4659695d4fd5993df98f13c714d1, user_id=3b2f9e5dfcba4aec9da33fe39f643695, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:51.483359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.518709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2705/5436] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:51 2022] PUT /identity/v3/projects/f4dc4659695d4fd5993df98f13c714d1/users/3b2f9e5dfcba4aec9da33fe39f643695/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:51.531386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50decc69-f463-4573-8f3c-663f5e6f3d19 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.532201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50decc69-f463-4573-8f3c-663f5e6f3d19 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.532825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50decc69-f463-4573-8f3c-663f5e6f3d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.577937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-50decc69-f463-4573-8f3c-663f5e6f3d19 None None] MFA Rules not processed for user `3b2f9e5dfcba4aec9da33fe39f643695`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:51.579412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-50decc69-f463-4573-8f3c-663f5e6f3d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:51.638327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:51.735867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:51.738930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:51.801021 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2732/5437] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:51.808285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50decc69-f463-4573-8f3c-663f5e6f3d19 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:51.823063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50decc69-f463-4573-8f3c-663f5e6f3d19 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:51.830463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:51.830755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.831245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.831245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.831862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:create_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:51.833109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.874362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d3a3-8e49-464d-a40a-e355f658e69d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:51.889103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:51.890108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:51.890462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:51.890810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] PATH_INFO: `/v3/projects/db124c1623c148a5be49241f606003e1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:51.892285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db124c1623c148a5be49241f606003e1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:51.894537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:51.975363 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2733/5438] 10.222.0.26 () {60 vars in 1314 bytes} [Tue Sep 27 10:17:51 2022] POST /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 788 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:51.979342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.002853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ca4e-87e0-4386-bb30-57382869c9a6 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.011718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db124c1623c148a5be49241f606003e1', 'request_id': 'req-bc991737-a7dd-4d7e-9638-2e0f2155444e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:52.025686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db124c1623c148a5be49241f606003e1', 'request_id': 'req-bc991737-a7dd-4d7e-9638-2e0f2155444e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:52.057758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db124c1623c148a5be49241f606003e1', 'request_id': 'req-bc991737-a7dd-4d7e-9638-2e0f2155444e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:52.087533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] The token cache is being invalidate because project db124c1623c148a5be49241f606003e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:52.088133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:52.089033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc991737-a7dd-4d7e-9638-2e0f2155444e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:52.095559 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2706/5439] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:51 2022] DELETE /identity/v3/projects/db124c1623c148a5be49241f606003e1 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:52.101343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a41ebf-7d41-4775-baee-db7712c5cbb2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.101970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a41ebf-7d41-4775-baee-db7712c5cbb2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.102400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a41ebf-7d41-4775-baee-db7712c5cbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.119812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4bdcd-0f59-4e98-95d6-e16e914101f5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.119812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4bdcd-0f59-4e98-95d6-e16e914101f5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.119812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4bdcd-0f59-4e98-95d6-e16e914101f5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.120212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4bdcd-0f59-4e98-95d6-e16e914101f5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/a06bbf7b2efa4cc9a5901af2f616e4b5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.155065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e2a41ebf-7d41-4775-baee-db7712c5cbb2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:52.169121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e2a41ebf-7d41-4775-baee-db7712c5cbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:52.169260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4bdcd-0f59-4e98-95d6-e16e914101f5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:get_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=a06bbf7b2efa4cc9a5901af2f616e4b5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.173289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4bdcd-0f59-4e98-95d6-e16e914101f5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:52.181265 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2734/5440] 10.222.0.26 () {58 vars in 1392 bytes} [Tue Sep 27 10:17:51 2022] GET /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/a06bbf7b2efa4cc9a5901af2f616e4b5 => generated 688 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:52.187490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.187778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.188126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.229325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] MFA Rules not processed for user `3b2f9e5dfcba4aec9da33fe39f643695`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:52.230612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:52.315527 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2707/5441] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:52.324164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922b77-a5a8-44fa-abef-f2dc0b3c4f5c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.324769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922b77-a5a8-44fa-abef-f2dc0b3c4f5c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.325265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922b77-a5a8-44fa-abef-f2dc0b3c4f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.328079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:52.374092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-74922b77-a5a8-44fa-abef-f2dc0b3c4f5c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:52.375536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74922b77-a5a8-44fa-abef-f2dc0b3c4f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:52.397349 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2708/5442] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:52.401241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74922b77-a5a8-44fa-abef-f2dc0b3c4f5c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.411545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74922b77-a5a8-44fa-abef-f2dc0b3c4f5c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.417913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.418734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.419220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.420213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.425777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:52.426217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.428207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:52.450163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:52.464585 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2709/5443] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:52.466014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.469263 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2735/5444] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:52 2022] POST /identity/v3/auth/tokens => generated 3424 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 27 10:17:52.473454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.474881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:52.489225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bee5d1-7b19-4fcd-af47-6219b5a06dc6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.491753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ceea1-7943-4a32-b8cf-b67259212f5b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.498294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.499193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.499318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.499475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.499754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/a06bbf7b2efa4cc9a5901af2f616e4b5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.499995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.500450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.500748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.501114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:delete_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=a06bbf7b2efa4cc9a5901af2f616e4b5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.501398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.502625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:52.503479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:52.589776 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2736/5445] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:17:52 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/a06bbf7b2efa4cc9a5901af2f616e4b5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:52.599603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.633319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662c684-7ade-419a-97b9-9d1bd8942135 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.637644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2710/5446] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:52 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:52.643851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.739394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.740828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.741321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.741704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.742441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:create_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.744048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:52.775848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a1a4d-f39b-49e4-9a34-fe7bd05dfe15 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.789176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.790147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.792459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.793361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.795706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.801207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:52.857671 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2711/5447] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:52 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:52.858534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.911719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a131-d028-4644-8a7f-df73a2d96394 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.924976 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2737/5448] 10.222.0.26 () {60 vars in 1315 bytes} [Tue Sep 27 10:17:52 2022] POST /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 921 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:52.929380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.931235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.931881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.932164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.932439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.933130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.934854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:52.945544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5ecb-776b-4730-9101-3b6cc277cb38 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:52.951802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:52.952965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:52.953344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:52.953741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/7c8cfd870b5f47dda16e84cc3c937fa0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:52.964588 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2712/5449] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:52.968550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:52.986464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:delete_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=7c8cfd870b5f47dda16e84cc3c937fa0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:52.990292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.004411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563c986-0891-42b3-91b6-0b7debad3dde admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.009935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:53.023013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.023656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.023942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.024235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.024975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.026529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.036238 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2738/5450] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:17:52 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/7c8cfd870b5f47dda16e84cc3c937fa0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:53.041411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:53.056441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2c46c-6299-4f5b-9eff-9e39abc0f3f7 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.061316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.062755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.063276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.064258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.065178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:create_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.066619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.068080 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2713/5451] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:53.068947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:53.094720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ba1c0-7f65-49b6-ac63-ade009fc9b78 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.102241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cc28d-9a47-4917-84a9-1d271dafb170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.102928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cc28d-9a47-4917-84a9-1d271dafb170 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.103207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cc28d-9a47-4917-84a9-1d271dafb170 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.103475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cc28d-9a47-4917-84a9-1d271dafb170 admin admin] PATH_INFO: `/v3/projects/fb57a25dc6b944f18a5eec9f6ac089ca/users/807fffcc89334a54b52cddf575180929/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.137928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cc28d-9a47-4917-84a9-1d271dafb170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb57a25dc6b944f18a5eec9f6ac089ca, user_id=807fffcc89334a54b52cddf575180929, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.140594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cc28d-9a47-4917-84a9-1d271dafb170 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.172677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2714/5452] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:53 2022] PUT /identity/v3/projects/fb57a25dc6b944f18a5eec9f6ac089ca/users/807fffcc89334a54b52cddf575180929/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:53.183541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.183831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.184135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.222938 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2739/5453] 10.222.0.26 () {60 vars in 1315 bytes} [Tue Sep 27 10:17:53 2022] POST /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 811 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:53.225767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] MFA Rules not processed for user `807fffcc89334a54b52cddf575180929`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:53.227674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:53.231329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:53.271127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c79591-63be-4716-89de-ae7d9790ede5 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.363907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2715/5454] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:53.374350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:53.376606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:53.402110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:53.483516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.484408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.485745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.486086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/9bdb4a5b45f84574b2fbb471df8bbb94` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.506407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:delete_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=9bdb4a5b45f84574b2fbb471df8bbb94)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.510623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.525366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45e907-dea1-4f0f-9ab9-461ea9fffa1b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.534691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.535760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.536055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.536405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.549178 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2740/5455] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:17:53 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/9bdb4a5b45f84574b2fbb471df8bbb94 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:53.555574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:53.560769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:53.585774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.587942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.635356 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2716/5456] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:17:53.636820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:53.648433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa397ee6-8d5e-429c-afd3-eb9ab64aa600 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.657076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.657828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.658114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.658865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.659308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:create_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.660636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.667058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c452ecb-20d7-4b7c-97cd-1a1611696092 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.680543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.681446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.681732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.682041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] PATH_INFO: `/v3/users/ed44be35b33a4a7a83ced9086d21cb02` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.683193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed44be35b33a4a7a83ced9086d21cb02)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.685608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.703913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:53.848105 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2717/5457] 10.222.0.26 () {60 vars in 1314 bytes} [Tue Sep 27 10:17:53 2022] POST /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 788 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:53.852692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:53.865543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed44be35b33a4a7a83ced9086d21cb02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:53.867251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed44be35b33a4a7a83ced9086d21cb02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:53.878727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbd58-b417-4b87-a796-e661dfdb4123 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:53.899095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed44be35b33a4a7a83ced9086d21cb02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:53.908452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:53.909670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.910493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.910926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:53.911955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed44be35b33a4a7a83ced9086d21cb02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:53.912333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:create_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:53.912798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-499316a1-dc88-4f09-a1f0-9f8ac2d3c849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed44be35b33a4a7a83ced9086d21cb02'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:53.914044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:53.940086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2741/5458] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:53 2022] DELETE /identity/v3/users/ed44be35b33a4a7a83ced9086d21cb02 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:53.946499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:53.947982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:53.947982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.025734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:54.030648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:54.104960 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2718/5459] 10.222.0.26 () {60 vars in 1314 bytes} [Tue Sep 27 10:17:53 2022] POST /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 789 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:54.108732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:54.112636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:54.138960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a745b9-25c7-4310-b168-d87400a2b1d0 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:54.165458 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2742/5460] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:54.166494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:54.167928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:54.223656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:54.241236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:54.242063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:54.242421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:54.242763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.243244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:list_application_credentials(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:54.246132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883a2b4-4384-4909-9f33-100f3f0b8ac3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:54.250847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:54.251560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:54.251843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:54.252164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.257542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:54.264019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:54.266533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:54.278944 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2743/5461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:54.299238 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2719/5462] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:17:54 2022] GET /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 1501 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:54.300177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:54.315323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5008b3-d7c7-442a-8a18-77225dfba141 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:54.329973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:54.331162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:54.331621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:54.332392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/6e314981b59847f5a3172c6a50c6ec78` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.345957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:delete_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=6e314981b59847f5a3172c6a50c6ec78)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:54.348168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:54.383083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:54.415072 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2744/5463] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:17:54 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/6e314981b59847f5a3172c6a50c6ec78 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:54.419038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:54.448448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca1584-1599-40e7-87eb-c7ce1fb76ebf tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:54.465785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:54.466476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:54.466767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:54.467145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/e50ca03d091e45ab98d6f9e5ea1768f7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.502591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:delete_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=e50ca03d091e45ab98d6f9e5ea1768f7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:54.504435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:54.564900 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2720/5464] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:17:54 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/e50ca03d091e45ab98d6f9e5ea1768f7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:54.573613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:54.589368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d5a95-b071-4008-9956-4be368c64831 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:54.616826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:54.617475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:54.618152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:54.618433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.619317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:create_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:54.620783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:54.753406 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2745/5465] 10.222.0.26 () {60 vars in 1314 bytes} [Tue Sep 27 10:17:54 2022] POST /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 789 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:54.757358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:54.769390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05631bc-dc8c-4b69-b882-94a5e7ed0e5a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:54.775300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:54.776332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:54.776881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:54.777504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.778506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:create_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:54.780072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:54.846440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:54.849540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:54.915147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba134359-ec83-4ae8-862e-e35173a9e8d1 tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:54.942211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:54.942821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:54.943234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:54.948727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] PATH_INFO: `/v3/projects/642d881cee9343deb91c1cffb84568ba` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:54.954115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=642d881cee9343deb91c1cffb84568ba)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:54.960857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.016802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2721/5466] 10.222.0.26 () {60 vars in 1314 bytes} [Tue Sep 27 10:17:54 2022] POST /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials => generated 788 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:55.021506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:55.034875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05c2e-6cd4-4406-90e5-a79d78bad67a tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:55.046234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:55.047046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.047191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.047437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.048123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Adding query filter params (name=tempest-application_credential-440713030) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:55.048458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:list_application_credentials(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:55.050323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.068999 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2722/5467] 10.222.0.26 () {58 vars in 1384 bytes} [Tue Sep 27 10:17:55 2022] GET /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials?name=tempest-application_credential-440713030 => generated 885 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:17:55.074166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:55.086479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3f69-ec80-4423-8d1a-e56fd3b0983b tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:55.132013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642d881cee9343deb91c1cffb84568ba', 'request_id': 'req-24db60a3-5570-4b09-8e9e-8a33f645eb9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.150113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642d881cee9343deb91c1cffb84568ba', 'request_id': 'req-24db60a3-5570-4b09-8e9e-8a33f645eb9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.160699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642d881cee9343deb91c1cffb84568ba', 'request_id': 'req-24db60a3-5570-4b09-8e9e-8a33f645eb9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.179926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] The token cache is being invalidate because project 642d881cee9343deb91c1cffb84568ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:55.180251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.181207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.196893 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2746/5468] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:54 2022] DELETE /identity/v3/projects/642d881cee9343deb91c1cffb84568ba => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:55.200074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:55.202346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:55.213352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:55.214167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.214539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.214816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/b58fd4c12c954bdb856e05a5ba8e26f5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.231033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:delete_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=b58fd4c12c954bdb856e05a5ba8e26f5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:55.233830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.269119 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2723/5469] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:17:55 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/b58fd4c12c954bdb856e05a5ba8e26f5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:55.273423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:55.275673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:55.296954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bd964-597c-4154-8b1e-3001e7f9afea tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:55.301868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db60a3-5570-4b09-8e9e-8a33f645eb9c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:55.303244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2f9e5dfcba4aec9da33fe39f643695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4dc4659695d4fd5993df98f13c714d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:55.304051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.304435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.304938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/5c1bf69f4a7a454188977303855f37d4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.308335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:55.309251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.309529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.310141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] PATH_INFO: `/v3/users/877a76fa20df40ceb8f9855b11b6d976` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.311308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877a76fa20df40ceb8f9855b11b6d976)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:55.313821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.314554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:55.319617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorizing `identity:delete_application_credential(user_id=3b2f9e5dfcba4aec9da33fe39f643695, application_credential_id=5c1bf69f4a7a454188977303855f37d4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:55.321690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.346286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:55.358686 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2724/5470] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:17:55 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695/application_credentials/5c1bf69f4a7a454188977303855f37d4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:55.365611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:55.369141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:55.386280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ed7fe-140f-4d86-a768-9ddaa76fd4cc tempest-ApplicationCredentialsV3Test-676401996 tempest-ApplicationCredentialsV3Test-676401996-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:55.386910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877a76fa20df40ceb8f9855b11b6d976'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.387570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877a76fa20df40ceb8f9855b11b6d976'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.391692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:55.392452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.392997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.393414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] PATH_INFO: `/v3/users/3b2f9e5dfcba4aec9da33fe39f643695` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.395087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2f9e5dfcba4aec9da33fe39f643695)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:55.397619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.398912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877a76fa20df40ceb8f9855b11b6d976'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.410308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877a76fa20df40ceb8f9855b11b6d976'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.411069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dafd3f00-7f7c-4c81-ba47-c56e5f07c6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877a76fa20df40ceb8f9855b11b6d976'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.432294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2747/5471] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:55 2022] DELETE /identity/v3/users/877a76fa20df40ceb8f9855b11b6d976 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:55.464585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2f9e5dfcba4aec9da33fe39f643695'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.465949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2f9e5dfcba4aec9da33fe39f643695'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.473808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2f9e5dfcba4aec9da33fe39f643695'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.495031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2f9e5dfcba4aec9da33fe39f643695'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.548804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2f9e5dfcba4aec9da33fe39f643695'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.572373 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2725/5472] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:55 2022] DELETE /identity/v3/users/3b2f9e5dfcba4aec9da33fe39f643695 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:55.574524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.574524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.574774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.650420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:55.654349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:55.690041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:55.691761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:55.800562 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2748/5473] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:55.801675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:55.805263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:55.852053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a7a8e-9ecf-4fe8-a105-4f80f052f2f8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:55.860340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:55.861291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.861755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.862393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] PATH_INFO: `/v3/projects/98a7351f5f4344e29eacf1c98e642f93` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.865244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a7351f5f4344e29eacf1c98e642f93)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:55.867863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.898914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efc0d9-fa0a-4048-8679-8fb395d1f584 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:55.905641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c0638-3db3-4b77-a8f6-51b752b76823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:55.906373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c0638-3db3-4b77-a8f6-51b752b76823 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:55.906728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c0638-3db3-4b77-a8f6-51b752b76823 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:55.907105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c0638-3db3-4b77-a8f6-51b752b76823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:55.941342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a7351f5f4344e29eacf1c98e642f93', 'request_id': 'req-f0790453-d198-47c1-a63f-d87b946ae24b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.943244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c0638-3db3-4b77-a8f6-51b752b76823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:55.945742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c0638-3db3-4b77-a8f6-51b752b76823 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:55.946885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:55.957913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a7351f5f4344e29eacf1c98e642f93', 'request_id': 'req-f0790453-d198-47c1-a63f-d87b946ae24b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:55.971971 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2749/5474] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:55.977892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a7351f5f4344e29eacf1c98e642f93', 'request_id': 'req-f0790453-d198-47c1-a63f-d87b946ae24b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:56.009456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] The token cache is being invalidate because project 98a7351f5f4344e29eacf1c98e642f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:56.010127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:56.011148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f0790453-d198-47c1-a63f-d87b946ae24b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:56.016608 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2726/5475] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:55 2022] DELETE /identity/v3/projects/98a7351f5f4344e29eacf1c98e642f93 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:56.022156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660fdc1-8f8d-43f0-a7a9-64103d477fd2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:56.022928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660fdc1-8f8d-43f0-a7a9-64103d477fd2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:56.022928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660fdc1-8f8d-43f0-a7a9-64103d477fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:56.056438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6660fdc1-8f8d-43f0-a7a9-64103d477fd2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:56.057916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6660fdc1-8f8d-43f0-a7a9-64103d477fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:56.113100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:56.147465 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2750/5476] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:56.158473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d598cf-5fce-4234-8f18-7b44cb85aee2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:56.160102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d598cf-5fce-4234-8f18-7b44cb85aee2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:56.160948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d598cf-5fce-4234-8f18-7b44cb85aee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:56.198746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-08d598cf-5fce-4234-8f18-7b44cb85aee2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:56.219928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-08d598cf-5fce-4234-8f18-7b44cb85aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:56.233023 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2727/5477] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:56.237341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660fdc1-8f8d-43f0-a7a9-64103d477fd2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:56.258660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660fdc1-8f8d-43f0-a7a9-64103d477fd2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:56.270919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:56.273338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:56.274454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:56.275329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:56.282191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:56.283357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:56.286348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:56.343708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2751/5478] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:56.345542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d598cf-5fce-4234-8f18-7b44cb85aee2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:56.360125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d598cf-5fce-4234-8f18-7b44cb85aee2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:56.379711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:56.380383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:56.380760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:56.381038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:56.381944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:56.392904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:56.491337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2728/5479] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:56 2022] POST /identity/v3/projects => generated 389 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:56.495483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:56.525651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba976ff-7e1c-4967-88cf-9e1ace604057 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:56.530397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:56.532874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:56.733637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:56.734299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:56.734518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:56.734782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:56.735416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:56.741430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:56.782603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a3d87-c0e7-486e-aa05-fc4a61de6247 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:56.791462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:56.792823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:56.793157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:56.793483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] PATH_INFO: `/v3/projects/f4dc4659695d4fd5993df98f13c714d1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:56.805005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4dc4659695d4fd5993df98f13c714d1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:56.805118 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2752/5480] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:56 2022] POST /identity/v3/users => generated 295 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:56.805212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:56.808396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:56.838135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50ca3-28e3-400d-a2c1-dc01d069e443 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:56.852597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:56.854430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:56.854925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:56.857244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:56.860947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:56.881489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:56.915245 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2753/5481] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:56.920542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:56.940119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4dc4659695d4fd5993df98f13c714d1', 'request_id': 'req-18cfabfc-ec44-41c9-af04-936b94b8368d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:56.955732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4dc4659695d4fd5993df98f13c714d1', 'request_id': 'req-18cfabfc-ec44-41c9-af04-936b94b8368d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:56.957203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d5f09-0447-4895-9f6c-cbf40d0e1239 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:56.977510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4dc4659695d4fd5993df98f13c714d1', 'request_id': 'req-18cfabfc-ec44-41c9-af04-936b94b8368d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:56.998915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] The token cache is being invalidate because project f4dc4659695d4fd5993df98f13c714d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:17:57.005259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:57.005259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18cfabfc-ec44-41c9-af04-936b94b8368d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:17:57.005259 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2729/5482] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:17:56 2022] DELETE /identity/v3/projects/f4dc4659695d4fd5993df98f13c714d1 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:57.012363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbb9b7-47da-485a-93e1-d789ed37b8f7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.012363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbb9b7-47da-485a-93e1-d789ed37b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.012634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbb9b7-47da-485a-93e1-d789ed37b8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.048563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-74dbb9b7-47da-485a-93e1-d789ed37b8f7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:57.049922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74dbb9b7-47da-485a-93e1-d789ed37b8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:57.127320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:57.128102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.128387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.128692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.129335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:57.141790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:57.171754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:57.180369 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2754/5483] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:57.185852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:57.189603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:57.235625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef80acb-905e-4e48-82d8-967e325350cf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:57.256481 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2730/5484] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:57.259108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9b5d-6f8c-4167-8999-3be3ca5528c8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.259400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9b5d-6f8c-4167-8999-3be3ca5528c8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.259680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9b5d-6f8c-4167-8999-3be3ca5528c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.264449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6562ff-e534-4ec9-9471-7f63927d2e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:57.265447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6562ff-e534-4ec9-9471-7f63927d2e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.266828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6562ff-e534-4ec9-9471-7f63927d2e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.266828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6562ff-e534-4ec9-9471-7f63927d2e57 admin admin] PATH_INFO: `/v3/projects/fe4207ab2f07457e9716c96c84c49172/users/6e2b0d38f0b44a559bf172245beb8a9c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.307342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6562ff-e534-4ec9-9471-7f63927d2e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4207ab2f07457e9716c96c84c49172, user_id=6e2b0d38f0b44a559bf172245beb8a9c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:57.312548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6562ff-e534-4ec9-9471-7f63927d2e57 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:57.317279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3b7d9b5d-6f8c-4167-8999-3be3ca5528c8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:57.318574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3b7d9b5d-6f8c-4167-8999-3be3ca5528c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:57.339659 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2731/5485] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:57.348544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d9b5d-6f8c-4167-8999-3be3ca5528c8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:57.358409 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2755/5486] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:57 2022] PUT /identity/v3/projects/fe4207ab2f07457e9716c96c84c49172/users/6e2b0d38f0b44a559bf172245beb8a9c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:57.363202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.363513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.367308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.367470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d9b5d-6f8c-4167-8999-3be3ca5528c8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:57.439675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] MFA Rules not processed for user `6e2b0d38f0b44a559bf172245beb8a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:57.444863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:57.468031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:57.635187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:57.636506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.648869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.650658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.652126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:17:57.653729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:57.681326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:57.727324 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2732/5487] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:57.733821 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2756/5488] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:57.734131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:57.746673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:57.757259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:57.776274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afbade-9f40-471b-b7ba-4c4267b5fa36 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:57.782983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:57.783596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.783871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.784213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.784993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:57.787925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:57.851176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2733/5489] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:57 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:57.856227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:57.878529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497e692-4313-4938-9e61-38ba32d7712b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:57.966843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1bdeb-15c5-4eef-9e7f-004c3c4126df admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:57.973804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:57.974458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:57.974794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.975436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.975634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.975753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:57.975997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:57.976314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:57.976762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:57.978887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.040970 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2734/5490] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:57 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:58.045950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:58.073196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032bf3b-622e-4388-8901-431b5920a24c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:58.087458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:58.090097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:58.090388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:58.090731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:58.091403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:58.095208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.114914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:58.117598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.123451 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2735/5491] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:58.129025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:58.139005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2757/5492] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:17:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:17:58.145034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4ea70-e04e-4ead-9ce9-6073e7499c63 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:58.150729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d449461-e7d6-4d0f-bc69-f054e2b64c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:58.151359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d449461-e7d6-4d0f-bc69-f054e2b64c38 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:58.151641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d449461-e7d6-4d0f-bc69-f054e2b64c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:58.151917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d449461-e7d6-4d0f-bc69-f054e2b64c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:58.152821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d449461-e7d6-4d0f-bc69-f054e2b64c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:58.154450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d449461-e7d6-4d0f-bc69-f054e2b64c38 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.171823 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2736/5493] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:17:58.179094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:58.195043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca545c3-7c3d-4ccf-ac73-6f58923067f1 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:58.218833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:58.219307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:58.221967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:58.222265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] PATH_INFO: `/v3/projects/352225c0e2e34e9fa67ecc2047c31ad6/users/ed8ef6be665240e08079e86b1074aa7f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:58.255908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] RBAC: Authorizing `identity:create_grant(project_id=352225c0e2e34e9fa67ecc2047c31ad6, user_id=ed8ef6be665240e08079e86b1074aa7f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:58.265275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.275244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:58.338091 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2758/5494] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:58 2022] PUT /identity/v3/projects/352225c0e2e34e9fa67ecc2047c31ad6/users/ed8ef6be665240e08079e86b1074aa7f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:17:58.340643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73564c2d-d5ea-4cd4-a7aa-fd08485140be None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:58.341046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73564c2d-d5ea-4cd4-a7aa-fd08485140be None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:58.341331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73564c2d-d5ea-4cd4-a7aa-fd08485140be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:58.420801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-73564c2d-d5ea-4cd4-a7aa-fd08485140be None None] MFA Rules not processed for user `ed8ef6be665240e08079e86b1074aa7f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:58.422521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-73564c2d-d5ea-4cd4-a7aa-fd08485140be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:58.428304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:58.563393 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2737/5495] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:58.569460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:58.678786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cd4e8-91b7-454e-afc3-300826b8125e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:58.693195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:58.694904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:58.695299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:58.695662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:58.696537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:58.698477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.734389 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2759/5496] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:17:58 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:17:58.738572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73564c2d-d5ea-4cd4-a7aa-fd08485140be None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:58.765060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73564c2d-d5ea-4cd4-a7aa-fd08485140be None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:58.871351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:58.873058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:58.873641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:58.874133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:58.875050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:58.877157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.924776 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2738/5497] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:17:58 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:17:58.925843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:58.942445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431550d-19ec-4ddf-8ed9-651bf6142868 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:58.954753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:58.956717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:58.957008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:58.957347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:58.958931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:58.966827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:58.997436 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2760/5498] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:59.001492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:59.017829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d012e3-5913-49b4-b0eb-19bb31c86781 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:59.028322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d145d2-9e91-451d-8dbb-8d5c1657b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:59.029006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d145d2-9e91-451d-8dbb-8d5c1657b1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.029383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d145d2-9e91-451d-8dbb-8d5c1657b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.029653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d145d2-9e91-451d-8dbb-8d5c1657b1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.030305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d145d2-9e91-451d-8dbb-8d5c1657b1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:59.032402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d145d2-9e91-451d-8dbb-8d5c1657b1d4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:59.049278 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2739/5499] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:17:59.050173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:59.062519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706c5a5-975e-4ea8-ab2e-1ee6d46d053c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:59.069878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:59.070692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.071058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.071747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] PATH_INFO: `/v3/projects/2fd299d9c108482f93d0c72bb3e1109f/users/4ea1195e8066474d9c1af93de39183cc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.094307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd299d9c108482f93d0c72bb3e1109f, user_id=4ea1195e8066474d9c1af93de39183cc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:59.097699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:59.126421 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2761/5500] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:17:59 2022] PUT /identity/v3/projects/2fd299d9c108482f93d0c72bb3e1109f/users/4ea1195e8066474d9c1af93de39183cc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:17:59.139179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b3899-7dbe-4fae-b00c-ce660fa6983b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.139466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b3899-7dbe-4fae-b00c-ce660fa6983b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.139736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b3899-7dbe-4fae-b00c-ce660fa6983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.173985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a42b3899-7dbe-4fae-b00c-ce660fa6983b None None] MFA Rules not processed for user `4ea1195e8066474d9c1af93de39183cc`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:59.175582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a42b3899-7dbe-4fae-b00c-ce660fa6983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:59.332723 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2740/5501] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:17:59.336380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:59.495834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcbfa-92ff-4780-84d8-28b833b9a27b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:59.512085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bbe03-71f3-4eb6-85b3-953819ce592a tempest-IdentityV3ProjectsTest-938715588 tempest-IdentityV3ProjectsTest-938715588-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed8ef6be665240e08079e86b1074aa7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '352225c0e2e34e9fa67ecc2047c31ad6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:59.515251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbe03-71f3-4eb6-85b3-953819ce592a tempest-IdentityV3ProjectsTest-938715588 tempest-IdentityV3ProjectsTest-938715588-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.516301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbe03-71f3-4eb6-85b3-953819ce592a tempest-IdentityV3ProjectsTest-938715588 tempest-IdentityV3ProjectsTest-938715588-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.517986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbe03-71f3-4eb6-85b3-953819ce592a tempest-IdentityV3ProjectsTest-938715588 tempest-IdentityV3ProjectsTest-938715588-project] PATH_INFO: `/v3/users/ed8ef6be665240e08079e86b1074aa7f/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.526622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bbe03-71f3-4eb6-85b3-953819ce592a tempest-IdentityV3ProjectsTest-938715588 tempest-IdentityV3ProjectsTest-938715588-project] RBAC: Authorizing `identity:list_user_projects(user_id=ed8ef6be665240e08079e86b1074aa7f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:59.532068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bbe03-71f3-4eb6-85b3-953819ce592a tempest-IdentityV3ProjectsTest-938715588 tempest-IdentityV3ProjectsTest-938715588-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:59.667062 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2762/5502] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:17:59 2022] GET /identity/v3/users/ed8ef6be665240e08079e86b1074aa7f/projects => generated 515 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:17:59.669121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.669456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.669780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.723439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:59.731899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] MFA Rules not processed for user `ed8ef6be665240e08079e86b1074aa7f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:59.734192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:59.752353 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2741/5503] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:17:59.760068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.760751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.761237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.808142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:59.810026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:59.816194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] MFA Rules not processed for user `ed8ef6be665240e08079e86b1074aa7f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:17:59.820253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:17:59.852962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:59.923680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.models.token_model [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] User ed8ef6be665240e08079e86b1074aa7f has no access to project 2fd299d9c108482f93d0c72bb3e1109f {{(pid=104132) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Sep 27 10:17:59.928621 ncdv-master-1 devstack@keystone.service[104132]: WARNING keystone.server.flask.application [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 27 10:17:59.930545 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2763/5504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:17:59 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 172 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 27 10:17:59.936492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:17:59.953572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588a1e8-2754-4418-ae20-3b66a0e43594 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:59.958779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720635-c673-4646-a9ed-5f118d2c45d6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:17:59.963500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:59.964219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.964502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.964810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] PATH_INFO: `/v3/users/ed8ef6be665240e08079e86b1074aa7f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.966136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8ef6be665240e08079e86b1074aa7f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:59.966696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:17:59.967579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:17:59.968044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:17:59.968154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:17:59.968584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] PATH_INFO: `/v3/users/807fffcc89334a54b52cddf575180929` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:17:59.969991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=807fffcc89334a54b52cddf575180929)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:17:59.972275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:00.042809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8ef6be665240e08079e86b1074aa7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.043717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8ef6be665240e08079e86b1074aa7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.060377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8ef6be665240e08079e86b1074aa7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.063855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807fffcc89334a54b52cddf575180929'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.067136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807fffcc89334a54b52cddf575180929'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.072989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8ef6be665240e08079e86b1074aa7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.073667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4de7e8b2-21ca-40e6-919f-1baf0571e064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8ef6be665240e08079e86b1074aa7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.083379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807fffcc89334a54b52cddf575180929'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.097364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807fffcc89334a54b52cddf575180929'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.110265 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2764/5505] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:59 2022] DELETE /identity/v3/users/ed8ef6be665240e08079e86b1074aa7f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:00.111984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:00.112648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:00.113176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:00.133875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b23e4d2d-136c-4d9c-a69a-9545450ec59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807fffcc89334a54b52cddf575180929'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:00.142021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:00.143690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:00.150712 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2742/5506] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:17:59 2022] DELETE /identity/v3/users/807fffcc89334a54b52cddf575180929 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:00.152609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:00.153309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:00.153595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:00.212260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:00.214067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:00.255925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2765/5507] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:00.275873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:00.281640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:00.328405 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2743/5508] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:00.346155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:00.347715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:00.378883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:00.396751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45cea2-7792-45b0-b281-a202bb10c893 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:00.399283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:00.506572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6888c-93ee-4fb7-a605-fbe48dc3f7dc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:00.519719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:00.522484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:00.523037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:00.523836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:00.524115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:00.524492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:00.525196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:00.525437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:00.556772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:00.557315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:00.561517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:00.563535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:00.582837 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2766/5509] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:00.584357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2744/5510] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:18:01.217801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:01.219966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:01.243259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2533775-db15-4e6a-bc92-e8b9080dfec1 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:01.251668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:01.252192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:01.252474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:01.252832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] PATH_INFO: `/v3/projects/352225c0e2e34e9fa67ecc2047c31ad6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:01.255029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=352225c0e2e34e9fa67ecc2047c31ad6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:01.257177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:01.324477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '352225c0e2e34e9fa67ecc2047c31ad6', 'request_id': 'req-eb215e9d-07a7-4993-8194-38b0b9f6a488'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.356307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '352225c0e2e34e9fa67ecc2047c31ad6', 'request_id': 'req-eb215e9d-07a7-4993-8194-38b0b9f6a488'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.368546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '352225c0e2e34e9fa67ecc2047c31ad6', 'request_id': 'req-eb215e9d-07a7-4993-8194-38b0b9f6a488'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.413635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] The token cache is being invalidate because project 352225c0e2e34e9fa67ecc2047c31ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:01.414400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.418275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.424583 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2767/5511] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:01 2022] DELETE /identity/v3/projects/352225c0e2e34e9fa67ecc2047c31ad6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:01.429916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:01.445588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607451f8-d0bf-4533-a52c-08a9040727db service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:01.586813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:01.588139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:01.588607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:01.589354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] PATH_INFO: `/v3/users/4ea1195e8066474d9c1af93de39183cc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:01.590694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea1195e8066474d9c1af93de39183cc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:01.593178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:01.757054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea1195e8066474d9c1af93de39183cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.759283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea1195e8066474d9c1af93de39183cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.781299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea1195e8066474d9c1af93de39183cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.801814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea1195e8066474d9c1af93de39183cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.843049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea1195e8066474d9c1af93de39183cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:01.857536 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2745/5512] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:01 2022] DELETE /identity/v3/users/4ea1195e8066474d9c1af93de39183cc => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:01.949211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:01.953231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:01.986502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:01.988036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:02.159603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb215e9d-07a7-4993-8194-38b0b9f6a488 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:02.172754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:02.243784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca6ff4-9b2a-4071-8941-b67131414474 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:02.262163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:02.262836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:02.263113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:02.263389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] PATH_INFO: `/v3/projects/fb57a25dc6b944f18a5eec9f6ac089ca` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:02.267137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb57a25dc6b944f18a5eec9f6ac089ca)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:02.269127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:02.347660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:02.419452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc773c6d-bbc7-4c4c-9a7b-5e1e8ffeaf9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:02.420568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773c6d-bbc7-4c4c-9a7b-5e1e8ffeaf9a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:02.424954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773c6d-bbc7-4c4c-9a7b-5e1e8ffeaf9a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:02.425468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc773c6d-bbc7-4c4c-9a7b-5e1e8ffeaf9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:02.428060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bc773c6d-bbc7-4c4c-9a7b-5e1e8ffeaf9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:02.445957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb57a25dc6b944f18a5eec9f6ac089ca', 'request_id': 'req-72d07935-3c5d-473f-9157-a66380d9f8f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:02.453282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc773c6d-bbc7-4c4c-9a7b-5e1e8ffeaf9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:02.455111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc773c6d-bbc7-4c4c-9a7b-5e1e8ffeaf9a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:02.478737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb57a25dc6b944f18a5eec9f6ac089ca', 'request_id': 'req-72d07935-3c5d-473f-9157-a66380d9f8f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:02.481416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2768/5513] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:02.499224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb57a25dc6b944f18a5eec9f6ac089ca', 'request_id': 'req-72d07935-3c5d-473f-9157-a66380d9f8f6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:02.531438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] The token cache is being invalidate because project fb57a25dc6b944f18a5eec9f6ac089ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:02.531791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:02.533212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72d07935-3c5d-473f-9157-a66380d9f8f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:02.546416 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2746/5514] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:01 2022] DELETE /identity/v3/projects/fb57a25dc6b944f18a5eec9f6ac089ca => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:18:02.551811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06a62-1969-4dad-918a-285a79911865 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:02.552085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06a62-1969-4dad-918a-285a79911865 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:02.552381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06a62-1969-4dad-918a-285a79911865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:02.576819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-07c06a62-1969-4dad-918a-285a79911865 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:02.578048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-07c06a62-1969-4dad-918a-285a79911865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:02.682760 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2769/5515] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:02.684376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a498169-4dcb-4927-8072-69279a66f27c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:02.684708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a498169-4dcb-4927-8072-69279a66f27c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:02.684959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a498169-4dcb-4927-8072-69279a66f27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:02.717989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5a498169-4dcb-4927-8072-69279a66f27c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:02.719594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a498169-4dcb-4927-8072-69279a66f27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:02.738309 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2747/5516] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:02.740281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06a62-1969-4dad-918a-285a79911865 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:02.753879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06a62-1969-4dad-918a-285a79911865 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:02.763642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:02.768303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:02.774578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:02.777230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:02.780776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:02.781663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:02.789249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:02.828108 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2770/5517] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:02.829683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a498169-4dcb-4927-8072-69279a66f27c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:02.845771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a498169-4dcb-4927-8072-69279a66f27c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:02.858994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:02.860839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:02.861171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:02.861425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:02.862297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:02.865486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:02.928864 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2748/5518] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:02.934021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:02.948410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d268a40-2b63-4956-8380-981693bc6843 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:03.057128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:03.057954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.058565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.058987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.059916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:03.062072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:03.118588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:03.131499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2771/5519] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:02 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:03.135808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c709d9-bae4-4ac5-acfe-d75f077b0b43 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:03.147633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:03.149647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.149981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.150259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.151467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:03.155239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:03.171022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2749/5520] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:18:03.175693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:03.191973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f635e-636e-433a-a11e-a0af7a9b6493 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:03.204318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a65529-9512-4a37-8993-9caf09dc6788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:03.206023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a65529-9512-4a37-8993-9caf09dc6788 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.206396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a65529-9512-4a37-8993-9caf09dc6788 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.206736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a65529-9512-4a37-8993-9caf09dc6788 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.207415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a65529-9512-4a37-8993-9caf09dc6788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:03.218381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a65529-9512-4a37-8993-9caf09dc6788 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:03.239563 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2772/5521] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:03.244492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:03.263464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b5919-51e5-4220-8725-c38449c4e741 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:03.277584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:03.278296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.278576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.278854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] PATH_INFO: `/v3/projects/e70b2dec4993431c865eb7d2f28e32bf/users/110cdeadcd4842f3b9a3e284c05d955a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.310851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70b2dec4993431c865eb7d2f28e32bf, user_id=110cdeadcd4842f3b9a3e284c05d955a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:03.314997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:03.344914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2750/5522] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:03 2022] PUT /identity/v3/projects/e70b2dec4993431c865eb7d2f28e32bf/users/110cdeadcd4842f3b9a3e284c05d955a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:03.356046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb0002-1f0f-4657-aeb1-d85c8349e69e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.356873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb0002-1f0f-4657-aeb1-d85c8349e69e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.357401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb0002-1f0f-4657-aeb1-d85c8349e69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.432351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-02cb0002-1f0f-4657-aeb1-d85c8349e69e None None] MFA Rules not processed for user `110cdeadcd4842f3b9a3e284c05d955a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:03.434174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-02cb0002-1f0f-4657-aeb1-d85c8349e69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:03.469213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:03.472439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:03.513177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:03.596256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0481d8-7cd4-4107-b65b-31b79ad8d58f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:03.623673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:03.624483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.624621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.624957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] PATH_INFO: `/v3/projects/2fd299d9c108482f93d0c72bb3e1109f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.626283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd299d9c108482f93d0c72bb3e1109f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:03.628292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:03.650294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2773/5523] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:03.654581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb0002-1f0f-4657-aeb1-d85c8349e69e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:03.655778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:03.674808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb0002-1f0f-4657-aeb1-d85c8349e69e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:03.686556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:03.687605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.688041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.688608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.689698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:03.691791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:03.736458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd299d9c108482f93d0c72bb3e1109f', 'request_id': 'req-948f446d-07d1-48db-aacf-e9bd67fe974c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:03.745274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd299d9c108482f93d0c72bb3e1109f', 'request_id': 'req-948f446d-07d1-48db-aacf-e9bd67fe974c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:03.763432 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2774/5524] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:03 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:03.769631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:03.773526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd299d9c108482f93d0c72bb3e1109f', 'request_id': 'req-948f446d-07d1-48db-aacf-e9bd67fe974c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:03.808958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] The token cache is being invalidate because project 2fd299d9c108482f93d0c72bb3e1109f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:03.809701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:03.810860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-948f446d-07d1-48db-aacf-e9bd67fe974c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:03.813348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2751/5525] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:03 2022] DELETE /identity/v3/projects/2fd299d9c108482f93d0c72bb3e1109f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:03.835021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601e137-e399-4cf3-917c-2c2b9478a3a1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.835222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601e137-e399-4cf3-917c-2c2b9478a3a1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.835515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601e137-e399-4cf3-917c-2c2b9478a3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.858035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c601e137-e399-4cf3-917c-2c2b9478a3a1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:03.859266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c601e137-e399-4cf3-917c-2c2b9478a3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:03.911611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7372fdf-bffa-4da6-9ff4-2230f6cd8637 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:03.930407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:03.931391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.931882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.932408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:03.933698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:03.935894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:03.978471 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2752/5526] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:03.991876 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2775/5527] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:03 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:03.994433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a75a8-177f-4500-a65b-6e8c042cf4bf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:03.994996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a75a8-177f-4500-a65b-6e8c042cf4bf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:03.996213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:03.997148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a75a8-177f-4500-a65b-6e8c042cf4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.000811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:04.027378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1f4a75a8-177f-4500-a65b-6e8c042cf4bf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:04.027897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c00c71-fe76-4d95-bbd3-3980dc7c5cca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.028651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f4a75a8-177f-4500-a65b-6e8c042cf4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:04.042090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.042852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.042997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.043259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.044510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.047731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.050199 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2753/5528] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:04.055724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a75a8-177f-4500-a65b-6e8c042cf4bf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.069102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a75a8-177f-4500-a65b-6e8c042cf4bf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.075765 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2776/5529] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:04.076955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.079638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.080569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.081545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.082002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.083589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:04.084940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.088793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.099379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373060c5-37e4-404b-b740-1fca0ffa391b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.112983 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2754/5530] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:04.117139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.117876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.118220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.118573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.118974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.119357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.121080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.132061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2afa0-8d47-42e4-bedf-eeb2cfed4928 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.141912 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2777/5531] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:04.142742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.143632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.144443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.145039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.146206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.147455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.148877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.174905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1efc5-a869-448f-b14b-f4352dd9c34c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.186218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f92d18-4335-4af2-b527-d35bcb076607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.186218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f92d18-4335-4af2-b527-d35bcb076607 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.187609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f92d18-4335-4af2-b527-d35bcb076607 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.188733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f92d18-4335-4af2-b527-d35bcb076607 admin admin] PATH_INFO: `/v3/projects/74d018c846834d05bc5d274e8044443f/users/08cb108f02c247af822b1cf686a2bf22/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.195722 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2755/5532] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:04 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:04.200146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.238922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f92d18-4335-4af2-b527-d35bcb076607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d018c846834d05bc5d274e8044443f, user_id=08cb108f02c247af822b1cf686a2bf22, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.241886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f92d18-4335-4af2-b527-d35bcb076607 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.274770 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2778/5533] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:04 2022] PUT /identity/v3/projects/74d018c846834d05bc5d274e8044443f/users/08cb108f02c247af822b1cf686a2bf22/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:04.285356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494b8a-4b5f-4fe4-a309-252ee58c233e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.286006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494b8a-4b5f-4fe4-a309-252ee58c233e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.286495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494b8a-4b5f-4fe4-a309-252ee58c233e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.316892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:04.322429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942878f-6f73-4235-bf1c-c70af7cb70c1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.330832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf494b8a-4b5f-4fe4-a309-252ee58c233e None None] MFA Rules not processed for user `08cb108f02c247af822b1cf686a2bf22`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:04.332820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf494b8a-4b5f-4fe4-a309-252ee58c233e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:04.407960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:04.438562 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2779/5534] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:04.439757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf494b8a-4b5f-4fe4-a309-252ee58c233e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.459839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.460530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.462485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.462779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.463445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.466613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.539756 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2756/5535] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:04 2022] POST /identity/v3/users => generated 291 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:04.546802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.579906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7db58-fe00-4980-adaa-79f8afe38fa0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.595994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.596547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.596851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.597121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.597784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.618633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.654123 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2757/5536] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:04.660494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf494b8a-4b5f-4fe4-a309-252ee58c233e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.660773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.666133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.667386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.667921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.668345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.669186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.672017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.699176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2561e-7db6-43ff-b811-0f6031c6c111 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.709059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.710849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.710849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.710849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.711303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.713023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.725143 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2780/5537] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:04 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:04.729120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.734233 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2758/5538] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:04.741730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.763968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e06bb-01e1-4897-b1b9-0a780ab4eaa6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.856911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0372911-9496-4116-a424-c70e0e8b8595 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.862118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.863329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.863650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.864013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.864763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.867263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.914393 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2781/5539] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:04 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:04.915706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.916932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939612c2-8720-4700-aa23-120afb9a7c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.917628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939612c2-8720-4700-aa23-120afb9a7c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.917907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939612c2-8720-4700-aa23-120afb9a7c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.918182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939612c2-8720-4700-aa23-120afb9a7c52 admin admin] PATH_INFO: `/v3/projects/57a094520e3b42abb0f48aff45534e11/users/1cd19a2d65be4865ba9b9223dde7242b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.940639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d4657-b35d-4390-9b0f-c4f379b040e3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.946112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939612c2-8720-4700-aa23-120afb9a7c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a094520e3b42abb0f48aff45534e11, user_id=1cd19a2d65be4865ba9b9223dde7242b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.946317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:04.947189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.947474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.947744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:04.948387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:04.950524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.951977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939612c2-8720-4700-aa23-120afb9a7c52 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:04.964992 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2782/5540] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:04.969285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:04.975851 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2759/5541] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:04 2022] PUT /identity/v3/projects/57a094520e3b42abb0f48aff45534e11/users/1cd19a2d65be4865ba9b9223dde7242b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:04.981805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99fa2f-0808-4591-8507-66e836d3440f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:04.989020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd8f7e-822c-4f18-83d5-69ac1faf355a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:04.989637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd8f7e-822c-4f18-83d5-69ac1faf355a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:04.990380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd8f7e-822c-4f18-83d5-69ac1faf355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.065260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-93bd8f7e-822c-4f18-83d5-69ac1faf355a None None] MFA Rules not processed for user `1cd19a2d65be4865ba9b9223dde7242b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:05.066593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-93bd8f7e-822c-4f18-83d5-69ac1faf355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:05.072699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:05.090083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.092096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.092999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.093496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] PATH_INFO: `/v3/projects/384c1751fbaf4706bdb5306702facd1f/users/5e440bb19dae4ecc958981257a239ccb/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.115328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=384c1751fbaf4706bdb5306702facd1f, user_id=5e440bb19dae4ecc958981257a239ccb, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.119242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.124159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:05.146249 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2783/5542] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:04 2022] PUT /identity/v3/projects/384c1751fbaf4706bdb5306702facd1f/users/5e440bb19dae4ecc958981257a239ccb/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:05.149780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.168874 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2760/5543] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:05.174799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd8f7e-822c-4f18-83d5-69ac1faf355a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.198234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd8f7e-822c-4f18-83d5-69ac1faf355a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.250778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087011b-b911-4e85-b695-9eacbf297b1a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.256640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.257445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.258290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.258576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.259236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.262541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.279659 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2784/5544] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:18:05.284148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.289528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.290474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.290866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.291926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.292668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.295216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.300050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a622e6e-cdfa-47c5-b287-29ce22566cac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.306790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00307b97-562c-4df9-ba0a-b37400de584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.307482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00307b97-562c-4df9-ba0a-b37400de584a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.308336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00307b97-562c-4df9-ba0a-b37400de584a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.308646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00307b97-562c-4df9-ba0a-b37400de584a admin admin] PATH_INFO: `/v3/projects/384c1751fbaf4706bdb5306702facd1f/users/5e440bb19dae4ecc958981257a239ccb/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.310936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00307b97-562c-4df9-ba0a-b37400de584a admin admin] RBAC: Authorizing `identity:create_grant(project_id=384c1751fbaf4706bdb5306702facd1f, user_id=5e440bb19dae4ecc958981257a239ccb, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.314544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00307b97-562c-4df9-ba0a-b37400de584a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.339657 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2785/5545] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:05 2022] PUT /identity/v3/projects/384c1751fbaf4706bdb5306702facd1f/users/5e440bb19dae4ecc958981257a239ccb/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:05.341638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.341927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.342207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.343157 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2761/5546] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:05 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:05.346907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.381091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] MFA Rules not processed for user `5e440bb19dae4ecc958981257a239ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:05.382563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:05.524308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a944996-4a12-427f-90ab-bda97dbf5b00 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.540881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.541531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.541800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.542121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.542734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.549757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.569462 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2786/5547] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:05.582005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.584071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:05.625567 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2762/5548] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:05 2022] POST /identity/v3/users => generated 291 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:05.629268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.646970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3241f-a83c-48be-a610-2b758bde2b2a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.656470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.657102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.657373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.657638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.658772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.661240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.688481 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2763/5549] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:18:05.694606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:05.694930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.712130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c814-fa51-418c-8237-89645e82b249 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.720587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.722030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.722485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.722908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.724868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.725248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6442f4-2e6c-49a4-965c-a4ee0a9b23fb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.727183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.729070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.729693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.729978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.730649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.742365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:05.756034 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2764/5550] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:05.756034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:05.777878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cffc-8e0e-41a4-b8e4-4b3bdc39b61c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:05.789069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbafe9c-00ee-40c1-93d5-e75dcbc1a82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:05.789741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbafe9c-00ee-40c1-93d5-e75dcbc1a82f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.789932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbafe9c-00ee-40c1-93d5-e75dcbc1a82f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.790197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbafe9c-00ee-40c1-93d5-e75dcbc1a82f admin admin] PATH_INFO: `/v3/projects/4b28c4bc8e9c4db7bb88c32daf09eaab/users/c73883f8f5ae47fabfe0b41595d41765/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.817761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbafe9c-00ee-40c1-93d5-e75dcbc1a82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b28c4bc8e9c4db7bb88c32daf09eaab, user_id=c73883f8f5ae47fabfe0b41595d41765, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.821210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbafe9c-00ee-40c1-93d5-e75dcbc1a82f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.851099 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2765/5551] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:05 2022] PUT /identity/v3/projects/4b28c4bc8e9c4db7bb88c32daf09eaab/users/c73883f8f5ae47fabfe0b41595d41765/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:05.865906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:05.867399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:05.898985 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2787/5552] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:18:05.958748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213bfcb-5825-4d08-826c-4b80942387fe None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:05.959026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213bfcb-5825-4d08-826c-4b80942387fe None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:05.959293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213bfcb-5825-4d08-826c-4b80942387fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:05.993231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:06.007191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2213bfcb-5825-4d08-826c-4b80942387fe None None] MFA Rules not processed for user `c73883f8f5ae47fabfe0b41595d41765`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:06.008710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2213bfcb-5825-4d08-826c-4b80942387fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:06.134974 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2766/5553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:06.138900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:06.162027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991598a-3fed-4944-8736-3c76fe83a641 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:06.323918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:06.326236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:06.327424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:06.329132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:06.330548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:06.340977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:06.431074 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2788/5554] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:06 2022] POST /identity/v3/projects => generated 383 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:06.433359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213bfcb-5825-4d08-826c-4b80942387fe None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:06.447183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213bfcb-5825-4d08-826c-4b80942387fe None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:06.589111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:06.596365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:06.597558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:06.598941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:06.601106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:06.603149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:06.665409 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2767/5555] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:06 2022] POST /identity/v3/users => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:06.666771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:06.685820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04125d3c-3b3c-43de-b51c-c1d3aac6120f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:06.698284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:06.698284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:06.698284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:06.698284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:06.699271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:06.710056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:06.757238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:06.757435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2789/5556] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:06.773306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77edd4c-9998-4599-8bcc-9bd3d12dffba admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:06.791605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:06.793578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:06.793910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:06.794307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] PATH_INFO: `/v3/projects/ccf6dde33c064e69aa8eb8317514de9b/users/60e697df0e9d438bb496f76e937aa422/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:06.844733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf6dde33c064e69aa8eb8317514de9b, user_id=60e697df0e9d438bb496f76e937aa422, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:06.850866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:06.874989 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2768/5557] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:06 2022] PUT /identity/v3/projects/ccf6dde33c064e69aa8eb8317514de9b/users/60e697df0e9d438bb496f76e937aa422/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:18:06.879239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:06.894012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7f06a-eb36-454f-ba5f-bcf51077259d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:07.449235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db414-fe1e-4f91-a7a0-d7927b78e6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:07.449235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db414-fe1e-4f91-a7a0-d7927b78e6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:07.449235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db414-fe1e-4f91-a7a0-d7927b78e6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:07.449235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db414-fe1e-4f91-a7a0-d7927b78e6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:07.449706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db414-fe1e-4f91-a7a0-d7927b78e6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:07.449706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db414-fe1e-4f91-a7a0-d7927b78e6c3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:07.464150 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2790/5558] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 585 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:07.468293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:07.479626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47439a6-afbf-49f6-af25-4ae15fe5ba5b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:07.484760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:07.486122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:07.486351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:07.486722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] PATH_INFO: `/v3/projects/ccf6dde33c064e69aa8eb8317514de9b/users/60e697df0e9d438bb496f76e937aa422/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:07.489246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf6dde33c064e69aa8eb8317514de9b, user_id=60e697df0e9d438bb496f76e937aa422, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:07.493455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:07.521612 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2769/5559] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:07 2022] PUT /identity/v3/projects/ccf6dde33c064e69aa8eb8317514de9b/users/60e697df0e9d438bb496f76e937aa422/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:07.532201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:07.532479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:07.532815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:07.584497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:07.638208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] MFA Rules not processed for user `60e697df0e9d438bb496f76e937aa422`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:07.639630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:07.873535 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2791/5560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:07.945876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:07.947273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:07.950233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:07.953117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:08.170209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5268-7eff-414c-94b8-29d0ee302c6d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:08.173892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e35c28-cdd8-49db-85a2-53948890200b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:08.185953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:08.186504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:08.186782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:08.187054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:08.189029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:08.189666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:08.189945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:08.190220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:08.221421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:08.340011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:08.341632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:08.369982 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2792/5561] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:08.385712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:08.387855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:08.404412 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2770/5562] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:09.660261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:09.677071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e004-a228-4d73-9902-434562e1e5cb service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:09.682107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093c82e-b10c-48a9-ac8d-be0c2a71e09d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:09.683157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c82e-b10c-48a9-ac8d-be0c2a71e09d service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:09.683592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c82e-b10c-48a9-ac8d-be0c2a71e09d service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:09.684114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c82e-b10c-48a9-ac8d-be0c2a71e09d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:09.742478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:09.763933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093c82e-b10c-48a9-ac8d-be0c2a71e09d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:09.765932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093c82e-b10c-48a9-ac8d-be0c2a71e09d service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:09.778083 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2793/5563] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:14.161891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:14.355183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797b7a-30e3-440b-8dff-b9d2bf19910b service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:14.363107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:14.364734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:14.365307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:14.366012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] PATH_INFO: `/v3/users/110cdeadcd4842f3b9a3e284c05d955a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:14.368891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=110cdeadcd4842f3b9a3e284c05d955a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:14.372330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:14.459926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110cdeadcd4842f3b9a3e284c05d955a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:14.461883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110cdeadcd4842f3b9a3e284c05d955a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:14.479302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110cdeadcd4842f3b9a3e284c05d955a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:14.494147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110cdeadcd4842f3b9a3e284c05d955a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:14.516564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110cdeadcd4842f3b9a3e284c05d955a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:14.526902 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2771/5564] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:14 2022] DELETE /identity/v3/users/110cdeadcd4842f3b9a3e284c05d955a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:14.535045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:14.536886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:14.537387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:14.579530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:14.579530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:14.679284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:14.680805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:14.732813 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2794/5565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:14.756048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:14.766853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:14.771718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:14.802193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193c31-161f-4e65-af13-c61f0df48c94 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:14.817863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:14.818971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:14.818971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:14.822547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:14.827798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:14.856544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:14.860954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:14.888955 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2772/5566] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:14.921041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa5ae9-640f-4700-adb5-881ecdb45b91 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:14.928416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:14.929278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:14.929624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:14.929951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:14.944161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:14.946364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:14.959855 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2795/5567] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:16.425735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:16.427940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:16.454394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3634-60e9-47e4-b7a8-05bb0f0cb496 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:16.461329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:16.477384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:16.477865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:16.479352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] PATH_INFO: `/v3/projects/e70b2dec4993431c865eb7d2f28e32bf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:16.482139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70b2dec4993431c865eb7d2f28e32bf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:16.484508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:16.582247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70b2dec4993431c865eb7d2f28e32bf', 'request_id': 'req-c6d0a1de-3c7d-4e53-9561-49be09499f39'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.600545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70b2dec4993431c865eb7d2f28e32bf', 'request_id': 'req-c6d0a1de-3c7d-4e53-9561-49be09499f39'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.626161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70b2dec4993431c865eb7d2f28e32bf', 'request_id': 'req-c6d0a1de-3c7d-4e53-9561-49be09499f39'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.649182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] The token cache is being invalidate because project e70b2dec4993431c865eb7d2f28e32bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:16.650029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.652389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.664023 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2773/5568] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:16 2022] DELETE /identity/v3/projects/e70b2dec4993431c865eb7d2f28e32bf => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:16.667969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:16.679486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27fc5c-075f-4a29-a0ea-7ac4afb1ac1b service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:16.755235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:16.757364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:16.757841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:16.758270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] PATH_INFO: `/v3/users/08cb108f02c247af822b1cf686a2bf22` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:16.759748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08cb108f02c247af822b1cf686a2bf22)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:16.761891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:16.803028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:16.834631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08cb108f02c247af822b1cf686a2bf22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.835656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08cb108f02c247af822b1cf686a2bf22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.850697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08cb108f02c247af822b1cf686a2bf22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.865612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08cb108f02c247af822b1cf686a2bf22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.866706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08cb108f02c247af822b1cf686a2bf22'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:16.890260 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2796/5569] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:16 2022] DELETE /identity/v3/users/08cb108f02c247af822b1cf686a2bf22 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:17.070402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:17.072022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:17.089113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0a1de-3c7d-4e53-9561-49be09499f39 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:17.161207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:17.162336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:17.162336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:17.162594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] PATH_INFO: `/v3/users/e4772d35d57f478d929eea70fd11e147` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:17.163603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4772d35d57f478d929eea70fd11e147)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:17.165954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:17.217195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4772d35d57f478d929eea70fd11e147'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.217905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4772d35d57f478d929eea70fd11e147'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.227031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4772d35d57f478d929eea70fd11e147'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.228940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:17.237619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4772d35d57f478d929eea70fd11e147'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.252027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4772d35d57f478d929eea70fd11e147'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.258975 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2774/5570] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:17 2022] DELETE /identity/v3/users/e4772d35d57f478d929eea70fd11e147 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:17.464131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:17.465953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:17.487671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faddfd4-6b61-4824-ae05-1aa5b7229de6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:17.571516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:17.572379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:17.572529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:17.572750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] PATH_INFO: `/v3/projects/9c60d0e3692444e39346d2c296c32ac2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:17.574888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c60d0e3692444e39346d2c296c32ac2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:17.577275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:17.674874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c60d0e3692444e39346d2c296c32ac2', 'request_id': 'req-70dfcee6-0f90-4664-a8ce-b52f490195d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.688119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c60d0e3692444e39346d2c296c32ac2', 'request_id': 'req-70dfcee6-0f90-4664-a8ce-b52f490195d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.698830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c60d0e3692444e39346d2c296c32ac2', 'request_id': 'req-70dfcee6-0f90-4664-a8ce-b52f490195d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.716490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] The token cache is being invalidate because project 9c60d0e3692444e39346d2c296c32ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:17.717862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.718810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.721468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2797/5571] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:17 2022] DELETE /identity/v3/projects/9c60d0e3692444e39346d2c296c32ac2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:17.725568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:17.727571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:17.745604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db64b6-d3a7-4e55-b9a5-32d49310e16d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:17.847699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:17.848587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:17.849344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:17.849781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] PATH_INFO: `/v3/users/79ad6239ed5d4b0d91b35d961ec79f50` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:17.851603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ad6239ed5d4b0d91b35d961ec79f50)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:17.857548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:17.937567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ad6239ed5d4b0d91b35d961ec79f50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.938249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ad6239ed5d4b0d91b35d961ec79f50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.947343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ad6239ed5d4b0d91b35d961ec79f50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.957543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ad6239ed5d4b0d91b35d961ec79f50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.977055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ad6239ed5d4b0d91b35d961ec79f50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:17.991802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2775/5572] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:17 2022] DELETE /identity/v3/users/79ad6239ed5d4b0d91b35d961ec79f50 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:18.076386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:18.078986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:18.102905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfcee6-0f90-4664-a8ce-b52f490195d0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:18.160142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:18.212090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:18.213358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:18.213859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:18.214327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] PATH_INFO: `/v3/projects/74d018c846834d05bc5d274e8044443f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:18.215704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d018c846834d05bc5d274e8044443f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:18.218071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:18.295886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d018c846834d05bc5d274e8044443f', 'request_id': 'req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.319182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d018c846834d05bc5d274e8044443f', 'request_id': 'req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.329346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d018c846834d05bc5d274e8044443f', 'request_id': 'req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.344761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] The token cache is being invalidate because project 74d018c846834d05bc5d274e8044443f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:18.345035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.345847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.347712 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2798/5573] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:18 2022] DELETE /identity/v3/projects/74d018c846834d05bc5d274e8044443f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:18.352008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:18.353453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:18.368229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f587f8-cc01-4f43-96ac-79d90eb6c8e0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:18.436647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:18.449986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:18.451020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:18.451310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:18.451589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] PATH_INFO: `/v3/users/5e440bb19dae4ecc958981257a239ccb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:18.453295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e440bb19dae4ecc958981257a239ccb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:18.455491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:18.522803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e440bb19dae4ecc958981257a239ccb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.523594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e440bb19dae4ecc958981257a239ccb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.534419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e440bb19dae4ecc958981257a239ccb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.546907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e440bb19dae4ecc958981257a239ccb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.563743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2ecf2b8f-9dad-48d1-a403-25e9784247d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e440bb19dae4ecc958981257a239ccb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:18.571819 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2776/5574] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:18 2022] DELETE /identity/v3/users/5e440bb19dae4ecc958981257a239ccb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:18.928173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:18.929824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:18.956102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0a0b9-80c5-4fa7-b46f-d1fa1807790f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.052120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:19.053472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.055281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.055867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] PATH_INFO: `/v3/projects/f8c8e28396c5494c9d0e82a3bec39844` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.057596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c8e28396c5494c9d0e82a3bec39844)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:19.061932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:19.154998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c8e28396c5494c9d0e82a3bec39844', 'request_id': 'req-afbda19d-d046-4e0b-9f28-b39aa98013ac'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:19.169684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c8e28396c5494c9d0e82a3bec39844', 'request_id': 'req-afbda19d-d046-4e0b-9f28-b39aa98013ac'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:19.176589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c8e28396c5494c9d0e82a3bec39844', 'request_id': 'req-afbda19d-d046-4e0b-9f28-b39aa98013ac'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:19.180701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:19.197034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] The token cache is being invalidate because project f8c8e28396c5494c9d0e82a3bec39844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:19.197361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:19.198533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afbda19d-d046-4e0b-9f28-b39aa98013ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:19.201488 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2799/5575] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:18 2022] DELETE /identity/v3/projects/f8c8e28396c5494c9d0e82a3bec39844 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:19.209028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72820e2-0e8e-4825-94f3-e386b23db636 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.209777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72820e2-0e8e-4825-94f3-e386b23db636 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.209777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72820e2-0e8e-4825-94f3-e386b23db636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.244165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b72820e2-0e8e-4825-94f3-e386b23db636 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:19.246409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b72820e2-0e8e-4825-94f3-e386b23db636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:19.355353 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2777/5576] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:19.363914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79520320-cc18-432c-a1a5-12f5b3bb4dc9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.363914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79520320-cc18-432c-a1a5-12f5b3bb4dc9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.364186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79520320-cc18-432c-a1a5-12f5b3bb4dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.391771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-79520320-cc18-432c-a1a5-12f5b3bb4dc9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:19.393814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79520320-cc18-432c-a1a5-12f5b3bb4dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:19.433196 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2800/5577] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:19.436971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72820e2-0e8e-4825-94f3-e386b23db636 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:19.450461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72820e2-0e8e-4825-94f3-e386b23db636 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.460703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:19.461468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.461823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.463505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.466706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:19.467489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:19.469896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:19.510633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2778/5578] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:19.514973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79520320-cc18-432c-a1a5-12f5b3bb4dc9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:19.530593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79520320-cc18-432c-a1a5-12f5b3bb4dc9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.546121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:19.547027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.547462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.547881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.548764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:19.557148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:19.616205 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2801/5579] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:19 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:19.617872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:19.631278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb971e-808d-4ad0-a0c5-bb6383ba3a28 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.715147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:19.716110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.716989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.724345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.724345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:19.724345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:19.729910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:19.755566 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2779/5580] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:19 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:19.757084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:19.769123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be845c5-6e87-4ee0-86d2-6c21d50ecec5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.775729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:19.776447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.776828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.777167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.777885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:19.779919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:19.795940 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2802/5581] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:18:19.797545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:19.820145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519b26c-180e-48e8-b1c7-992d91477596 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.829255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:19.829742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.830100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.830443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.831496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:19.833218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:19.845588 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2780/5582] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:19.850035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:19.864639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e230c3-8b13-4162-b7c2-3f7d09c6adb6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.870219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64ced8-0e9d-4f87-823f-3ee004334aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:19.870957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64ced8-0e9d-4f87-823f-3ee004334aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.871295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64ced8-0e9d-4f87-823f-3ee004334aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.871593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64ced8-0e9d-4f87-823f-3ee004334aaa admin admin] PATH_INFO: `/v3/projects/cd00595fe8dc4d9594513f900172d7ff/users/6efd3d39610444b99c7bb245337c2ac3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.914359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64ced8-0e9d-4f87-823f-3ee004334aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd00595fe8dc4d9594513f900172d7ff, user_id=6efd3d39610444b99c7bb245337c2ac3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:19.917439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64ced8-0e9d-4f87-823f-3ee004334aaa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:19.920101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:19.921799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:19.951011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98a91e-f65d-4900-9e9b-958bf5080ffb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:19.952131 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2803/5583] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:19 2022] PUT /identity/v3/projects/cd00595fe8dc4d9594513f900172d7ff/users/6efd3d39610444b99c7bb245337c2ac3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:18:19.957313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:19.957654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:19.958001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:19.981102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:19.983236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:20.041999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.042763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.044607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.044917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] PATH_INFO: `/v3/projects/384c1751fbaf4706bdb5306702facd1f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.049313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=384c1751fbaf4706bdb5306702facd1f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.051236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.100704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2804/5584] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:20.118492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:20.120625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:20.137678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384c1751fbaf4706bdb5306702facd1f', 'request_id': 'req-5594e8b1-a133-4dd3-9afa-0742cb0ba329'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:20.147029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384c1751fbaf4706bdb5306702facd1f', 'request_id': 'req-5594e8b1-a133-4dd3-9afa-0742cb0ba329'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:20.166986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384c1751fbaf4706bdb5306702facd1f', 'request_id': 'req-5594e8b1-a133-4dd3-9afa-0742cb0ba329'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:20.212014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887a8a2-1d57-4d81-9f55-aeb7073b57d0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:20.221177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] The token cache is being invalidate because project 384c1751fbaf4706bdb5306702facd1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:20.221689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:20.222747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5594e8b1-a133-4dd3-9afa-0742cb0ba329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:20.232541 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2781/5585] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:19 2022] DELETE /identity/v3/projects/384c1751fbaf4706bdb5306702facd1f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:20.235180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c23084-d68a-4f05-9403-babe11aebc5b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.235494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c23084-d68a-4f05-9403-babe11aebc5b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.237868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c23084-d68a-4f05-9403-babe11aebc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.286411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-10c23084-d68a-4f05-9403-babe11aebc5b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:20.290028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-10c23084-d68a-4f05-9403-babe11aebc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:20.303170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:20.312079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.314430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.315923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.317812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.321768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:20.414990 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2782/5586] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:20.422619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f8604-5831-4024-8555-0a25049fc979 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.423325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f8604-5831-4024-8555-0a25049fc979 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.424001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f8604-5831-4024-8555-0a25049fc979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.449780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-712f8604-5831-4024-8555-0a25049fc979 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:20.451452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-712f8604-5831-4024-8555-0a25049fc979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:20.461694 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2783/5587] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:20.465658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f8604-5831-4024-8555-0a25049fc979 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:20.466755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.468265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.472582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:20.480501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f8604-5831-4024-8555-0a25049fc979 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:20.480816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2805/5588] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:18:20.486502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.487537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.487842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.488238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.489032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:20.489597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.491882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.509595 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2784/5589] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:18:20.511292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:20.524000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80fdb-2cbe-4865-9f6e-77711b8830dc service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:20.546162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.547376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.547905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.548977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.550121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.552475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.618648 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2806/5590] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:20 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:20.620376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:20.636970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2a98-ba30-4516-978c-16aadce5173b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:20.742029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.742667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.745280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.745588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.746827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.749117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.803348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2785/5591] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:20 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:20.807491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:20.819129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6a64d-c368-47e4-81a2-4342f619d9cf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:20.825969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.826603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.827345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.827661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.828545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.830340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.844788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2807/5592] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:20.854547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:20.868528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afaa64-0daf-4cbd-8240-c5ae6cf2551a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:20.874959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fac038-ef54-4c51-8521-ac25951228da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.875717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fac038-ef54-4c51-8521-ac25951228da admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.876078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fac038-ef54-4c51-8521-ac25951228da admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.876423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fac038-ef54-4c51-8521-ac25951228da admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.877193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fac038-ef54-4c51-8521-ac25951228da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.879510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fac038-ef54-4c51-8521-ac25951228da admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.905019 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2786/5593] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:18:20.909377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:20.922414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7e0bb-7774-43f2-8d22-87b49d15a3c1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:20.929687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:20.930784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.931522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.931966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] PATH_INFO: `/v3/projects/884264769e5b4dd8b661de39c466e76a/users/7f35330693e947a6a7021290be4c946f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:20.949626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=884264769e5b4dd8b661de39c466e76a, user_id=7f35330693e947a6a7021290be4c946f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:20.953378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:20.974954 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2808/5594] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:20 2022] PUT /identity/v3/projects/884264769e5b4dd8b661de39c466e76a/users/7f35330693e947a6a7021290be4c946f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:18:20.980720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:20.981244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:20.981541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:21.009154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:21.013054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:21.113429 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2787/5595] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:21.141966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:21.145324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:21.284269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51613b4-9cf8-4f10-a339-28def7e95a4d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:21.294254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:21.295352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:21.295842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:21.296314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:21.309921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:21.311648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:21.325351 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2809/5596] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:22.871791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:22.874945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:22.898703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7a248-741b-4fa7-a1ee-f44f940f936b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:22.915929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee88fab-c10f-4e60-afe6-8e34fdc59b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:22.917534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee88fab-c10f-4e60-afe6-8e34fdc59b9b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:22.917534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee88fab-c10f-4e60-afe6-8e34fdc59b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:22.917534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee88fab-c10f-4e60-afe6-8e34fdc59b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:22.921817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ee88fab-c10f-4e60-afe6-8e34fdc59b9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:23.118383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee88fab-c10f-4e60-afe6-8e34fdc59b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:23.121636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee88fab-c10f-4e60-afe6-8e34fdc59b9b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:23.158951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2788/5597] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:23.522203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:23.523837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:23.540915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595daf-23bb-4489-8643-fc884d476b11 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:23.548015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:23.548978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:23.549622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:23.550165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] PATH_INFO: `/v3/users/6e2b0d38f0b44a559bf172245beb8a9c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:23.551465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2b0d38f0b44a559bf172245beb8a9c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:23.554737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:23.636878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:23.651791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2b0d38f0b44a559bf172245beb8a9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:23.652461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2b0d38f0b44a559bf172245beb8a9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:23.662172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2b0d38f0b44a559bf172245beb8a9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:23.694559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2b0d38f0b44a559bf172245beb8a9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:23.698040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2b0d38f0b44a559bf172245beb8a9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:23.729690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2810/5598] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:23 2022] DELETE /identity/v3/users/6e2b0d38f0b44a559bf172245beb8a9c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:23.732311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:23.732786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:23.736132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:23.747577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:23.765303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:23.767772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:23.963182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2789/5599] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:24.028736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:24.030329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:24.151569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff1684-a516-4fe8-bd51-c9d989b91499 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:24.160908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:24.162014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:24.163471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:24.163966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:24.197687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:24.200382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:24.214053 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2811/5600] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:24.723464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:24.725723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:24.748546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed9d53-d944-4da3-86d5-09c7ec3aa3b0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:24.760550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:24.761505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:24.761998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:24.762280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] PATH_INFO: `/v3/users/279d3662f60c4a979c22ee3e8fa92865` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:24.768264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] RBAC: Authorizing `identity:delete_user(user_id=279d3662f60c4a979c22ee3e8fa92865)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:24.774165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:24.849450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:24.853136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:24.887849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41daeac-177a-4112-b328-89c7256a61fd service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:24.895169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:24.895857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:24.896182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:24.896793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] PATH_INFO: `/v3/projects/fe4207ab2f07457e9716c96c84c49172` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:24.898225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279d3662f60c4a979c22ee3e8fa92865'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:24.898374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe4207ab2f07457e9716c96c84c49172)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:24.899606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279d3662f60c4a979c22ee3e8fa92865'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:24.901938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:24.916104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279d3662f60c4a979c22ee3e8fa92865'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:24.943165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279d3662f60c4a979c22ee3e8fa92865'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:24.967993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d7ffe7-b08b-4d58-97a3-ee00f4156afb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279d3662f60c4a979c22ee3e8fa92865'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:24.980006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2790/5601] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:24 2022] DELETE /identity/v3/users/279d3662f60c4a979c22ee3e8fa92865 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:25.004312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe4207ab2f07457e9716c96c84c49172', 'request_id': 'req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:25.029629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe4207ab2f07457e9716c96c84c49172', 'request_id': 'req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:25.055891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe4207ab2f07457e9716c96c84c49172', 'request_id': 'req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:25.079042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] The token cache is being invalidate because project fe4207ab2f07457e9716c96c84c49172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:25.079977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:25.081148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f1a23dac-4d7a-4f68-89e6-80b670ae8a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:25.083890 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2812/5602] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:24 2022] DELETE /identity/v3/projects/fe4207ab2f07457e9716c96c84c49172 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:25.089957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d77aa-d514-491c-afc4-907b3fd1d7b5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.089957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d77aa-d514-491c-afc4-907b3fd1d7b5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.089957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d77aa-d514-491c-afc4-907b3fd1d7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.114977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1d1d77aa-d514-491c-afc4-907b3fd1d7b5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:25.116734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1d1d77aa-d514-491c-afc4-907b3fd1d7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:25.223773 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2791/5603] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:25.233487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cdc4b-386d-43d9-948b-19d85c252847 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.235616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cdc4b-386d-43d9-948b-19d85c252847 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.235616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cdc4b-386d-43d9-948b-19d85c252847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.256847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-670cdc4b-386d-43d9-948b-19d85c252847 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:25.258097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-670cdc4b-386d-43d9-948b-19d85c252847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:25.274724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2813/5604] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:25.278901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d77aa-d514-491c-afc4-907b3fd1d7b5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:25.295116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d77aa-d514-491c-afc4-907b3fd1d7b5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:25.301029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:25.302076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.302305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.302758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.303798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:25.304338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:25.306322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:25.329268 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2792/5605] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:25.334090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cdc4b-386d-43d9-948b-19d85c252847 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:25.344513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cdc4b-386d-43d9-948b-19d85c252847 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:25.355711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:25.356790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.357274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.357768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.358649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:25.360830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:25.421477 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2814/5606] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:25 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:25.422710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:25.428926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:25.437494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42008e29-a672-4191-9d95-48a58dd7edb2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:25.582722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:25.583558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.583987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.584408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.586506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:25.588528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:25.660422 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2793/5607] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:25 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:25.664414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:25.705802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d5f05-3be8-495a-a7ab-cfab8be38250 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:25.714664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:25.715506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.715934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.716350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.717440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:25.719242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:25.735219 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2815/5608] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:25.739109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:25.753638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9bcf5-f7ff-4f93-b5e1-58bc39d65f3b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:25.759410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f053-2dee-433e-840f-c0f64f44e556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:25.760382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f053-2dee-433e-840f-c0f64f44e556 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.761322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f053-2dee-433e-840f-c0f64f44e556 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.761845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f053-2dee-433e-840f-c0f64f44e556 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.762684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f053-2dee-433e-840f-c0f64f44e556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:25.764506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f053-2dee-433e-840f-c0f64f44e556 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:25.785078 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2794/5609] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:25.790197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:25.796818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:25.808244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24dd1d-9abd-47b3-8d56-7979d8669566 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:25.815345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:25.816029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.816324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.817557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] PATH_INFO: `/v3/projects/c344fff70741430c8b375ff2f2fe0b55/users/563ca4b3ccd94ca7a62eea40a55f903a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.841188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c344fff70741430c8b375ff2f2fe0b55, user_id=563ca4b3ccd94ca7a62eea40a55f903a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:25.847172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:25.888472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2816/5610] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:25 2022] PUT /identity/v3/projects/c344fff70741430c8b375ff2f2fe0b55/users/563ca4b3ccd94ca7a62eea40a55f903a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:25.890204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:25.890681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:25.891140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:25.953389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] MFA Rules not processed for user `563ca4b3ccd94ca7a62eea40a55f903a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:25.954814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:26.074842 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2795/5611] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:26.197656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:26.200946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:26.281868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:26.283542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:26.386741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12807a98-eb68-4382-9312-24916a81e33b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:26.391651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ea0c8-0603-492f-8ce5-900d241b2747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:26.392277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea0c8-0603-492f-8ce5-900d241b2747 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:26.392549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea0c8-0603-492f-8ce5-900d241b2747 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:26.392892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea0c8-0603-492f-8ce5-900d241b2747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:26.412401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ea0c8-0603-492f-8ce5-900d241b2747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:26.413914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ea0c8-0603-492f-8ce5-900d241b2747 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:26.420295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f454d57-d9ea-415b-b89f-2f2a4063d472 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:26.426333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:26.427815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:26.428581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:26.429261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] PATH_INFO: `/v3/projects/e2e00c90216b4d1b80ac7892ee23550f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:26.430603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e00c90216b4d1b80ac7892ee23550f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:26.432841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:26.433279 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2817/5612] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:18:26.448208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:26.523500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e00c90216b4d1b80ac7892ee23550f', 'request_id': 'req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:26.534349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e00c90216b4d1b80ac7892ee23550f', 'request_id': 'req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:26.552877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e00c90216b4d1b80ac7892ee23550f', 'request_id': 'req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:26.570439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] The token cache is being invalidate because project e2e00c90216b4d1b80ac7892ee23550f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:26.570822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:26.571649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8cc4a23d-73eb-4743-8ae8-dab084bbba2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:26.574486 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2796/5613] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:26 2022] DELETE /identity/v3/projects/e2e00c90216b4d1b80ac7892ee23550f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:26.579940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023835b2-1589-4761-9704-1f5bc563d829 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:26.580290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023835b2-1589-4761-9704-1f5bc563d829 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:26.580679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023835b2-1589-4761-9704-1f5bc563d829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:26.633068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-023835b2-1589-4761-9704-1f5bc563d829 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:26.634300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-023835b2-1589-4761-9704-1f5bc563d829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:26.727681 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2818/5614] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:26.735959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad5cc-64a7-4f07-b97a-365f79c4b5a9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:26.736238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad5cc-64a7-4f07-b97a-365f79c4b5a9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:26.736515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad5cc-64a7-4f07-b97a-365f79c4b5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:26.760269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-838ad5cc-64a7-4f07-b97a-365f79c4b5a9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:26.761960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-838ad5cc-64a7-4f07-b97a-365f79c4b5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:26.775507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023835b2-1589-4761-9704-1f5bc563d829 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:26.783179 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2797/5615] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:26.788062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023835b2-1589-4761-9704-1f5bc563d829 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:26.793999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:26.794967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:26.795244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:26.795520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:26.796169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:26.796553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:26.798898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:26.821577 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2819/5616] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:18:26.822857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad5cc-64a7-4f07-b97a-365f79c4b5a9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:26.842774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad5cc-64a7-4f07-b97a-365f79c4b5a9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:26.855673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:26.856479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:26.856792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:26.857067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:26.857724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:26.859669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:26.902452 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2798/5617] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:26 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:26.906493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:26.917673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065388ac-b629-4a55-9522-b3c7c70d1c0c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:27.027653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:27.028697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:27.029243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:27.029759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:27.030668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:27.033211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:27.059996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:27.076532 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2820/5618] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:26 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:27.080704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:27.097635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c6678-0a71-42e7-b857-422f0f828ccc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:27.103087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:27.103719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:27.104005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:27.104279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:27.105120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:27.106843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:27.121923 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2799/5619] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:27.124252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:27.138972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ad9d3-512a-4239-8533-14e3d24ee38d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:27.145321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb53816-6adf-4dfa-a088-3894150e37a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:27.146725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53816-6adf-4dfa-a088-3894150e37a8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:27.148613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53816-6adf-4dfa-a088-3894150e37a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:27.150684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53816-6adf-4dfa-a088-3894150e37a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:27.151909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb53816-6adf-4dfa-a088-3894150e37a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:27.154185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb53816-6adf-4dfa-a088-3894150e37a8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:27.176616 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2821/5620] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:27.177987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:27.194485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b3c6f-fa8d-4590-832c-5710f31e6747 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:27.213302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:27.213983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:27.214422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:27.215937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] PATH_INFO: `/v3/projects/a069fad215134b79b6b4949f412edc53/users/426331348a8e4d4cba7022ad81ccd1f6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:27.235196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a069fad215134b79b6b4949f412edc53, user_id=426331348a8e4d4cba7022ad81ccd1f6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:27.238979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:27.267200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2800/5621] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:27 2022] PUT /identity/v3/projects/a069fad215134b79b6b4949f412edc53/users/426331348a8e4d4cba7022ad81ccd1f6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:18:27.280889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7536e53-04ff-4ac2-ac14-e1c7d28cdbfc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:27.281171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7536e53-04ff-4ac2-ac14-e1c7d28cdbfc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:27.281914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7536e53-04ff-4ac2-ac14-e1c7d28cdbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:27.323018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c7536e53-04ff-4ac2-ac14-e1c7d28cdbfc None None] MFA Rules not processed for user `426331348a8e4d4cba7022ad81ccd1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:27.324066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7536e53-04ff-4ac2-ac14-e1c7d28cdbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:27.544846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2822/5622] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:27.559555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:27.561664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:27.741415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb5de5-a22b-4585-bce0-511998c0a544 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:27.763467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08632f15-767b-479d-b42d-52f23129c126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:27.764277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632f15-767b-479d-b42d-52f23129c126 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:27.764683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632f15-767b-479d-b42d-52f23129c126 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:27.765267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632f15-767b-479d-b42d-52f23129c126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:27.809091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08632f15-767b-479d-b42d-52f23129c126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:27.811640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08632f15-767b-479d-b42d-52f23129c126 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:27.833861 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2801/5623] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:28.126393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7536e53-04ff-4ac2-ac14-e1c7d28cdbfc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:28.207983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7536e53-04ff-4ac2-ac14-e1c7d28cdbfc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:28.215167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:28.216243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:28.216863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:28.217392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] PATH_INFO: `/v3/users/426331348a8e4d4cba7022ad81ccd1f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:28.219696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=426331348a8e4d4cba7022ad81ccd1f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:28.222241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:28.305881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426331348a8e4d4cba7022ad81ccd1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:28.307899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426331348a8e4d4cba7022ad81ccd1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:28.322281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426331348a8e4d4cba7022ad81ccd1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:28.332990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426331348a8e4d4cba7022ad81ccd1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:28.334244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426331348a8e4d4cba7022ad81ccd1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:28.352405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:28.364193 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2823/5624] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:28 2022] DELETE /identity/v3/users/426331348a8e4d4cba7022ad81ccd1f6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:28.369320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:28.370225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:28.371825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:28.399212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:28.400806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:28.423959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:28.494706 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2802/5625] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:28.529913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:28.531519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:28.666086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed63b7-f4db-49f1-a46b-8237a1349107 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:28.681471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec6762-361f-4026-9bbb-58426714f227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:28.682149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6762-361f-4026-9bbb-58426714f227 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:28.682490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6762-361f-4026-9bbb-58426714f227 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:28.683231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6762-361f-4026-9bbb-58426714f227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:28.709938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec6762-361f-4026-9bbb-58426714f227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:28.711407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec6762-361f-4026-9bbb-58426714f227 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:28.730302 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2824/5626] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:30.378617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:30.380345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:30.402781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff1e9d-5334-4389-b297-2fbfa9cac7c4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:30.410078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:30.411959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:30.412825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:30.413363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] PATH_INFO: `/v3/projects/a069fad215134b79b6b4949f412edc53` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:30.415020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a069fad215134b79b6b4949f412edc53)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:30.417367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:30.545319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a069fad215134b79b6b4949f412edc53', 'request_id': 'req-9506277c-a8d0-40fb-82e9-1d36366ed7dc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:30.565758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a069fad215134b79b6b4949f412edc53', 'request_id': 'req-9506277c-a8d0-40fb-82e9-1d36366ed7dc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:30.592538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a069fad215134b79b6b4949f412edc53', 'request_id': 'req-9506277c-a8d0-40fb-82e9-1d36366ed7dc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:30.628588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] The token cache is being invalidate because project a069fad215134b79b6b4949f412edc53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:30.629174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:30.630139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9506277c-a8d0-40fb-82e9-1d36366ed7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:30.637631 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2803/5627] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:30 2022] DELETE /identity/v3/projects/a069fad215134b79b6b4949f412edc53 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:30.641505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08654943-0493-4cf5-939e-98ed4069cbc5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:30.642965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08654943-0493-4cf5-939e-98ed4069cbc5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:30.643480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08654943-0493-4cf5-939e-98ed4069cbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:30.673630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-08654943-0493-4cf5-939e-98ed4069cbc5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:30.675281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08654943-0493-4cf5-939e-98ed4069cbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:30.810204 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2825/5628] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:30.819296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c98cb4-108e-4917-b272-8936f82ebf65 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:30.819592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c98cb4-108e-4917-b272-8936f82ebf65 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:30.819879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c98cb4-108e-4917-b272-8936f82ebf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:30.860518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c7c98cb4-108e-4917-b272-8936f82ebf65 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:30.861819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c7c98cb4-108e-4917-b272-8936f82ebf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:30.881737 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2804/5629] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:30.882875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08654943-0493-4cf5-939e-98ed4069cbc5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:30.898049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08654943-0493-4cf5-939e-98ed4069cbc5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:30.908798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:30.909628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:30.910124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:30.910460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:30.911275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:30.911837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:30.913735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:30.938622 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2826/5630] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:30.950097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c98cb4-108e-4917-b272-8936f82ebf65 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:30.964824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c98cb4-108e-4917-b272-8936f82ebf65 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:30.978430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:30.979135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:30.980382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:30.980382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:30.980382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:30.985372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:31.044957 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2805/5631] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:30 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:31.049590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:31.066792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9ada1-5b03-4c07-a32d-488c4b649942 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:31.234624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:31.235254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:31.235636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:31.235973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:31.236865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:31.244957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:31.300368 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2827/5632] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:31 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:31.305663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:31.321786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6538b0-05cc-4e7d-8dec-b4994f86c087 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:31.337688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:31.338679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:31.339173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:31.339649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:31.340517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:31.343967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:31.372680 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2806/5633] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:31.374733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:31.394470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa543772-a62c-4fee-aaa6-d78d71cba1c8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:31.424841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca9d2e-89e7-439b-88fc-9b7dfa6f71dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:31.426914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca9d2e-89e7-439b-88fc-9b7dfa6f71dd admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:31.427774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca9d2e-89e7-439b-88fc-9b7dfa6f71dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:31.430286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca9d2e-89e7-439b-88fc-9b7dfa6f71dd admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:31.434255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca9d2e-89e7-439b-88fc-9b7dfa6f71dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:31.436260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca9d2e-89e7-439b-88fc-9b7dfa6f71dd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:31.476847 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2828/5634] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:31.480418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:31.498769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f07b58-14ab-4c38-8b2b-5e26d7566182 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:31.514592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:31.515558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:31.516132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:31.516627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] PATH_INFO: `/v3/projects/cac636bbca374e43a63450241b5c0aaa/users/bf21275945424f5e83bc37d7dfe0cd90/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:31.554513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac636bbca374e43a63450241b5c0aaa, user_id=bf21275945424f5e83bc37d7dfe0cd90, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:31.564406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:31.594029 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2807/5635] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:31 2022] PUT /identity/v3/projects/cac636bbca374e43a63450241b5c0aaa/users/bf21275945424f5e83bc37d7dfe0cd90/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:31.610770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9819-6874-4021-8817-d667bcd4f12c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:31.611579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9819-6874-4021-8817-d667bcd4f12c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:31.612114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9819-6874-4021-8817-d667bcd4f12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:31.628234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:31.674113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-11bf9819-6874-4021-8817-d667bcd4f12c None None] MFA Rules not processed for user `bf21275945424f5e83bc37d7dfe0cd90`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:31.675543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-11bf9819-6874-4021-8817-d667bcd4f12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:31.834488 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2829/5636] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:31.839214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:31.923685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:31.979982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e458e-12ac-449b-ab86-eed9ee198a99 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:31.987224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:31.988880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:31.989376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:31.990081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:31.991059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:31.993574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:32.092856 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2808/5637] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:31 2022] POST /identity/v3/projects => generated 389 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:32.099985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf9819-6874-4021-8817-d667bcd4f12c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:32.139269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf9819-6874-4021-8817-d667bcd4f12c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:32.288188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:32.289671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:32.290566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:32.290566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:32.292934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:32.298758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:32.344948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2830/5638] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:32 2022] POST /identity/v3/users => generated 301 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:32.349809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:32.372045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c477d-773c-4b3e-9053-8a3ca26fc8e5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:32.383763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:32.384926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:32.385694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:32.386117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:32.386996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:32.390177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:32.414000 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2809/5639] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:32.418053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:32.439051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38cfb0-d812-41a6-8b74-f199f02292ef admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:32.450003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:32.452777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:32.453639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:32.453639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] PATH_INFO: `/v3/projects/7d0015a5c4eb45e7a56470b27ed52f35/users/6f0718469be94286abd8f7c55671806e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:32.511487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d0015a5c4eb45e7a56470b27ed52f35, user_id=6f0718469be94286abd8f7c55671806e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:32.516745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:32.557382 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2831/5640] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:32 2022] PUT /identity/v3/projects/7d0015a5c4eb45e7a56470b27ed52f35/users/6f0718469be94286abd8f7c55671806e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:32.559844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:32.578956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e0267-d38d-4a59-be0e-be92ba5234a6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:32.753245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ea0b4-a2c2-462c-8a2e-a63fa5f9f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:32.754426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ea0b4-a2c2-462c-8a2e-a63fa5f9f4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:32.755219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ea0b4-a2c2-462c-8a2e-a63fa5f9f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:32.755703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ea0b4-a2c2-462c-8a2e-a63fa5f9f4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:32.756206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ea0b4-a2c2-462c-8a2e-a63fa5f9f4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:32.767432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ea0b4-a2c2-462c-8a2e-a63fa5f9f4a4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:32.795129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2810/5641] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:32.799197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:32.815181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913927c-2522-4224-af9f-333f409bda91 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:32.828716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:32.829657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:32.830091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:32.832463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] PATH_INFO: `/v3/projects/7d0015a5c4eb45e7a56470b27ed52f35/users/6f0718469be94286abd8f7c55671806e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:32.836133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d0015a5c4eb45e7a56470b27ed52f35, user_id=6f0718469be94286abd8f7c55671806e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:32.840359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:32.904448 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2832/5642] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:32 2022] PUT /identity/v3/projects/7d0015a5c4eb45e7a56470b27ed52f35/users/6f0718469be94286abd8f7c55671806e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:32.909030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61908c85-9b55-4ec2-bd4f-38644071e926 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:32.910683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61908c85-9b55-4ec2-bd4f-38644071e926 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:32.911150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61908c85-9b55-4ec2-bd4f-38644071e926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:32.995056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-61908c85-9b55-4ec2-bd4f-38644071e926 None None] MFA Rules not processed for user `6f0718469be94286abd8f7c55671806e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:32.996710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-61908c85-9b55-4ec2-bd4f-38644071e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:33.267010 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2811/5643] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:33.268338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:33.283150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633af9e-12b2-4f86-bc57-1f90886f74f7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:33.299131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:33.300106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:33.300391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:33.300698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:33.301382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:33.307511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:33.367187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2833/5644] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:33 2022] POST /identity/v3/domains => generated 275 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:33.369351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61908c85-9b55-4ec2-bd4f-38644071e926 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:33.387069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61908c85-9b55-4ec2-bd4f-38644071e926 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:33.565476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:33.692897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:33.694236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:33.694933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:33.695445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:33.696409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=6a853bf2d3b446109c31505e68cb9363)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:33.698636 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:33.698636 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:18:33.698636 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:18:33.700138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:33.850912 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2812/5645] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:18:33 2022] PUT /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363/config => generated 138 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:33.853797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:33.871489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf411d-9217-48a6-94d8-03d20d3299d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:33.882251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:33.884197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:33.886468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:33.886875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:33.891856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=6a853bf2d3b446109c31505e68cb9363, group=identity)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:33.895604 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:33.895604 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:18:33.895604 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:18:33.897511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:33.915490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:33.937140 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2834/5646] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:18:33 2022] GET /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/identity => generated 45 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:33.943301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0bc72-63e8-42d2-a117-21f47f1ff55f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:33.950399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:33.952019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:33.953082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:33.953824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/identity/driver` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:33.964398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=6a853bf2d3b446109c31505e68cb9363, group=identity, option=driver)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:33.966020 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:33.966020 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:18:33.966020 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:18:33.967401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:33.979708 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2813/5647] 10.222.0.26 () {58 vars in 1296 bytes} [Tue Sep 27 10:18:33 2022] GET /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/identity/driver => generated 31 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:33.985708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:34.004101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabb911-f15d-48cb-b9f5-6de015ed3152 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:34.014620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:34.015927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.016421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.016954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/ldap` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.018508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=6a853bf2d3b446109c31505e68cb9363, group=ldap)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:34.022150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:34.046875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2835/5648] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:18:33 2022] GET /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/ldap => generated 106 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:34.052420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:34.097321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f583d-537d-4cfc-990f-c679828d790c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:34.105816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:34.107001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.107899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.108273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/ldap/url` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.109892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=6a853bf2d3b446109c31505e68cb9363, group=ldap, option=url)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:34.113009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:34.124422 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2814/5649] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:18:34 2022] GET /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/ldap/url => generated 46 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:34.130626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:34.158674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cbb6a-a36b-4eab-a118-06f8e50b0f0a tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:34.167733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53729bcf-ba35-4f97-8147-4f3de6efa2e0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:34.168970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729bcf-ba35-4f97-8147-4f3de6efa2e0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.169391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729bcf-ba35-4f97-8147-4f3de6efa2e0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.169981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729bcf-ba35-4f97-8147-4f3de6efa2e0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/ldap/user_tree_dn` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.171363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53729bcf-ba35-4f97-8147-4f3de6efa2e0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=6a853bf2d3b446109c31505e68cb9363, group=ldap, option=user_tree_dn)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:34.173593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53729bcf-ba35-4f97-8147-4f3de6efa2e0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:34.188083 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2836/5650] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:18:34 2022] GET /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363/config/ldap/user_tree_dn => generated 63 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:34.192616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:34.207575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e8d9-a3cc-4f1c-9d24-e195ffff1573 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:34.213466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:34.215195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.215480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.215978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.248924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31d647-bc52-448b-abdd-9b519e8f556e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.249250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31d647-bc52-448b-abdd-9b519e8f556e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.249472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31d647-bc52-448b-abdd-9b519e8f556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.250423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=6a853bf2d3b446109c31505e68cb9363)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:34.251571 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:34.251571 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:18:34.251571 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:18:34.254178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:34.283692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:34.318586 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2815/5651] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:18:34 2022] DELETE /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363/config => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:34.322691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:34.325159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf31d647-bc52-448b-abdd-9b519e8f556e None None] MFA Rules not processed for user `6efd3d39610444b99c7bb245337c2ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:34.326953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf31d647-bc52-448b-abdd-9b519e8f556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:34.355938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960947a-4c90-4e54-a09b-2589a2052108 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:34.380066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:34.380972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.381258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.381536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.386868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6a853bf2d3b446109c31505e68cb9363)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:34.388527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:34.467077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6a853bf2d3b446109c31505e68cb9363'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:34.474703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6a853bf2d3b446109c31505e68cb9363'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:34.588389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6a853bf2d3b446109c31505e68cb9363'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:34.603203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6a853bf2d3b446109c31505e68cb9363'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:34.611398 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2816/5652] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:34 2022] PATCH /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363 => generated 276 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:34.617788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:34.619648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:34.639708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2837/5653] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:34.641316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.641610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.642179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.705194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] MFA Rules not processed for user `6efd3d39610444b99c7bb245337c2ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:34.707362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:34.811593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddbef0-0d03-4a16-ba39-3ff698bf409d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:34.827840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:34.828500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:34.828912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:34.829189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:34.879834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6a853bf2d3b446109c31505e68cb9363)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:34.882131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:34.889746 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2838/5654] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:34.905649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:34.915123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:34.954072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6a853bf2d3b446109c31505e68cb9363'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:34.963640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6a853bf2d3b446109c31505e68cb9363'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:34.982791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:35.124754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a853bf2d3b446109c31505e68cb9363', 'request_id': 'req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.132861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a853bf2d3b446109c31505e68cb9363', 'request_id': 'req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.152417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a853bf2d3b446109c31505e68cb9363', 'request_id': 'req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.172710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] The token cache is being invalidate because project 6a853bf2d3b446109c31505e68cb9363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:35.173050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.180027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.184175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b756bf4-c4ee-4d09-b329-56f0695206cf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:35.210543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6a853bf2d3b446109c31505e68cb9363', 'request_id': 'req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.211437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6a853bf2d3b446109c31505e68cb9363', 'request_id': 'req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.223647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6a853bf2d3b446109c31505e68cb9363', 'request_id': 'req-80268e2e-52b3-48fc-8ce6-82b6a8a85ac5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:35.238887 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2817/5655] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:34 2022] DELETE /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:35.241993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:35.257341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:35.258075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:35.331621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] MFA Rules not processed for user `7f35330693e947a6a7021290be4c946f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:35.331954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:35.368411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:35.457703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:35.459455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:35.459455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:35.459455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:35.463363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:35.502655 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2818/5656] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:35.506724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:35.507701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:35.715400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:35.717029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:35.751683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2839/5657] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 848 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:35.752747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:35.757085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:35.779845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631df1a-415b-4bd8-887d-08c45026027f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:35.802097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:35.802856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:35.803141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:35.803413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:35.804092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:35.805764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:35.851510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:35.860929 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2819/5658] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:35 2022] POST /identity/v3/domains => generated 276 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:35.866538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:35.984392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9312fe-8fbf-42a9-98bc-749d2856c80b service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.076229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce36f8-9799-4114-ac50-00128f7ec08e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.103756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.104979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.105274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.105557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.108387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.111973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.164518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.168070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.168985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.171090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.172940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.175760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.233683 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2820/5659] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:18:35 2022] PUT /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config => generated 138 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:36.235901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.260895 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2840/5660] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:35 2022] POST /identity/v3/projects => generated 381 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:36.267434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.275261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6c65-4ff6-498c-833f-44c5e283f114 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.290913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:36.406415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded5d58-e121-4aca-85ab-58e900ada2d1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.412767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.415084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.415862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.416149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.416932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.419176 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:36.419176 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:18:36.419176 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:18:36.420924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.422023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.423165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.423514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.423788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.424438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.426446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.483506 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2821/5661] 10.222.0.26 () {60 vars in 1271 bytes} [Tue Sep 27 10:18:36 2022] PATCH /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config => generated 142 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:18:36.485040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.495440 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2841/5662] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:36 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:36.498836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.507371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6995fb-5f24-414c-b32e-cc48edfcd89f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.520355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.521473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.524347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.524796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.526029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55c2d4-8a0a-4029-bd80-93ca6aa50c8d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.526907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.531795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.532634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.533032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.533363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.533801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.534610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.536948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.555439 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2822/5663] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:36 2022] GET /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config => generated 142 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:18:36.557114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.576866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945a362-e649-4d4c-ad85-a8e363c843b4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.582352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2842/5664] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:36.583790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.591745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.592594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.592965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.593341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.593962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.595901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.597181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8a33-f28f-4bcd-9eeb-dc859cb99046 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.628886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2823/5665] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:18:36 2022] DELETE /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:36.629947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.633683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.634716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.635222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.635844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] PATH_INFO: `/v3/projects/1097aafb312045ddb9210e60b6af5e60/users/3d0ab16486364c59909350229b72f376/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.645399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343f851-8285-41f5-a8a6-b3de28a0d593 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.663609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.663817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.664036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.664379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.666409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.679872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.690598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1097aafb312045ddb9210e60b6af5e60, user_id=3d0ab16486364c59909350229b72f376, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.699419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Could not find any options in domain configuration for domain 901250ba7bc6453aa83b97c3097c8ecf.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 901250ba7bc6453aa83b97c3097c8ecf. Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 27 10:18:36.715295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 901250ba7bc6453aa83b97c3097c8ecf. Sep 27 10:18:36.716926 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:36.728361 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2824/5666] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:36 2022] GET /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config => generated 151 bytes in 99 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:18:36.732503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.737887 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2843/5667] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:36 2022] PUT /identity/v3/projects/1097aafb312045ddb9210e60b6af5e60/users/3d0ab16486364c59909350229b72f376/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:36.739008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.741028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:36.756350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873befce-a2d8-4ce1-9d93-a71a7da2bb2b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.786037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:36.838757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa80a-f6e3-41e3-9bd2-02d5c32947f3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:36.851038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.851730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.852011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.852262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.857920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:36.882283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:36.882668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:36.885035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:36.885035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:36.887745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:36.890288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:36.909662 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2825/5668] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:18:36 2022] DELETE /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf/config => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:18:36.911733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:36.997843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:37.001054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:37.025435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2844/5669] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:37.026465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:37.057527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec0307-6ae5-4455-a630-93f893223254 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:37.067401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9e151-d66b-4d50-9987-d0fbc701b20d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:37.070388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:37.071736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:37.072157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:37.072639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:37.074751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:37.076794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:37.097151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:37.098110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:37.099396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:37.099892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:37.102119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:37.102453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:37.104156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:37.115057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.117155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.134380 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2826/5670] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:37.135303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:37.142322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:37.172424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.173903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9c137-c9d3-4d34-88f8-71a149130f8f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:37.174301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.180420 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2845/5671] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:37 2022] PATCH /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf => generated 277 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:37.181341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:37.186278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:37.295868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:37.298771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:37.299776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:37.300262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:37.304421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:37.333068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9df02-9ecb-46cc-899d-1c84f4b6eb56 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:37.335223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:37.337328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:37.346910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:37.348412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:37.348611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:37.348960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] PATH_INFO: `/v3/projects/1097aafb312045ddb9210e60b6af5e60/users/3d0ab16486364c59909350229b72f376/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:37.351460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1097aafb312045ddb9210e60b6af5e60, user_id=3d0ab16486364c59909350229b72f376, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:37.355606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:37.363996 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2827/5672] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:37.365884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:37.380735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:37.382195 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2846/5673] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:37 2022] PUT /identity/v3/projects/1097aafb312045ddb9210e60b6af5e60/users/3d0ab16486364c59909350229b72f376/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:37.495270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:37.535334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b03af-290c-4521-ba5a-12062bd64fda service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:37.680471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:37.681315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:37.681666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:37.682661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:37.707022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=901250ba7bc6453aa83b97c3097c8ecf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:37.712579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:37.810520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.817649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.901684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf', 'request_id': 'req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.915040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf', 'request_id': 'req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.930391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf', 'request_id': 'req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.950802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] The token cache is being invalidate because project 901250ba7bc6453aa83b97c3097c8ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:37.951576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.952784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.966412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf', 'request_id': 'req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.967181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf', 'request_id': 'req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.972503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '901250ba7bc6453aa83b97c3097c8ecf', 'request_id': 'req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:37.982863 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2828/5674] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:37 2022] DELETE /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:37.988178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:37.991004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:38.018262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cab789-1422-4320-a112-7df2f34cf720 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:38.142103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:38.142778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:38.143135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:38.143699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:38.144382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:38.151175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:38.179926 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2847/5675] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:37 2022] POST /identity/v3/domains => generated 277 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:38.184245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:38.201801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c794-c7d7-496f-abf4-b0b52ec1b7fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:38.294086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:38.346763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:38.347597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:38.348045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:38.348324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:38.349093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:38.351012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:38.425844 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2829/5676] 10.222.0.26 () {60 vars in 1269 bytes} [Tue Sep 27 10:18:38 2022] PUT /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config => generated 138 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:38.432982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:38.447975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef64424-59cc-4e54-a2cc-332c19f9bcb1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:38.458691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:38.461614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:38.463859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:38.464155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:38.466623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e, group=identity)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:38.469043 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:38.469043 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:18:38.469043 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:18:38.471129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:38.575480 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2848/5677] 10.222.0.26 () {60 vars in 1297 bytes} [Tue Sep 27 10:18:38 2022] PATCH /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity => generated 177 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:38.577548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:38.603929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c345964-a75d-4158-941d-e7332fb00172 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:38.610181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:38.611093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:38.611385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:38.611649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity/driver` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:38.612398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e, group=identity, option=driver)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:38.615085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:38.762561 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2830/5678] 10.222.0.26 () {60 vars in 1318 bytes} [Tue Sep 27 10:18:38 2022] PATCH /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity/driver => generated 177 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:38.766226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:38.792002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c913-4e42-4f9b-98bf-be61352a5425 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:38.816187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:38.819699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:38.822356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:38.824294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity/driver` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:38.824998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e, group=identity, option=driver)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:38.826043 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:38.826043 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:18:38.826043 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:18:38.826948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:38.881285 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2849/5679] 10.222.0.26 () {58 vars in 1299 bytes} [Tue Sep 27 10:18:38 2022] DELETE /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity/driver => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:18:38.885385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:38.914676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603ec2-1116-4ae2-a0a1-680b1ce389e9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:38.942492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:38.943235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:38.945315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:38.945612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity/driver` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:38.949844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e, group=identity, option=driver)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:38.953601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Could not find option driver in group identity in domain configuration for domain cd99a041d0af425aa1999dc0d5deba5e.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain cd99a041d0af425aa1999dc0d5deba5e. Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:38.984090 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain cd99a041d0af425aa1999dc0d5deba5e. Sep 27 10:18:38.985223 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:38.993582 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2831/5680] 10.222.0.26 () {58 vars in 1296 bytes} [Tue Sep 27 10:18:38 2022] GET /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity/driver => generated 171 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:18:38.997591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:39.012914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4302d-51cb-432f-a2a0-d29b925140fe tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:39.027722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:39.028384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:39.028693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:39.029141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:39.030218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e, group=identity)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:39.064243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:39.073393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:UPDATE=1,keystone:INSERT=4,keystone:DELETE=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:39.103884 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2850/5681] 10.222.0.26 () {58 vars in 1278 bytes} [Tue Sep 27 10:18:38 2022] DELETE /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:39.108902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:39.126787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6688c48-1fbd-4904-adba-18826d31b527 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:39.138448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:39.139054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:39.139364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:39.139609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:39.141916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e, group=identity)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:39.146593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Could not find group identity in domain configuration for domain cd99a041d0af425aa1999dc0d5deba5e.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain cd99a041d0af425aa1999dc0d5deba5e. Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:39.166797 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain cd99a041d0af425aa1999dc0d5deba5e. Sep 27 10:18:39.167877 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:39.172380 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2832/5682] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:18:39 2022] GET /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config/identity => generated 154 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:18:39.173463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:39.194251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8a6c-611c-41f0-9758-a4df2a6c8105 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:39.211119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:39.212108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:39.212908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:39.213406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:39.214626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=cd99a041d0af425aa1999dc0d5deba5e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:39.219455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:39.244979 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2851/5683] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:18:39 2022] DELETE /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e/config => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:39.250014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:39.265467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c41e7-5950-4a36-8a58-8e00d51899fd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:39.272015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:39.272872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:39.273227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:39.273678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:39.297370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=cd99a041d0af425aa1999dc0d5deba5e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:39.299565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:39.333002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.333894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.384178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.384971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.392712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:39.393484 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2833/5684] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:39 2022] PATCH /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e => generated 278 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:39.395283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:39.423765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a26f4-a67e-476d-9f6f-35ca916a2d7c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:39.563809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:39.565880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:39.566592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:39.567100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:39.592082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=cd99a041d0af425aa1999dc0d5deba5e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:39.597038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:39.662386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.686729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.774349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e', 'request_id': 'req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.795926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e', 'request_id': 'req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.815474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e', 'request_id': 'req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.845555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] The token cache is being invalidate because project cd99a041d0af425aa1999dc0d5deba5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:39.846108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.854373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.888173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e', 'request_id': 'req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.890432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e', 'request_id': 'req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.905046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'cd99a041d0af425aa1999dc0d5deba5e', 'request_id': 'req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:39.924677 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2852/5685] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:39 2022] DELETE /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:39.930122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:39.950612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ae2ba-b48b-4f7b-a322-9d8e70494367 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.051711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.052647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.053100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.053483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.054439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.055725 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:40.055725 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:18:40.055725 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:18:40.059382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.070268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.071986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:40.079992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2834/5686] 10.222.0.26 () {58 vars in 1173 bytes} [Tue Sep 27 10:18:39 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:40.107680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca3cd4-ed06-4a6d-b242-e3de76bf5f95 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.118809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.119777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.120198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.120570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.125949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.127494 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:18:40.127494 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:18:40.127494 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:18:40.130786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.134060 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2853/5687] 10.222.0.26 () {58 vars in 1200 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.138975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.151702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9fbc0-a7ae-4318-b857-8440815f2e47 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.164141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.165435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.165934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.166446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.167361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.169845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.178747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.188014 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2835/5688] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.193208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8aba5-9f3a-4057-8eaf-1dcdd01d2348 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.198993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.199941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.200521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.201441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.201593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.206220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.211727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.221136 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2854/5689] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.226771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e859bd2-acdb-4c6d-8138-01e86945e933 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.233528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.234306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.234731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.235105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.235797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.237863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.243922 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2836/5690] 10.222.0.26 () {58 vars in 1188 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:40.245734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.262275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2c8a4-9e2a-4e3c-ad95-108f1556a0e4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.277284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.278350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.278634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.278908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.279539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.281346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.282895 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2855/5691] 10.222.0.26 () {58 vars in 1200 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:18:40.287311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.299888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d6932-77d2-43b3-8713-cf611fed14d8 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.312063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.314737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.314942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.315245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.315861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.320357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.339304 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2837/5692] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.344311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.352164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:40.357521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fafe0-f348-4f53-aad0-101393941ed6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.374181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.375524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.375937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.376350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.377332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.381404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.387852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.398878 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2856/5693] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.401166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e98f6-c959-4120-ba13-0904c2ce999c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.407283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.408050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.408413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.409035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.409912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.412494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.419564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.446812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2838/5694] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.457695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da188b42-ab82-4632-8959-001eb82745d9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.469647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.470874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.471813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.472451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.473450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.475939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.485321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.500118 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2857/5695] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.504350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7799b43-147d-41da-b1b4-b5f44e7cf3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.512181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.513631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.514119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.514621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.515665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.517796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.528273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.536570 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2839/5696] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.542284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd650106-9e54-409b-83ea-a11c38820a14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.548988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.550803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.551357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.551844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.553318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.555343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.557000 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2858/5697] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.562377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.589193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92f9b2-f2db-41e3-8b57-feaa7d6468cc tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.597179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.598449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.619302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.619626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.621490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.625619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.629314 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2840/5698] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.633745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.647181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae4665-fadc-4aa5-a837-4f2e3c94a3e1 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.652110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.653305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.653305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.653525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.653933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.655608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.661135 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2859/5699] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.662177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.676753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:40.683791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb76545-9152-4a94-a3e9-faf72e46760d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.694808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.703945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.705269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.706744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.707760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.710722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.713111 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2841/5700] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:18:40.717742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.733414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25b438-acb4-408d-8e92-2ce6d5606f99 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.740395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.741571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.741952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.742327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.743044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.744817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.750995 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2860/5701] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.752211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.769713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96ec5b-026b-4f6c-992c-ef78339659f5 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.775972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.776819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.777226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.777962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.778677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.780744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.787616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.797799 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2842/5702] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.801105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba91f8-1bcc-4499-be45-54c2d25e64c6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.811006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.813305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.813692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.817492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.819240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.822442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.827745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2861/5703] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.831966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.848033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43794c-9d42-4501-a377-0c16e3ca09cd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.854186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.854859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.855141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.855424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.856026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.857730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.860941 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2843/5704] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.865131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.882665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8828ec3-d584-42d2-beb6-64f56dcf6306 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.889409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.890082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.890366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.890648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.891405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.894123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.898417 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2862/5705] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.900490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.935348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b6e2b-e2ae-4035-90d2-6fb7af90506c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:40.947432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:40.948395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:40.948942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:40.949740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:40.950541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:40.954052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:40.956352 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2844/5706] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:40.960565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:40.993061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559ed7-f238-4717-802c-df24a0018a4e tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.004746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.007694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.007694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.008577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.010838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.013552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.020711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2863/5707] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 10:18:40 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.023605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.036828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acbee1-e996-4783-b7d6-48373e4629bd tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.046250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.046720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.047661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.048070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.048871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.051822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.052701 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2845/5708] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.056530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.078024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5286f-b4ee-4935-8ebb-370cec253537 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.091515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.093407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.099620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.100472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.101247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.102950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.110156 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2864/5709] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.112701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.131653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec53afc-2411-46d7-befc-7b44186bd0d4 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.138463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.139489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.139950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.140403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.141206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.144231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.146996 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2846/5710] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:18:41.151030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.167643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743e7c4-e612-4f30-96b0-66f1c6f5498f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.183890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.185033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.185837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.186349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.187255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.189694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.194008 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2865/5711] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.199074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.232801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228d858-6dc4-42f9-9802-0aac1011914f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.238542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.239455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.239901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.240304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.241051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.242803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.244766 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2847/5712] 10.222.0.26 () {58 vars in 1290 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.248468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.266218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb7937-8129-4186-918c-190cdeb38d63 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.270870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.272922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.273282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.273815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.274488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.276285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.277802 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2866/5713] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.281903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.297486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cfc1d-40ba-49a4-a669-5f6cbccdb8f2 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.333107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.334424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.334718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.335000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.335611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.341251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.342633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2848/5714] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.347888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.364352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849090db-faac-449e-93a7-cd6ca590ab14 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.377285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.378275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.378404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.378671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.379335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.381076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.382321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2867/5715] 10.222.0.26 () {58 vars in 1308 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.386495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.415131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a90a6-5fb0-40a0-9f1d-170a0e2cae72 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.429891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.430581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.430865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.431144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.431737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.440969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.445419 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2849/5716] 10.222.0.26 () {58 vars in 1290 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.447140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.463904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f620c-300f-4618-b14f-af0cd4fcaeff tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.481529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.482474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.482909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.483436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.484181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.486069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.487614 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2868/5717] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:18:41.492144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.508535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eb5f4-da7c-4e6e-b45b-bc6477d43eae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.529373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.530603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.531113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.531578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.532450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.536846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.539595 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2850/5718] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.544499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.567209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac75b1-9c1f-4a52-b40c-841362b59dd3 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.573747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.574126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.574336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.574707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.575191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.577634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.584468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.588830 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2869/5719] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.594738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444aa47-c3ba-42cf-8807-b1c85a942141 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.612760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.613441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.614160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.614160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.614584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.617103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.623620 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2851/5720] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.624935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.644041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d455-8d05-4d9a-8f0c-f8e6b8f9c6ca tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.656632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.657342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.657624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.657897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.658497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.660134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.661403 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2870/5721] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.666020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.683148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82000762-e9e0-456c-9295-fefa28c3da3c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.695907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.696625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.697026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.697213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.700076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.700076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.703883 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2852/5722] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.705128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.719682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f436a09-25de-4de5-bf15-1afbe7c85b4b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.744205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.744933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.745211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.745486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.746079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.748100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.765747 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2871/5723] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.770004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.786813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d954f-328b-4a2b-9aa3-0295acd6def6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.800467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.805489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.806191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.806781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.807600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.815381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.828736 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2853/5724] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.830719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.850698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958ab56-ebe0-459d-b667-0472251c4af9 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.876005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.876714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.877066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.877338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.877945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.879609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.885213 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2872/5725] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:18:41.886123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.899594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320dc4-fca1-40d9-9c02-203201e5460b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.915446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.916153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.916518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.916752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.917903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.920156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.923582 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2854/5726] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.927874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.944768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd564e-c91a-446e-a7b5-598d4a640762 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:41.962213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:41.963260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:41.966253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:41.966803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:41.967598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:41.969460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:41.977086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2873/5727] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:41.978368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:41.994439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d4fcc-4f83-4086-b195-4c34a96a63e6 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.014702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.015401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.015686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.015965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.016574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.018275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.023811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.024263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2855/5728] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:41 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.047695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649cdf2-bc27-411f-b356-04c101a9fa8d tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.060997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.061675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.061954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.062233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.062818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.064645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.066537 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2874/5729] 10.222.0.26 () {58 vars in 1212 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:18:42.070954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.086989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4716a2-b0b4-4ffb-8fa8-85abd585323c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.102762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.104113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.106260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.106787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.107602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.113182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.118513 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2856/5730] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.122774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.135849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c13ec-04da-4632-962c-abc0d45aa3b0 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.152180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.153843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.154339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.154805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.155596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.157554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.159395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2875/5731] 10.222.0.26 () {58 vars in 1215 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.164175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.176527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a5672-e7a8-4b43-a2f6-1491e82ef031 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.181878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.183725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.184222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.184691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.185781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.187756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.189497 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2857/5732] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.193368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.206134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fc075-1a65-44b1-9a97-12490d3fdb02 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.211753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.212756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.213270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.213727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.214482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.216705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.218705 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2876/5733] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:18:42.223600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.237369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7e4b5-c957-4f8f-aff5-3b993b92764b tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.243553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.244715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.245240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.246040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.246860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.250643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.253450 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2858/5734] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.257795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.272102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac1f53-78fa-4d74-a699-e1cde14f284c tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.278244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.279630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.280133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.280748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.281592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.283489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.285121 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2877/5735] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.289958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.304022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ef01e-6b12-497e-a73c-11015c73396f tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.313362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.314501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.316158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.316158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.316158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.320126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.322765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2859/5736] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.326869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.342503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346d6d4-30fc-454b-abcc-1d630ff1ed03 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.367648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.367648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.367648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.368085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.368356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.370067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.374331 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2878/5737] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.375940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.391809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814a09d-a032-4650-bb24-a65f170b3826 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.410467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.411164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.411477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.411757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.412407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.414135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.419373 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2860/5738] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.420157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.432696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70381c-299b-47bc-8dc8-a900207d6110 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.437948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.438614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.438895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.439167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.439761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.441413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.448200 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2879/5739] 10.222.0.26 () {58 vars in 1278 bytes} [Tue Sep 27 10:18:42 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 27 10:18:42.449905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.469972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe7a7-c087-478b-87d5-07998407d4ae tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.479860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.480571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.481187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.481817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/cd99a041d0af425aa1999dc0d5deba5e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.492849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Could not find domain: cd99a041d0af425aa1999dc0d5deba5e.: keystone.exception.DomainNotFound: Could not find domain: cd99a041d0af425aa1999dc0d5deba5e. Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: cd99a041d0af425aa1999dc0d5deba5e. Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:42.501495 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:42.502691 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:18:42.507917 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: cd99a041d0af425aa1999dc0d5deba5e. Sep 27 10:18:42.509015 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:42.509015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2861/5740] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:42 2022] PATCH /identity/v3/domains/cd99a041d0af425aa1999dc0d5deba5e => generated 112 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:18:42.510566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.530336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d8eb8-ce9b-45fb-91d1-62d122bc9507 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.536044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.537681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.538186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.538673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/901250ba7bc6453aa83b97c3097c8ecf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.548050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Could not find domain: 901250ba7bc6453aa83b97c3097c8ecf.: keystone.exception.DomainNotFound: Could not find domain: 901250ba7bc6453aa83b97c3097c8ecf. Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 901250ba7bc6453aa83b97c3097c8ecf. Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:42.556207 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:42.557500 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:18:42.558849 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 901250ba7bc6453aa83b97c3097c8ecf. Sep 27 10:18:42.559885 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:42.562454 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2880/5741] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:42 2022] PATCH /identity/v3/domains/901250ba7bc6453aa83b97c3097c8ecf => generated 112 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:18:42.567226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.582119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a9643-b87c-4755-9bd4-967f0b472b70 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.588317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9be1d-2c41-4892-ae73-0ae3698bf986 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f0718469be94286abd8f7c55671806e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d0015a5c4eb45e7a56470b27ed52f35', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.588990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9be1d-2c41-4892-ae73-0ae3698bf986 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.589429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9be1d-2c41-4892-ae73-0ae3698bf986 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.589722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9be1d-2c41-4892-ae73-0ae3698bf986 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] PATH_INFO: `/v3/domains/6a853bf2d3b446109c31505e68cb9363` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.610704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9be1d-2c41-4892-ae73-0ae3698bf986 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-09a9be1d-2c41-4892-ae73-0ae3698bf986 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Could not find domain: 6a853bf2d3b446109c31505e68cb9363.: keystone.exception.DomainNotFound: Could not find domain: 6a853bf2d3b446109c31505e68cb9363. Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6a853bf2d3b446109c31505e68cb9363. Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:42.619025 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:42.620075 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:18:42.621161 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 6a853bf2d3b446109c31505e68cb9363. Sep 27 10:18:42.623044 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:42.627637 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2862/5742] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:42 2022] PATCH /identity/v3/domains/6a853bf2d3b446109c31505e68cb9363 => generated 112 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:18:42.629397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:42.631393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:42.770886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d55390-3c41-4f98-b2c2-d5680e2d8576 tempest-DomainConfigurationTestJSON-550855305 tempest-DomainConfigurationTestJSON-550855305-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:42.789835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:42.790652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:42.790790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:42.791052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] PATH_INFO: `/v3/users/bf21275945424f5e83bc37d7dfe0cd90` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:42.797855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf21275945424f5e83bc37d7dfe0cd90)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:42.801289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:42.973436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf21275945424f5e83bc37d7dfe0cd90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:42.974475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf21275945424f5e83bc37d7dfe0cd90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:42.980835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:42.992748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf21275945424f5e83bc37d7dfe0cd90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:43.021707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf21275945424f5e83bc37d7dfe0cd90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:43.022922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf21275945424f5e83bc37d7dfe0cd90'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:43.065301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:43.066545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:43.067012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:43.077668 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2881/5743] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:42 2022] DELETE /identity/v3/users/bf21275945424f5e83bc37d7dfe0cd90 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:43.117128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:43.118390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:43.205376 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2863/5744] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:43.286555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:43.286555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:43.396984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ec0b7-6bba-49d2-bdb8-d703be216385 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:43.408351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:43.410893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:43.411183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:43.411607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:43.449784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:43.451618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:43.485922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2882/5745] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:45.414286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:45.416419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:45.437831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cefe3-90c0-4a97-a83d-06c5c5e85e41 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:45.457247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:45.457918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:45.458197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:45.458780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] PATH_INFO: `/v3/projects/cac636bbca374e43a63450241b5c0aaa` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:45.459905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac636bbca374e43a63450241b5c0aaa)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:45.462240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:45.594169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac636bbca374e43a63450241b5c0aaa', 'request_id': 'req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.600113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:45.617562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac636bbca374e43a63450241b5c0aaa', 'request_id': 'req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.635595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac636bbca374e43a63450241b5c0aaa', 'request_id': 'req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.662705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] The token cache is being invalidate because project cac636bbca374e43a63450241b5c0aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:45.663214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.664413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf75f465-545b-4c8d-aa4c-8ce4a73d29bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.667565 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2864/5746] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:45 2022] DELETE /identity/v3/projects/cac636bbca374e43a63450241b5c0aaa => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:45.671346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:45.686564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac067b-9741-4913-ae0f-6dad3279753f service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:45.815407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:45.816267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:45.816551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:45.816854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] PATH_INFO: `/v3/users/6f0718469be94286abd8f7c55671806e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:45.819964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f0718469be94286abd8f7c55671806e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:45.821986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:45.963915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f0718469be94286abd8f7c55671806e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.964635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f0718469be94286abd8f7c55671806e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.979654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f0718469be94286abd8f7c55671806e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.995281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f0718469be94286abd8f7c55671806e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:45.996017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f0718469be94286abd8f7c55671806e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:46.010568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:46.044703 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2883/5747] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:45 2022] DELETE /identity/v3/users/6f0718469be94286abd8f7c55671806e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:46.541425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58597889-fe04-45c4-a0ed-902f4ae09637 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:46.541992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58597889-fe04-45c4-a0ed-902f4ae09637 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:46.541992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58597889-fe04-45c4-a0ed-902f4ae09637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:46.584946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-58597889-fe04-45c4-a0ed-902f4ae09637 None None] MFA Rules not processed for user `dbf876e1c23642578363f9c5995f3d86`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:46.586190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-58597889-fe04-45c4-a0ed-902f4ae09637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:46.654047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:46.656203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:46.748698 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2865/5748] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:46.808776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508572-69f5-4dc7-9e74-d487d44f18c0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:46.814274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:46.815034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:46.817029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:46.817516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] PATH_INFO: `/v3/projects/7d0015a5c4eb45e7a56470b27ed52f35` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:46.819194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d0015a5c4eb45e7a56470b27ed52f35)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:46.821688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:46.948376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d0015a5c4eb45e7a56470b27ed52f35', 'request_id': 'req-1ca7a412-38db-4b88-9b30-e0945e602891'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:46.975193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d0015a5c4eb45e7a56470b27ed52f35', 'request_id': 'req-1ca7a412-38db-4b88-9b30-e0945e602891'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:46.988468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d0015a5c4eb45e7a56470b27ed52f35', 'request_id': 'req-1ca7a412-38db-4b88-9b30-e0945e602891'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:47.011517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] The token cache is being invalidate because project 7d0015a5c4eb45e7a56470b27ed52f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:47.012595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:47.015836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ca7a412-38db-4b88-9b30-e0945e602891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:47.017772 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2884/5749] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:46 2022] DELETE /identity/v3/projects/7d0015a5c4eb45e7a56470b27ed52f35 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:47.023692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576973-b9ce-4f57-aa49-c07e1bc9b29d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.024893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576973-b9ce-4f57-aa49-c07e1bc9b29d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.025272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576973-b9ce-4f57-aa49-c07e1bc9b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.065413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-44576973-b9ce-4f57-aa49-c07e1bc9b29d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:47.067317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-44576973-b9ce-4f57-aa49-c07e1bc9b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:47.262545 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2866/5750] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:47.274320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b617283-d6e8-4257-9de4-95a860c260e2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.274603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b617283-d6e8-4257-9de4-95a860c260e2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.274885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b617283-d6e8-4257-9de4-95a860c260e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.342747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1b617283-d6e8-4257-9de4-95a860c260e2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:47.345248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1b617283-d6e8-4257-9de4-95a860c260e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:47.359412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576973-b9ce-4f57-aa49-c07e1bc9b29d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:47.360581 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2885/5751] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:47.371495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576973-b9ce-4f57-aa49-c07e1bc9b29d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:47.379598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:47.380250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.381341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.381341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.384045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:47.384294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:47.388317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:47.412224 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2867/5752] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:47.413601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b617283-d6e8-4257-9de4-95a860c260e2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:47.426543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b617283-d6e8-4257-9de4-95a860c260e2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:47.431911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:47.441552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.442085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.442359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.443162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:47.445998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:47.493137 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2886/5753] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:47 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:47.496137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:47.510527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e4fcb-9984-4375-a2f0-822fd402b1a0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:47.642562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:47.643221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.643440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.644541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.645693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:47.650349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:47.700501 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2868/5754] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:47 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:47.702069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:47.718901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981362f8-ae56-46ca-92b2-1575459ae03c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:47.729876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:47.730750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.731219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.731490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.732386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:47.734133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:47.749226 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2887/5755] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:47.752526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:47.758788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:47.767559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29d54f-86bb-47b1-bebf-4f140fd4b7a4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:47.777391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400909-1521-40a7-89f1-374c85e0f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:47.778574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400909-1521-40a7-89f1-374c85e0f7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.779100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400909-1521-40a7-89f1-374c85e0f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.779726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400909-1521-40a7-89f1-374c85e0f7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.781849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400909-1521-40a7-89f1-374c85e0f7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:47.785680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400909-1521-40a7-89f1-374c85e0f7c2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:47.807287 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2869/5756] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:47.808769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:47.827865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914cd3b-ca9d-4d94-b5aa-4eedec141b62 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:47.857361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:47.858169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.858308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.858578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] PATH_INFO: `/v3/projects/4e62544719584ef39a24e7969ecacd7c/users/d4a85d8c08dc4becaa84b89cfdb986c0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:47.905818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e62544719584ef39a24e7969ecacd7c, user_id=d4a85d8c08dc4becaa84b89cfdb986c0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:47.909962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:47.952824 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2888/5757] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:47 2022] PUT /identity/v3/projects/4e62544719584ef39a24e7969ecacd7c/users/d4a85d8c08dc4becaa84b89cfdb986c0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:47.966182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e066194-b65c-4b50-a2ab-3551a8fb61cc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:47.966469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e066194-b65c-4b50-a2ab-3551a8fb61cc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:47.966742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e066194-b65c-4b50-a2ab-3551a8fb61cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:48.032672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4e066194-b65c-4b50-a2ab-3551a8fb61cc None None] MFA Rules not processed for user `d4a85d8c08dc4becaa84b89cfdb986c0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:48.034217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4e066194-b65c-4b50-a2ab-3551a8fb61cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:48.217698 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2870/5758] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:48.218823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:48.299473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:48.409511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68665b9-3317-4381-9079-9950b9a73c92 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:48.420925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:48.421584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:48.422053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:48.422329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:48.425979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:48.428450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:48.466603 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2889/5759] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:48 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:48.470551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e066194-b65c-4b50-a2ab-3551a8fb61cc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:48.483274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e066194-b65c-4b50-a2ab-3551a8fb61cc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:48.557086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:48.557960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:48.558284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:48.558563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:48.559562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:48.562175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:48.603169 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2871/5760] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:48 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:48.606817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:48.625597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4a89-31a6-4f7b-99c0-35808a5e640e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:48.647160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:48.648435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:48.648740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:48.649553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:48.653148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:48.655785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:48.687555 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2890/5761] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:48.691603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:48.711249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c193ed-692c-4d3e-948b-8ad9a5585fc8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:48.720691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:48.721692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:48.722128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:48.722586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] PATH_INFO: `/v3/projects/c4d35dcbb5c84c54aa345c7961d020cf/users/3f291437a37d4268b5f90ae7934b4c97/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:48.750723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d35dcbb5c84c54aa345c7961d020cf, user_id=3f291437a37d4268b5f90ae7934b4c97, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:48.754545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:48.783426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2872/5762] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:48 2022] PUT /identity/v3/projects/c4d35dcbb5c84c54aa345c7961d020cf/users/3f291437a37d4268b5f90ae7934b4c97/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:48.787878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:48.804295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe569a3-2301-4dfc-b34a-e347b520cf08 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:48.915822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc171d-33ba-40f6-bc23-6081c7b39b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:48.916446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc171d-33ba-40f6-bc23-6081c7b39b28 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:48.919044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc171d-33ba-40f6-bc23-6081c7b39b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:48.920894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc171d-33ba-40f6-bc23-6081c7b39b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:48.921741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc171d-33ba-40f6-bc23-6081c7b39b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:48.949541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc171d-33ba-40f6-bc23-6081c7b39b28 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:48.964336 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2891/5763] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:48.968492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:48.984743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f5a7e-12a8-4123-84aa-645fcd5d88f8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:49.008783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:49.012496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:49.013830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:49.015181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] PATH_INFO: `/v3/projects/c4d35dcbb5c84c54aa345c7961d020cf/users/3f291437a37d4268b5f90ae7934b4c97/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:49.024465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d35dcbb5c84c54aa345c7961d020cf, user_id=3f291437a37d4268b5f90ae7934b4c97, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:49.039757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:49.095587 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2873/5764] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:48 2022] PUT /identity/v3/projects/c4d35dcbb5c84c54aa345c7961d020cf/users/3f291437a37d4268b5f90ae7934b4c97/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:49.110842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a315305-e621-437b-ae64-4dc71546ab8d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:49.111122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a315305-e621-437b-ae64-4dc71546ab8d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:49.111579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a315305-e621-437b-ae64-4dc71546ab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:49.191676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1a315305-e621-437b-ae64-4dc71546ab8d None None] MFA Rules not processed for user `3f291437a37d4268b5f90ae7934b4c97`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:49.201126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1a315305-e621-437b-ae64-4dc71546ab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:49.357844 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2892/5765] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:49.362748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:49.378073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266d5f-0d3c-426b-8486-2fdfe739482e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:49.394189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:49.394862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:49.395149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:49.395560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:49.396362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:49.398113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invalid input for field 'name': '' is too short Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: On instance['name']: Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: On instance['name']: Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: '' Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise error Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application '' Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:49.408160 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:18:49.409745 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application '' Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:49.411469 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2874/5766] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:49 2022] POST /identity/v3/domains => generated 283 bytes in 48 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:18:49.417088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a315305-e621-437b-ae64-4dc71546ab8d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:49.430109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a315305-e621-437b-ae64-4dc71546ab8d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:49.437353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:49.438208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:49.438668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:49.439097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:49.440082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:49.442176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: On instance['name']: Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: On instance['name']: Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise error Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.451634 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 27 10:18:49.453504 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application On instance['name']: Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:18:49.454618 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.462268 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2893/5767] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:49 2022] POST /identity/v3/domains => generated 412 bytes in 42 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 27 10:18:49.463160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:49.487994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b70457-781f-4dfd-8553-ead63602452b tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:49.508971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:49.509636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:49.509940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:49.510213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:49.510898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:49.513301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:49.555491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2875/5768] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:49 2022] POST /identity/v3/domains => generated 276 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:49.559526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:49.579483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae476cb-6a71-4501-a822-3c42767017ad tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:49.595315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:49.690976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:49.691600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:49.691882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:49.693056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains/0791469814fa4909be1dcd7e0b207e91` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:49.716520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0791469814fa4909be1dcd7e0b207e91)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:49.718216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Sep 27 10:18:49.754052 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 27 10:18:49.755168 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:49.756175 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2894/5769] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:49 2022] DELETE /identity/v3/domains/0791469814fa4909be1dcd7e0b207e91 => generated 120 bytes in 196 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:18:49.760442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:49.785618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fce31b-d444-45c8-8972-8b9ca0275253 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:49.803194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:49.804528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:49.804841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:49.805115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains/0791469814fa4909be1dcd7e0b207e91` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:49.814280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0791469814fa4909be1dcd7e0b207e91)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:49.816204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:49.854520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:49.871095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '0791469814fa4909be1dcd7e0b207e91'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:49.872249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '0791469814fa4909be1dcd7e0b207e91'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:49.947134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '0791469814fa4909be1dcd7e0b207e91'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:49.949261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '0791469814fa4909be1dcd7e0b207e91'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:49.954366 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2876/5770] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:49 2022] PATCH /identity/v3/domains/0791469814fa4909be1dcd7e0b207e91 => generated 277 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:49.958500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:49.960998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:49.989083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7964dc-7ad6-4390-8f11-6feb3b536edd tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:50.084323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:50.085352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:50.086134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:50.086586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains/0791469814fa4909be1dcd7e0b207e91` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:50.101849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0791469814fa4909be1dcd7e0b207e91)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:50.103685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:50.145427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0791469814fa4909be1dcd7e0b207e91'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.158771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0791469814fa4909be1dcd7e0b207e91'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.229972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0791469814fa4909be1dcd7e0b207e91', 'request_id': 'req-bf112b03-caa0-49ef-a932-e456f500daa3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.242268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0791469814fa4909be1dcd7e0b207e91', 'request_id': 'req-bf112b03-caa0-49ef-a932-e456f500daa3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.252280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:50.254397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0791469814fa4909be1dcd7e0b207e91', 'request_id': 'req-bf112b03-caa0-49ef-a932-e456f500daa3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.281536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] The token cache is being invalidate because project 0791469814fa4909be1dcd7e0b207e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:50.282148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.282947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.295520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0791469814fa4909be1dcd7e0b207e91', 'request_id': 'req-bf112b03-caa0-49ef-a932-e456f500daa3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.296778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0791469814fa4909be1dcd7e0b207e91', 'request_id': 'req-bf112b03-caa0-49ef-a932-e456f500daa3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.301816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0791469814fa4909be1dcd7e0b207e91', 'request_id': 'req-bf112b03-caa0-49ef-a932-e456f500daa3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.309794 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2895/5771] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:49 2022] DELETE /identity/v3/domains/0791469814fa4909be1dcd7e0b207e91 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:50.314922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:50.331622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2c8f4-185a-46f5-9017-602f3c1ffbc7 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:50.412562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:50.412987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:50.413311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:50.414245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains/717f4a4c735d4c63b06eea0bd974b7a0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:50.429853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Could not find domain: 717f4a4c735d4c63b06eea0bd974b7a0.: keystone.exception.DomainNotFound: Could not find domain: 717f4a4c735d4c63b06eea0bd974b7a0. Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 717f4a4c735d4c63b06eea0bd974b7a0. Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:50.439913 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:50.441101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:18:50.442101 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 717f4a4c735d4c63b06eea0bd974b7a0. Sep 27 10:18:50.443315 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.443315 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2877/5772] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:50 2022] DELETE /identity/v3/domains/717f4a4c735d4c63b06eea0bd974b7a0 => generated 112 bytes in 128 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:18:50.447921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:50.450509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:50.479915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf112b03-caa0-49ef-a932-e456f500daa3 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:50.485201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:50.485756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:50.486029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:50.486297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:50.486954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:50.489085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:50.524891 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2896/5773] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:50 2022] POST /identity/v3/domains => generated 253 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:50.529171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:50.545880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5608-a330-4c8c-87f7-b225c254fcb9 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:50.656184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:50.657085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:50.657421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:50.657653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:50.658390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:50.660971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:50.669588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.sql.core [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1796416092' for key 'project.ixu_project_name_domain_id'") Sep 27 10:18:50.669588 ncdv-master-1 devstack@keystone.service[104131]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 10:18:50.669588 ncdv-master-1 devstack@keystone.service[104131]: [parameters: {'id': '5a6d6fa6fedc48f1802b6dbf25d71fea', 'name': 'tempest-domain-dup-1796416092', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 27 10:18:50.669588 ncdv-master-1 devstack@keystone.service[104131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=104131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1796416092.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1796416092. Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application conn.query(q) Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 10:18:50.681770 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result.read() Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-1796416092' for key 'project.ixu_project_name_domain_id'") Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 27 10:18:50.682816 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application yield resource Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise self.value Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application session.commit() Sep 27 10:18:50.684012 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.flush() Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application compat.raise_( Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 10:18:50.685093 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 10:18:50.686147 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application conn.query(q) Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result.read() Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 10:18:50.688812 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1796416092' for key 'project.ixu_project_name_domain_id'") Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [parameters: {'id': '5a6d6fa6fedc48f1802b6dbf25d71fea', 'name': 'tempest-domain-dup-1796416092', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.691447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-1796416092. Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:50.693843 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1796416092. Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:18:50.696803 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2878/5774] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:18:50 2022] POST /identity/v3/domains => generated 212 bytes in 155 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 27 10:18:50.700511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:50.700511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fba5a4-1069-41ea-bc75-b7831a579161 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:50.705479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:50.706128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:50.706771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:50.706771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains/caa385df084e419c9e71f2bdcc7f04f0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:50.722908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=caa385df084e419c9e71f2bdcc7f04f0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:50.726363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:50.774061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.774875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.828055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.829038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:50.832996 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2897/5775] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:50 2022] PATCH /identity/v3/domains/caa385df084e419c9e71f2bdcc7f04f0 => generated 254 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:50.837880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:50.841923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:50.864130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71c6a4-fb1c-4435-be66-1e6db483f381 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:51.002587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:51.003434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:51.003830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:51.004205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains/caa385df084e419c9e71f2bdcc7f04f0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:51.029491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=caa385df084e419c9e71f2bdcc7f04f0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:51.035285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:51.044855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:51.090355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.095149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.171074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0', 'request_id': 'req-fc8e4199-c426-41ef-a080-ef3da4aa8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.185633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0', 'request_id': 'req-fc8e4199-c426-41ef-a080-ef3da4aa8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.207282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0', 'request_id': 'req-fc8e4199-c426-41ef-a080-ef3da4aa8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.224536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] The token cache is being invalidate because project caa385df084e419c9e71f2bdcc7f04f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:51.224900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.226242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.237273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0', 'request_id': 'req-fc8e4199-c426-41ef-a080-ef3da4aa8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.237990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0', 'request_id': 'req-fc8e4199-c426-41ef-a080-ef3da4aa8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.244467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'caa385df084e419c9e71f2bdcc7f04f0', 'request_id': 'req-fc8e4199-c426-41ef-a080-ef3da4aa8f49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.276131 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2879/5776] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:18:50 2022] DELETE /identity/v3/domains/caa385df084e419c9e71f2bdcc7f04f0 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:51.284818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:51.298387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31134d6-8e0e-4b0c-9cfd-0e66fcdb6d63 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:51.394800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30ad08-c798-4fc4-9c3e-9cf0470a6a7e tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f291437a37d4268b5f90ae7934b4c97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4d35dcbb5c84c54aa345c7961d020cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:51.395852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30ad08-c798-4fc4-9c3e-9cf0470a6a7e tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:51.396402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30ad08-c798-4fc4-9c3e-9cf0470a6a7e tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:51.396888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30ad08-c798-4fc4-9c3e-9cf0470a6a7e tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] PATH_INFO: `/v3/domains/0791469814fa4909be1dcd7e0b207e91` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:51.418380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30ad08-c798-4fc4-9c3e-9cf0470a6a7e tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-bf30ad08-c798-4fc4-9c3e-9cf0470a6a7e tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Could not find domain: 0791469814fa4909be1dcd7e0b207e91.: keystone.exception.DomainNotFound: Could not find domain: 0791469814fa4909be1dcd7e0b207e91. Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0791469814fa4909be1dcd7e0b207e91. Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:18:51.423811 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:18:51.425060 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:18:51.426083 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 0791469814fa4909be1dcd7e0b207e91. Sep 27 10:18:51.427278 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:18:51.432405 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2898/5777] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:18:51 2022] PATCH /identity/v3/domains/0791469814fa4909be1dcd7e0b207e91 => generated 112 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:18:51.436881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:51.437986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:51.536001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e4199-c426-41ef-a080-ef3da4aa8f49 tempest-DomainsNegativeTestJSON-784778966 tempest-DomainsNegativeTestJSON-784778966-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:51.544357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:51.545224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:51.545657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:51.546023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] PATH_INFO: `/v3/users/d4a85d8c08dc4becaa84b89cfdb986c0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:51.548228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a85d8c08dc4becaa84b89cfdb986c0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:51.549858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:51.682286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a85d8c08dc4becaa84b89cfdb986c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.684376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a85d8c08dc4becaa84b89cfdb986c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.688701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:51.706236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a85d8c08dc4becaa84b89cfdb986c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.728648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a85d8c08dc4becaa84b89cfdb986c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.751308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a85d8c08dc4becaa84b89cfdb986c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:51.758988 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2880/5778] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:51 2022] DELETE /identity/v3/users/d4a85d8c08dc4becaa84b89cfdb986c0 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:51.763608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:51.764065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:51.764393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:51.790443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:51.792511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:51.892758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2899/5779] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:51.945658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:51.948235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:52.068535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d349f6-9979-4e00-b55a-4a5b9c1c637a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:52.076032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:52.079147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:52.080242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:52.080752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:52.096630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:52.099055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:52.116150 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2881/5780] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:18:54.603389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:54.606447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:54.622903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900f4a8-eed2-4280-b4b1-f89101dba9eb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:54.639234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:54.640031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:54.640328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:54.640697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] PATH_INFO: `/v3/projects/4e62544719584ef39a24e7969ecacd7c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:54.646974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e62544719584ef39a24e7969ecacd7c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:54.649247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:54.651284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:54.761296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e62544719584ef39a24e7969ecacd7c', 'request_id': 'req-83c32980-99d7-4c3f-be5c-080c95026029'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:54.778853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e62544719584ef39a24e7969ecacd7c', 'request_id': 'req-83c32980-99d7-4c3f-be5c-080c95026029'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:54.790504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e62544719584ef39a24e7969ecacd7c', 'request_id': 'req-83c32980-99d7-4c3f-be5c-080c95026029'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:54.814678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] The token cache is being invalidate because project 4e62544719584ef39a24e7969ecacd7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:54.815231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:54.816155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:54.818331 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2900/5781] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:54 2022] DELETE /identity/v3/projects/4e62544719584ef39a24e7969ecacd7c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:54.822862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:54.836732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa527df1-7177-4657-8a55-e066e133e285 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:54.972521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:54.973974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:54.974474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:54.975855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] PATH_INFO: `/v3/users/3f291437a37d4268b5f90ae7934b4c97` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:54.978260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f291437a37d4268b5f90ae7934b4c97)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:54.981268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:55.089131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f291437a37d4268b5f90ae7934b4c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:55.089665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f291437a37d4268b5f90ae7934b4c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:55.105397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f291437a37d4268b5f90ae7934b4c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:55.123507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f291437a37d4268b5f90ae7934b4c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:55.134598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:55.150810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1bd7194e-6cd7-4c6a-aca7-a7077216ac98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f291437a37d4268b5f90ae7934b4c97'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:55.160520 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2882/5782] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:18:54 2022] DELETE /identity/v3/users/3f291437a37d4268b5f90ae7934b4c97 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:57.592574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:57.594714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:57.622818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c32980-99d7-4c3f-be5c-080c95026029 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:57.742883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:57.743877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:57.744320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:57.745282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] PATH_INFO: `/v3/projects/c4d35dcbb5c84c54aa345c7961d020cf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:57.748042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d35dcbb5c84c54aa345c7961d020cf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:57.750580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:57.842768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d35dcbb5c84c54aa345c7961d020cf', 'request_id': 'req-18866382-c3bb-4025-a49f-31ea59ae1aa6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:57.859965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d35dcbb5c84c54aa345c7961d020cf', 'request_id': 'req-18866382-c3bb-4025-a49f-31ea59ae1aa6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:57.869911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d35dcbb5c84c54aa345c7961d020cf', 'request_id': 'req-18866382-c3bb-4025-a49f-31ea59ae1aa6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:57.895782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] The token cache is being invalidate because project c4d35dcbb5c84c54aa345c7961d020cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:18:57.896376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:57.897151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18866382-c3bb-4025-a49f-31ea59ae1aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:18:57.903795 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2901/5783] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:18:57 2022] DELETE /identity/v3/projects/c4d35dcbb5c84c54aa345c7961d020cf => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:57.905689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d1c0-f25a-4617-8827-a6f4a8955402 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:57.906773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d1c0-f25a-4617-8827-a6f4a8955402 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:57.907238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d1c0-f25a-4617-8827-a6f4a8955402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:57.933376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8bf6d1c0-f25a-4617-8827-a6f4a8955402 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:57.934856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8bf6d1c0-f25a-4617-8827-a6f4a8955402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:58.055644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2883/5784] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:58.066766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00bf01-4f73-40ee-a1a7-506eccdbef33 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.069845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00bf01-4f73-40ee-a1a7-506eccdbef33 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.070428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00bf01-4f73-40ee-a1a7-506eccdbef33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.148600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9b00bf01-4f73-40ee-a1a7-506eccdbef33 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:58.154447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9b00bf01-4f73-40ee-a1a7-506eccdbef33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:58.178143 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2902/5785] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:58.181989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d1c0-f25a-4617-8827-a6f4a8955402 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:58.194154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d1c0-f25a-4617-8827-a6f4a8955402 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:58.199662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:58.200526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.201002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.201422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.202265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:18:58.203076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:58.205025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:58.221526 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2884/5786] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:18:58.225837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00bf01-4f73-40ee-a1a7-506eccdbef33 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:58.239076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00bf01-4f73-40ee-a1a7-506eccdbef33 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:58.255400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:58.256113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.256549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.256830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.257485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:58.259502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:58.284374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:58.326987 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2903/5787] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:58 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:18:58.331373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:58.345747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058df662-8db1-4ae9-8d64-ae37c91aeb53 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:58.446621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36469d4f-6e12-464f-96a2-3530bc0b820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:58.447797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469d4f-6e12-464f-96a2-3530bc0b820b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.448073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469d4f-6e12-464f-96a2-3530bc0b820b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.448343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36469d4f-6e12-464f-96a2-3530bc0b820b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.449024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36469d4f-6e12-464f-96a2-3530bc0b820b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:58.451133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36469d4f-6e12-464f-96a2-3530bc0b820b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:58.483416 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2885/5788] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:58 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:58.487426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:58.501230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e884b5-8bf1-4033-abb1-73ef2c47c6ab admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:58.516587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.516587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.517570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:58.518726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.519555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.523044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.523543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.525220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:58.531437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:58.556254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] MFA Rules not processed for user `3d0ab16486364c59909350229b72f376`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:58.557813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:58.568208 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2904/5789] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:18:58.569515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:58.597386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051a021-b10c-491c-837c-fd7dbe0debb2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:58.636052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:58.637971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.640023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.640858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.643191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:58.649548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:58.686382 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2905/5790] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:58.692882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:58.715664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:58.727849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32879fc-c39c-4ec0-bd0d-5937ed8b83b4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:58.744996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d6559-e70a-4816-9745-50108dbafe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:58.748021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6559-e70a-4816-9745-50108dbafe07 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.748702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6559-e70a-4816-9745-50108dbafe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.749458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6559-e70a-4816-9745-50108dbafe07 admin admin] PATH_INFO: `/v3/projects/cd97d8be1e044827a4f324bb53573556/users/4c732ef43af54e22a9779405cc464c96/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.763986 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2886/5791] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:18:58.774722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:58.776165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:58.813694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d6559-e70a-4816-9745-50108dbafe07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd97d8be1e044827a4f324bb53573556, user_id=4c732ef43af54e22a9779405cc464c96, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:58.818264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d6559-e70a-4816-9745-50108dbafe07 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:58.871793 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2906/5792] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:58 2022] PUT /identity/v3/projects/cd97d8be1e044827a4f324bb53573556/users/4c732ef43af54e22a9779405cc464c96/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:58.885870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5445090-1eca-49bd-8059-cd18063f09ca None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:58.886325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5445090-1eca-49bd-8059-cd18063f09ca None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:58.886684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5445090-1eca-49bd-8059-cd18063f09ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:58.925172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429eea90-a474-44dd-86fc-8d11824d341b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:58.990849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c5445090-1eca-49bd-8059-cd18063f09ca None None] MFA Rules not processed for user `4c732ef43af54e22a9779405cc464c96`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:18:58.992801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c5445090-1eca-49bd-8059-cd18063f09ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:59.139788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:59.140444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:59.144689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:59.144689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:59.221629 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2907/5793] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:18:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:18:59.225409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5445090-1eca-49bd-8059-cd18063f09ca None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:59.270258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:59.272719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:59.295703 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2887/5794] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:18:59.365587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5445090-1eca-49bd-8059-cd18063f09ca None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:59.374288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:59.374933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:59.375206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:59.375520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:59.376455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:59.379222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:59.443889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2908/5795] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:18:59 2022] POST /identity/v3/projects => generated 371 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:59.445481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:59.467310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9644f-04bb-4270-9b4e-32ae9667ba33 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:59.504648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:59.592417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:59.593177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:59.593528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:59.593886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:59.594601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:59.596628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:59.658009 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2888/5796] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:18:59 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:18:59.659671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:59.675285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde40f-9fce-409f-934a-6ed80188ae20 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:59.689488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:59.690212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:59.690407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:59.690659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:59.691317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:59.694159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:59.720776 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2909/5797] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:18:59.725491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:59.736233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3aa13-0a6f-4ea5-9512-9180d03bffcc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:59.753556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:59.754216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:59.754557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:59.754835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] PATH_INFO: `/v3/projects/391ed4e12a904142a28f004c7b0d093a/users/2842c986cbeb4fc988fd2b3491002667/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:59.797237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391ed4e12a904142a28f004c7b0d093a, user_id=2842c986cbeb4fc988fd2b3491002667, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:59.800064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:18:59.823045 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2889/5798] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:18:59 2022] PUT /identity/v3/projects/391ed4e12a904142a28f004c7b0d093a/users/2842c986cbeb4fc988fd2b3491002667/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:18:59.824328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:59.840295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044eed5b-6855-462f-b231-4e057fa87a24 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:18:59.858133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:59.937154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:18:59.939183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:18:59.971723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:18:59.972317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:18:59.972600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:18:59.972896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:18:59.977426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:18:59.979768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:00.000298 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2910/5799] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:00.025096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:00.066459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72fb1d-691f-4440-b9be-021f8b6a676d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:00.079974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af7911-3855-419b-adf4-547deb607ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:00.080968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af7911-3855-419b-adf4-547deb607ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:00.081412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af7911-3855-419b-adf4-547deb607ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:00.081844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af7911-3855-419b-adf4-547deb607ddd admin admin] PATH_INFO: `/v3/projects/391ed4e12a904142a28f004c7b0d093a/users/2842c986cbeb4fc988fd2b3491002667/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:00.086145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af7911-3855-419b-adf4-547deb607ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=391ed4e12a904142a28f004c7b0d093a, user_id=2842c986cbeb4fc988fd2b3491002667, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:00.091964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af7911-3855-419b-adf4-547deb607ddd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:00.099781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3c355-9285-46dd-ad49-d19315340615 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:00.126895 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2911/5800] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:00 2022] PUT /identity/v3/projects/391ed4e12a904142a28f004c7b0d093a/users/2842c986cbeb4fc988fd2b3491002667/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:00.137762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d90606-8bb9-4c07-870f-6bcc649ff06f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:00.139735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d90606-8bb9-4c07-870f-6bcc649ff06f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:00.139735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d90606-8bb9-4c07-870f-6bcc649ff06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:00.213266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a1d90606-8bb9-4c07-870f-6bcc649ff06f None None] MFA Rules not processed for user `2842c986cbeb4fc988fd2b3491002667`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:00.215284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a1d90606-8bb9-4c07-870f-6bcc649ff06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:00.238192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:00.239287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:00.239692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:00.240367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:00.243147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:00.259048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:00.268518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:00.292247 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2890/5801] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:18:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:00.432761 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2912/5802] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:19:00.439396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:00.456238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf06cf5-0fe2-4f97-a126-e90f40f6e971 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:00.471514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:00.472175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:00.472537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:00.472825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:00.473613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:00.477410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:00.515549 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2891/5803] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:00 2022] POST /identity/v3/services => generated 289 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:00.521572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d90606-8bb9-4c07-870f-6bcc649ff06f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:00.537329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d90606-8bb9-4c07-870f-6bcc649ff06f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:00.612610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:00.615098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:00.615098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:00.618039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:00.618820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:00.619921 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:00.619921 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:19:00.619921 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:00.626904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:00.705164 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-673812856. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:00.705164 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:00.705164 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:00.798315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2913/5804] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:19:00 2022] POST /identity/v3/endpoints => generated 358 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:19:00.802718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:00.820809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905c464-3834-4886-96d1-53024009b55b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:00.845039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:00.875951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:00.876550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:00.876902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:00.877454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-673812856` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:00.878065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-673812856)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:00.879154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:00.890836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:00.901989 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2892/5805] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 10:19:00 2022] GET /identity/v3/regions/tempest-region-673812856 => generated 178 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:00.916337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ffa81-4e5b-4863-86f3-36f7ac37c75c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:00.936586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:00.937254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:00.937530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:00.937798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:00.938460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:00.939548 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:00.939548 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:19:00.939548 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:00.940453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.048824 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2914/5806] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:00 2022] POST /identity/v3/services => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:01.052970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.069869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b43b90-ee5c-40e7-89c6-7b8cc8a57313 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.145671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.146568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.146857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.147145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.147846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.149657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.228889 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1947604521. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:01.228889 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:01.228889 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:01.287743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2893/5807] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:19:01 2022] POST /identity/v3/endpoints => generated 363 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:01.290008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.308472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67ffd9-3c92-457a-b144-0a928cc54a51 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.341103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.341719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.342177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.342769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-1947604521` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.343668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1947604521)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.345333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.350473 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2915/5808] 10.222.0.26 () {58 vars in 1206 bytes} [Tue Sep 27 10:19:01 2022] GET /identity/v3/regions/tempest-region-1947604521 => generated 180 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:01.355485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.380358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d12a-1e07-4c26-ae51-1967eb88dbb1 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.395007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.399415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.401449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.401894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.405986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.407652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.464607 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-245905060. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:01.464607 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:01.464607 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:01.532914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2894/5809] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:19:01 2022] POST /identity/v3/endpoints => generated 359 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:19:01.537593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.545910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:01.556983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66085e6-e382-4ff0-bfce-494ec367597b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.599584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.600241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.601556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.601556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-245905060` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.602020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-245905060)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.603117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.621347 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2916/5810] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 10:19:01 2022] GET /identity/v3/regions/tempest-region-245905060 => generated 178 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:01.625374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.641586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71065a-1e9c-4dd0-bd93-a0b6e991a36c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.655947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.656639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.656971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.657244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.657887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.658905 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:01.658905 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:19:01.658905 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:01.659863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.686503 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2895/5811] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 10:19:01 2022] GET /identity/v3/endpoints => generated 4394 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:01.690456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.706426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199431c-a7d5-4eff-9970-5ab77d868160 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.714564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.715544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.716019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.716535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints/6210f58ed128416b9e21d59707fcd582` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.725884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=6210f58ed128416b9e21d59707fcd582)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.727235 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:01.727235 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:19:01.727235 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:01.728593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.733358 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2917/5812] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:01 2022] GET /identity/v3/endpoints/6210f58ed128416b9e21d59707fcd582 => generated 359 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:01.737359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.753727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3069ef-1dae-4975-942c-0327ff034bd7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.761955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.764173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.764951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.765732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints/6210f58ed128416b9e21d59707fcd582` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.767309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=6210f58ed128416b9e21d59707fcd582)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.771136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.784893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '6210f58ed128416b9e21d59707fcd582', 'request_id': 'req-53f63f3c-7169-4407-a0e2-8233eb6de386'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:01.801591 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2896/5813] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:01 2022] DELETE /identity/v3/endpoints/6210f58ed128416b9e21d59707fcd582 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:01.805915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.819740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495919-e3c3-463b-b47a-becb323cc24d tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.849902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.852246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.852999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.854070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.855386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:01.856886 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:01.856886 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:19:01.856886 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:01.858324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:01.871878 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2918/5814] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 10:19:01 2022] GET /identity/v3/endpoints => generated 4048 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:01.876424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:01.894541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f63f3c-7169-4407-a0e2-8233eb6de386 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:01.918243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:01.919191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:01.919636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:01.920097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints/6210f58ed128416b9e21d59707fcd582` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:01.935423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Could not find endpoint: 6210f58ed128416b9e21d59707fcd582.: keystone.exception.EndpointNotFound: Could not find endpoint: 6210f58ed128416b9e21d59707fcd582. Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._iter().one() Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._only_one_row( Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:19:01.976895 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 6210f58ed128416b9e21d59707fcd582. Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:19:01.978266 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:19:01.979600 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 6210f58ed128416b9e21d59707fcd582. Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:19:01.987470 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2897/5815] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:01 2022] DELETE /identity/v3/endpoints/6210f58ed128416b9e21d59707fcd582 => generated 114 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:19:01.988701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.004739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c837344-9f0d-4068-bb91-0745fcaab8a8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.019345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.020023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.020304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.020577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-245905060` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.021265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-245905060)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.022979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.062244 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-245905060. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:02.062244 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:02.062244 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:02.072326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2919/5816] 10.222.0.26 () {58 vars in 1206 bytes} [Tue Sep 27 10:19:01 2022] DELETE /identity/v3/regions/tempest-region-245905060 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:02.076898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.103359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e1ae9-0f15-4b4a-87c3-5f3c20ea63bd tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.146979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.147605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.147893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.148629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.149378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.151684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.174313 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2898/5817] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 10:19:02 2022] GET /identity/v3/endpoints => generated 4048 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:02.175749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.195145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65da9e5-c95f-4815-9cf9-251034b3b78f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.205212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.205878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.206205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.206479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.207146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Adding query filter params (service_id=3116dfa6af66482fb2224b6129963fd0) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:02.207453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.215034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.230454 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2920/5818] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:19:02 2022] GET /identity/v3/endpoints?service_id=3116dfa6af66482fb2224b6129963fd0 => generated 500 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:02.231643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.250800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fd940-0558-4363-91b9-a1827fd95963 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.258976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.259663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.259956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.260232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.260931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Adding query filter params (service_id=a9e6e8cb6b9045b18f4b81b532cba892) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:02.261241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.270280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.277845 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2899/5819] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:19:02 2022] GET /identity/v3/endpoints?service_id=a9e6e8cb6b9045b18f4b81b532cba892 => generated 505 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:02.282412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.301407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905e4cd-5d6b-47ff-9f82-c9308eda8ec6 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.314449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.315023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.315327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.315598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.316258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:02.316563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.318304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.336217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2921/5820] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 10:19:02 2022] GET /identity/v3/endpoints?interface=public => generated 3403 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:02.340471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.363225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fbc10-4ca3-46c0-8bae-625a55f0192f tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.372630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.376192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.376957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.378218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.379143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:02.380751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.385898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.393805 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2900/5821] 10.222.0.26 () {58 vars in 1171 bytes} [Tue Sep 27 10:19:02 2022] GET /identity/v3/endpoints?interface=internal => generated 480 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:02.399665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.415316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955cc9-85c1-421e-a99e-fd438eee7d3b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.423628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.424321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.424606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.425019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.429893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.431702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.475797 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2922/5822] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:02 2022] POST /identity/v3/services => generated 288 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:02.477732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.496344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63bc06-58fb-439a-8a8c-e1c11c716149 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.529185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.529986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.530343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.530691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.531436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.534034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.563953 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1495168450. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:02.563953 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:02.563953 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:02.629083 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2901/5823] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:19:02 2022] POST /identity/v3/endpoints => generated 361 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:02.630838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.645182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9e5ae-6bdf-49c8-829a-751aa5a4a5d4 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.683691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.684338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.684612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.684935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-1495168450` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.685519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1495168450)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.687218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.691858 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2923/5824] 10.222.0.26 () {58 vars in 1206 bytes} [Tue Sep 27 10:19:02 2022] GET /identity/v3/regions/tempest-region-1495168450 => generated 180 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:02.694754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.719186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e81da6-8ebf-4758-91f0-6e8eefca099c tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.732067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.733189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.734228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.734228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints/083857fec25743c9853ff81693a28fa0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.741346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=083857fec25743c9853ff81693a28fa0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.742741 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:02.742741 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:19:02.742741 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:02.744506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.750370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:02.775121 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-98717611. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:02.775121 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:02.775121 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:02.833044 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2902/5825] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:19:02 2022] PATCH /identity/v3/endpoints/083857fec25743c9853ff81693a28fa0 => generated 359 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:19:02.834770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.848023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ead9c-1b67-4b24-8131-769b59a65391 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.883273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.884513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.885048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.885590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-98717611` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.886438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-98717611)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.887814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.890427 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2924/5826] 10.222.0.26 () {58 vars in 1200 bytes} [Tue Sep 27 10:19:02 2022] GET /identity/v3/regions/tempest-region-98717611 => generated 176 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:02.894313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:02.909033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fd041-6b13-4ca1-9d9b-51e7214f59f9 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:02.917961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:02.919128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:02.919679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:02.920158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints/083857fec25743c9853ff81693a28fa0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:02.927298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=083857fec25743c9853ff81693a28fa0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:02.930315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:02.955524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '083857fec25743c9853ff81693a28fa0', 'request_id': 'req-516cef12-b2a4-480a-a351-4370c635a5bf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:02.985909 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2903/5827] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:02 2022] DELETE /identity/v3/endpoints/083857fec25743c9853ff81693a28fa0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:02.991468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.036819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839600-2129-4309-bdde-cb4f0c56b473 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.109604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.110569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.111346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.111840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-98717611` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.112857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-98717611)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.117164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.142120 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-98717611. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:03.142120 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:03.142120 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:03.153653 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2925/5828] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 10:19:02 2022] DELETE /identity/v3/regions/tempest-region-98717611 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:03.158037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.174403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cef12-b2a4-480a-a351-4370c635a5bf tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.202379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.203542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.203926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.204303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-1495168450` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.205033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1495168450)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.206549 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:03.206549 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:19:03.206549 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:03.207701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.256591 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2904/5829] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/regions/tempest-region-1495168450 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:03.262311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.276677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e17f4-5cf3-4899-adb8-8cf085fa1d1b tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.315902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.316888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.317334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.317862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/services/d2be47cab2fc4c7b8b2d2d83c6fae8d6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.319302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_service(service_id=d2be47cab2fc4c7b8b2d2d83c6fae8d6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.321225 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:03.321225 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:19:03.321225 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:03.322708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.371263 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2926/5830] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/services/d2be47cab2fc4c7b8b2d2d83c6fae8d6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:03.375652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.390245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aced93-f75b-4c63-bf5d-f6b44be4ecc8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.424149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.424849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.425296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.425576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints/953a93f3fea74c5681aa5f375d278d5c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.436003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=953a93f3fea74c5681aa5f375d278d5c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.437746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.456629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '953a93f3fea74c5681aa5f375d278d5c', 'request_id': 'req-a493afe9-1934-48e1-a73d-7be222d5ebfa'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:03.478070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2905/5831] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/endpoints/953a93f3fea74c5681aa5f375d278d5c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:03.483034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.498882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178889a-275f-4cce-85d2-51559fcdd4f7 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.528864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.529680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.530094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.530464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-1947604521` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.531332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1947604521)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.533525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.569965 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1947604521. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:03.569965 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:03.569965 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:03.585144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2927/5832] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/regions/tempest-region-1947604521 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:03.586124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.607043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493afe9-1934-48e1-a73d-7be222d5ebfa tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.676115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.677476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.677919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.678358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/services/a9e6e8cb6b9045b18f4b81b532cba892` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.679289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_service(service_id=a9e6e8cb6b9045b18f4b81b532cba892)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.681193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.735434 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2906/5833] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/services/a9e6e8cb6b9045b18f4b81b532cba892 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:03.739367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.752319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de557ef-35ef-4821-b287-a62e123978eb tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.768451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:03.795598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.796427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.796754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.797055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/endpoints/45d4f39b3a104301a67b915df1b0fef5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.816607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=45d4f39b3a104301a67b915df1b0fef5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.819532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.843456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '45d4f39b3a104301a67b915df1b0fef5', 'request_id': 'req-4772f5e4-8370-4f7f-90db-b306744f5785'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:03.861864 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2928/5834] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/endpoints/45d4f39b3a104301a67b915df1b0fef5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:03.863171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:03.886159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d92a0b-67ec-4bc4-a90b-85753bb20af8 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:03.919028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:03.919677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:03.919986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:03.920261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/regions/tempest-region-673812856` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:03.921366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-673812856)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:03.923756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:03.974509 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-673812856. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:19:03.974509 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:19:03.974509 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:19:03.995317 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2907/5835] 10.222.0.26 () {58 vars in 1206 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/regions/tempest-region-673812856 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:03.999826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:04.014486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772f5e4-8370-4f7f-90db-b306744f5785 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:04.034619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033675b-982a-40d5-9cd4-f7563908b701 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2842c986cbeb4fc988fd2b3491002667', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '391ed4e12a904142a28f004c7b0d093a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:04.035267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033675b-982a-40d5-9cd4-f7563908b701 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:04.035542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033675b-982a-40d5-9cd4-f7563908b701 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:04.035813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033675b-982a-40d5-9cd4-f7563908b701 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] PATH_INFO: `/v3/services/3116dfa6af66482fb2224b6129963fd0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:04.036393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033675b-982a-40d5-9cd4-f7563908b701 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorizing `identity:delete_service(service_id=3116dfa6af66482fb2224b6129963fd0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:04.038104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033675b-982a-40d5-9cd4-f7563908b701 tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:04.074642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:04.075766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2929/5836] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:03 2022] DELETE /identity/v3/services/3116dfa6af66482fb2224b6129963fd0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:04.140523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:04.212174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff600bb6-3114-438d-abce-34a570984a3e tempest-EndPointsTestJSON-1465203946 tempest-EndPointsTestJSON-1465203946-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:04.230598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:04.231556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:04.231985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:04.232413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] PATH_INFO: `/v3/users/4c732ef43af54e22a9779405cc464c96` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:04.233813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c732ef43af54e22a9779405cc464c96)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:04.235899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:04.325375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c732ef43af54e22a9779405cc464c96'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:04.327379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c732ef43af54e22a9779405cc464c96'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:04.339570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c732ef43af54e22a9779405cc464c96'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:04.361967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c732ef43af54e22a9779405cc464c96'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:04.386483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c732ef43af54e22a9779405cc464c96'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:04.397573 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2908/5837] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:04 2022] DELETE /identity/v3/users/4c732ef43af54e22a9779405cc464c96 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:04.402081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:04.404316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:04.404316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:04.459773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:04.461114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:04.558287 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2930/5838] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:04.667209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:04.669075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:04.791139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba851d1-462f-4751-bc0f-471677c55ef7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:04.795758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:04.796507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:04.796890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:04.797236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:04.811773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:04.813273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:04.825112 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2909/5839] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:06.739144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:06.741626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:06.758561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614c2ca-9d7b-466c-8634-70a1f8a5323c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:06.773429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:06.774833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:06.775348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:06.775829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] PATH_INFO: `/v3/projects/cd97d8be1e044827a4f324bb53573556` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:06.779166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd97d8be1e044827a4f324bb53573556)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:06.781495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:06.862921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd97d8be1e044827a4f324bb53573556', 'request_id': 'req-78f33c1c-5b79-4372-acc9-853bbc2a4c78'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:06.877838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd97d8be1e044827a4f324bb53573556', 'request_id': 'req-78f33c1c-5b79-4372-acc9-853bbc2a4c78'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:06.888950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd97d8be1e044827a4f324bb53573556', 'request_id': 'req-78f33c1c-5b79-4372-acc9-853bbc2a4c78'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:06.924132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] The token cache is being invalidate because project cd97d8be1e044827a4f324bb53573556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:06.924132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:06.924132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:06.924132 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2931/5840] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:06 2022] DELETE /identity/v3/projects/cd97d8be1e044827a4f324bb53573556 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:06.924498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:06.931376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364b086-54cb-4a7c-8d99-513ba28ea827 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:07.062407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:07.169748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:07.171314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:07.171809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:07.172292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] PATH_INFO: `/v3/users/2842c986cbeb4fc988fd2b3491002667` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:07.173903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2842c986cbeb4fc988fd2b3491002667)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:07.178387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:07.306376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2842c986cbeb4fc988fd2b3491002667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.307159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2842c986cbeb4fc988fd2b3491002667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.317296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2842c986cbeb4fc988fd2b3491002667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.337722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2842c986cbeb4fc988fd2b3491002667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.359548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-05e47249-12d8-46eb-98dd-2ff0543db212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2842c986cbeb4fc988fd2b3491002667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.378913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2910/5841] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:06 2022] DELETE /identity/v3/users/2842c986cbeb4fc988fd2b3491002667 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:07.512286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:07.514808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:07.531222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33c1c-5b79-4372-acc9-853bbc2a4c78 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:07.579891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:07.658277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:07.659679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:07.660136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:07.660790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] PATH_INFO: `/v3/users/1cd19a2d65be4865ba9b9223dde7242b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:07.663980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd19a2d65be4865ba9b9223dde7242b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:07.671185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:07.816328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd19a2d65be4865ba9b9223dde7242b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.817428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd19a2d65be4865ba9b9223dde7242b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.835701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd19a2d65be4865ba9b9223dde7242b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.851132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd19a2d65be4865ba9b9223dde7242b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.852224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd19a2d65be4865ba9b9223dde7242b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:07.895105 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2932/5842] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:07 2022] DELETE /identity/v3/users/1cd19a2d65be4865ba9b9223dde7242b => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:07.896108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:07.897376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:07.898265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:07.929340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:07.936330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:08.049863 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2911/5843] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:08.091796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:08.095873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:08.183265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7938f6-be48-45b7-87ad-3596d8739dbc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:08.187385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5b685-d1f5-40ac-aae3-e124bf229dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:08.188023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b685-d1f5-40ac-aae3-e124bf229dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:08.188305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b685-d1f5-40ac-aae3-e124bf229dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:08.188575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b685-d1f5-40ac-aae3-e124bf229dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:08.198876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5b685-d1f5-40ac-aae3-e124bf229dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:08.200361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5b685-d1f5-40ac-aae3-e124bf229dc7 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:08.212812 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2933/5844] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:08.576566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:08.579570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:08.596842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0fe38-4356-4219-a789-b12ea26c6843 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:08.616790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:08.617788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:08.618302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:08.618825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] PATH_INFO: `/v3/projects/391ed4e12a904142a28f004c7b0d093a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:08.621070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=391ed4e12a904142a28f004c7b0d093a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:08.625417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:08.738694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:08.742361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391ed4e12a904142a28f004c7b0d093a', 'request_id': 'req-4abfd144-300d-479b-bd0a-40ad9607e597'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:08.753190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391ed4e12a904142a28f004c7b0d093a', 'request_id': 'req-4abfd144-300d-479b-bd0a-40ad9607e597'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:08.780451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391ed4e12a904142a28f004c7b0d093a', 'request_id': 'req-4abfd144-300d-479b-bd0a-40ad9607e597'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:08.811095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] The token cache is being invalidate because project 391ed4e12a904142a28f004c7b0d093a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:08.812310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:08.815287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4abfd144-300d-479b-bd0a-40ad9607e597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:08.823793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:08.824888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:08.825359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:08.833906 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2912/5845] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:08 2022] DELETE /identity/v3/projects/391ed4e12a904142a28f004c7b0d093a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:08.863952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:08.866536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:08.937827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8faa4-ab5d-457b-aeac-e4893d631415 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:08.938446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8faa4-ab5d-457b-aeac-e4893d631415 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:08.938732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8faa4-ab5d-457b-aeac-e4893d631415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:08.960778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:08.975305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d3a8faa4-ab5d-457b-aeac-e4893d631415 None None] MFA Rules not processed for user `1d92f7d2ee2d47beaa4f5d43ac40cd82`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:08.977427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3a8faa4-ab5d-457b-aeac-e4893d631415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:08.995687 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2934/5846] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:08.998355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:09.003481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:09.031476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0b741-3f60-4b89-90f1-a75321053add None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:09.055290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:09.055959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:09.056237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:09.056531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] PATH_INFO: `/v3/projects/57a094520e3b42abb0f48aff45534e11` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:09.057850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a094520e3b42abb0f48aff45534e11)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:09.059774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:09.228533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a094520e3b42abb0f48aff45534e11', 'request_id': 'req-7b3db8f3-44b0-4075-98fc-d930efe79450'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.236389 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2913/5847] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:09.248974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:09.254858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:09.255672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:09.270657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a094520e3b42abb0f48aff45534e11', 'request_id': 'req-7b3db8f3-44b0-4075-98fc-d930efe79450'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.282874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a094520e3b42abb0f48aff45534e11', 'request_id': 'req-7b3db8f3-44b0-4075-98fc-d930efe79450'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.325409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] The token cache is being invalidate because project 57a094520e3b42abb0f48aff45534e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:09.325853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.327137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.328381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:09.331853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:09.337093 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2935/5848] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:08 2022] DELETE /identity/v3/projects/57a094520e3b42abb0f48aff45534e11 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:09.337093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:09.341076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:09.516435 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2914/5849] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:09.517500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:09.518879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:09.533468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8561-c0ca-488d-8ff8-cfaf09267af1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:09.539076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:09.539738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:09.540013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:09.540810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] PATH_INFO: `/v3/users/c73883f8f5ae47fabfe0b41595d41765` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:09.543053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73883f8f5ae47fabfe0b41595d41765)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:09.545894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db8f3-44b0-4075-98fc-d930efe79450 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:09.547247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:09.557570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:09.558150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:09.558431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:09.558813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] PATH_INFO: `/v3/users/a7b8599d0b40424087e52d8e35b71ec4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:09.560231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b8599d0b40424087e52d8e35b71ec4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:09.562489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:09.654998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b8599d0b40424087e52d8e35b71ec4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.663542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b8599d0b40424087e52d8e35b71ec4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.678585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b8599d0b40424087e52d8e35b71ec4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.684772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73883f8f5ae47fabfe0b41595d41765'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.688153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73883f8f5ae47fabfe0b41595d41765'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.691137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b8599d0b40424087e52d8e35b71ec4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.696138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b8599d0b40424087e52d8e35b71ec4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.704848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73883f8f5ae47fabfe0b41595d41765'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.714899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:09.723557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73883f8f5ae47fabfe0b41595d41765'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.728406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:09.736628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2936/5850] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:09 2022] DELETE /identity/v3/users/a7b8599d0b40424087e52d8e35b71ec4 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:09.738653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:09.740951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:09.760801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73883f8f5ae47fabfe0b41595d41765'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:09.772710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2915/5851] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:09 2022] DELETE /identity/v3/users/c73883f8f5ae47fabfe0b41595d41765 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:09.924904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac1fd1-792d-4a89-a4d2-92f209d3a2d0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:10.068251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:10.069128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:10.069576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:10.070100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:10.070861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:10.071440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:10.073985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:10.092493 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2937/5852] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:10.097054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:10.098643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:10.114410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb580e-526d-4f1f-bfed-7d8d4182b24b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:10.121210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:10.122160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:10.122590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:10.123011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:10.123812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:10.126138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:10.174812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:10.185407 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2916/5853] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:10 2022] POST /identity/v3/projects => generated 387 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:10.189833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fda3e-d2d7-47b3-8544-3592e4cae13b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:10.328947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:10.329594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:10.331211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:10.331420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:10.332731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:10.335739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:10.380840 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2938/5854] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:10 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:10.384972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:10.398398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6be9b8-393c-4d23-8131-4ad8bb81bc87 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:10.408067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:10.409085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:10.409543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:10.410008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:10.410823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:10.413140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:10.432631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:10.436972 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2917/5855] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:10.447345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ebaf2-8adf-4328-b787-d4faef0989a3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:10.462158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2e824-0cb2-4963-ae49-178b5b474b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:10.463063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2e824-0cb2-4963-ae49-178b5b474b45 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:10.463532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2e824-0cb2-4963-ae49-178b5b474b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:10.463871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2e824-0cb2-4963-ae49-178b5b474b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:10.464759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2e824-0cb2-4963-ae49-178b5b474b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:10.466595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2e824-0cb2-4963-ae49-178b5b474b45 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:10.503363 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2939/5856] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:10.506717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:10.523681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40458a-37a1-4538-9192-6711f7599836 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:10.545930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:10.546633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:10.546998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:10.547167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] PATH_INFO: `/v3/projects/31c6e547947b48069245a6d13c8b6ace/users/4cadaa04d6c745dda814566463731906/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:10.598443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c6e547947b48069245a6d13c8b6ace, user_id=4cadaa04d6c745dda814566463731906, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:10.609597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:10.652490 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2918/5857] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:10 2022] PUT /identity/v3/projects/31c6e547947b48069245a6d13c8b6ace/users/4cadaa04d6c745dda814566463731906/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:10.654031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:10.654543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:10.655094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:10.702461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:10.725763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] MFA Rules not processed for user `4cadaa04d6c745dda814566463731906`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:10.727649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:10.942550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2940/5858] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:19:10.947278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:11.022492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:11.024244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:11.154386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83534026-d372-4f6f-a275-53261a5823d7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:11.165840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:11.167896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:11.168338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:11.168825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:11.169714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:11.173792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:11.174990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510f47-c6cd-415d-bc85-50e1707f39ba None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:11.187843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:11.188085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:11.188454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:11.188841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] PATH_INFO: `/v3/projects/1d229e5b879b481082284236c9f8c446` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:11.190613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d229e5b879b481082284236c9f8c446)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:11.192986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:11.254677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2919/5859] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:10 2022] POST /identity/v3/projects => generated 387 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:11.259668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:11.264439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:11.422719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d229e5b879b481082284236c9f8c446', 'request_id': 'req-1347f072-091a-4159-89d1-aa20e7651807'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.458034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d229e5b879b481082284236c9f8c446', 'request_id': 'req-1347f072-091a-4159-89d1-aa20e7651807'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.462084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afb02f-8269-4f18-9e1f-f4ec9b6928d7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:11.484456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d229e5b879b481082284236c9f8c446', 'request_id': 'req-1347f072-091a-4159-89d1-aa20e7651807'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.513874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:11.529184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] The token cache is being invalidate because project 1d229e5b879b481082284236c9f8c446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:11.530221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.532378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.537824 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2941/5860] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:11 2022] DELETE /identity/v3/projects/1d229e5b879b481082284236c9f8c446 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:11.538670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:11.541736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:11.677420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:11.678311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:11.678777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:11.679230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] PATH_INFO: `/v3/projects/4b28c4bc8e9c4db7bb88c32daf09eaab` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:11.680577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:11.683145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b28c4bc8e9c4db7bb88c32daf09eaab)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:11.685489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:11.781647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b28c4bc8e9c4db7bb88c32daf09eaab', 'request_id': 'req-188c487d-1875-4612-b3fe-3744b5e5c584'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.791915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b28c4bc8e9c4db7bb88c32daf09eaab', 'request_id': 'req-188c487d-1875-4612-b3fe-3744b5e5c584'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.810207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b28c4bc8e9c4db7bb88c32daf09eaab', 'request_id': 'req-188c487d-1875-4612-b3fe-3744b5e5c584'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.837529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] The token cache is being invalidate because project 4b28c4bc8e9c4db7bb88c32daf09eaab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:11.837780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.842374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-188c487d-1875-4612-b3fe-3744b5e5c584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:11.845152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f072-091a-4159-89d1-aa20e7651807 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:11.850235 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2920/5861] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:11 2022] DELETE /identity/v3/projects/4b28c4bc8e9c4db7bb88c32daf09eaab => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:11.852169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:11.852430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:11.854948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:11.928055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:11.929496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:12.064794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e513-cf13-4af5-832d-1b75376c3d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:12.065375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e513-cf13-4af5-832d-1b75376c3d83 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:12.068004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e513-cf13-4af5-832d-1b75376c3d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:12.068248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e513-cf13-4af5-832d-1b75376c3d83 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:12.070058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38e513-cf13-4af5-832d-1b75376c3d83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:12.077083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38e513-cf13-4af5-832d-1b75376c3d83 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:12.131384 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2921/5862] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:12.132173 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2942/5863] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:11 2022] POST /identity/v3/users => generated 300 bytes in 594 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:12.133200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:12.134952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:12.142600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c246e-9bdb-468b-9688-70024acef96d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:12.142912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c246e-9bdb-468b-9688-70024acef96d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:12.146367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c246e-9bdb-468b-9688-70024acef96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:12.154075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c02a21-2b5d-4c06-8e3f-f903455c26af None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:12.162275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:12.165284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:12.179156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:12.180188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] PATH_INFO: `/v3/users/60e697df0e9d438bb496f76e937aa422` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:12.182643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e697df0e9d438bb496f76e937aa422)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:12.185152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:12.226720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-871c246e-9bdb-468b-9688-70024acef96d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:12.232747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-871c246e-9bdb-468b-9688-70024acef96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:12.266636 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2943/5864] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:12.279670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c246e-9bdb-468b-9688-70024acef96d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:12.290356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-871c246e-9bdb-468b-9688-70024acef96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:12.311761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:12.327701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e697df0e9d438bb496f76e937aa422'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:12.332596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e697df0e9d438bb496f76e937aa422'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:12.344287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c246e-9bdb-468b-9688-70024acef96d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:12.370694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e697df0e9d438bb496f76e937aa422'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:12.372350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:12.374569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:12.377053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:12.378995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:12.381758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:12.386962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:12.404982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e697df0e9d438bb496f76e937aa422'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:12.437966 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2944/5865] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:12.444426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:12.453638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:12.456138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e697df0e9d438bb496f76e937aa422'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:12.480258 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2922/5866] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:12 2022] DELETE /identity/v3/users/60e697df0e9d438bb496f76e937aa422 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:12.481625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:12.484941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:12.494330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc7420-3172-4243-8304-ccb87c224e62 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:12.658285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:12.659107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:12.659384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:12.659777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:12.660703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:12.660996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:12.663532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:12.694984 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2945/5867] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:12.696196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:12.698513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3064d3-ef64-491a-aab8-7d7128a65610 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:12.706754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:12.707452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:12.707984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:12.708249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] PATH_INFO: `/v3/projects/ab83b763c59f4dada89c6b4620ffed0b/users/e8bb5a48317d4b04993d3edd6b299f3e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:12.722939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3837c6-2bc3-4741-8abf-14ca54bce6ae admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:12.728032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab83b763c59f4dada89c6b4620ffed0b, user_id=e8bb5a48317d4b04993d3edd6b299f3e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:12.730470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:12.730962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:12.731887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:12.732407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:12.732992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:12.734180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:12.736082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:12.755380 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2923/5868] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:12 2022] PUT /identity/v3/projects/ab83b763c59f4dada89c6b4620ffed0b/users/e8bb5a48317d4b04993d3edd6b299f3e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:12.759498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:12.783853 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2946/5869] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:12 2022] POST /identity/v3/projects => generated 359 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:12.784964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:12.868509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:12.894644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7ee46-5c58-4e89-bb39-9cae01c6131e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:12.988724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189da0-a366-41fa-a121-d7259e7accc5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.000625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.001276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.001550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.004186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.006769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.010834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.011278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:13.037760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.038384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.038662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.040206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.041594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.045503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.066593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2947/5870] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:12 2022] POST /identity/v3/users => generated 280 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:13.067900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.077563 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2924/5871] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:19:13.079293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.092287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede522b-eba2-4151-9a50-2e66044bff33 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.099742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b36eef-17f1-456b-95c5-be4d8e8bd6ff admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.102016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.102700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.102975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.103245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.103895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.106012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564537cc-d12d-4919-9efd-59d39f382f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.106804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564537cc-d12d-4919-9efd-59d39f382f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.107362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564537cc-d12d-4919-9efd-59d39f382f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.107786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564537cc-d12d-4919-9efd-59d39f382f9f admin admin] PATH_INFO: `/v3/projects/ab83b763c59f4dada89c6b4620ffed0b/users/e8bb5a48317d4b04993d3edd6b299f3e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.110404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564537cc-d12d-4919-9efd-59d39f382f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab83b763c59f4dada89c6b4620ffed0b, user_id=e8bb5a48317d4b04993d3edd6b299f3e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.113798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564537cc-d12d-4919-9efd-59d39f382f9f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.115104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.136326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2948/5872] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:13.137327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.149517 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2925/5873] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:13 2022] PUT /identity/v3/projects/ab83b763c59f4dada89c6b4620ffed0b/users/e8bb5a48317d4b04993d3edd6b299f3e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:13.151831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea350-75d8-4187-ada7-9a3255c00ad3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.152267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea350-75d8-4187-ada7-9a3255c00ad3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.152787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea350-75d8-4187-ada7-9a3255c00ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.164050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659ace5-fa13-4c57-8f87-fc00e3a54060 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.196417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3faea350-75d8-4187-ada7-9a3255c00ad3 None None] MFA Rules not processed for user `e8bb5a48317d4b04993d3edd6b299f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:13.197831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3faea350-75d8-4187-ada7-9a3255c00ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:13.274724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.275611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.275732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.275936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.276579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.278725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.310137 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2926/5874] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:13.313010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea350-75d8-4187-ada7-9a3255c00ad3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.315415 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2949/5875] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:13.316516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.337867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea350-75d8-4187-ada7-9a3255c00ad3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.340917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240211c5-6cf5-43a8-a7c5-5854502c1ca3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.343646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975e65b-5bb9-4760-ab65-736e6414f472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.344386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975e65b-5bb9-4760-ab65-736e6414f472 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.344950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975e65b-5bb9-4760-ab65-736e6414f472 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.345347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975e65b-5bb9-4760-ab65-736e6414f472 admin admin] PATH_INFO: `/v3/projects/73c1c382922145d9ab011b08870b9b16/users/9bb6583714f54aefa5ad1c19a4a26ec3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.350376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.351293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.351678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.352063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects/31c6e547947b48069245a6d13c8b6ace` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.353563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:get_project(project_id=31c6e547947b48069245a6d13c8b6ace)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.356348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.358994 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2950/5876] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/projects/31c6e547947b48069245a6d13c8b6ace => generated 387 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:13.364683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.380765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec09fc-5b0e-45fe-b6a5-a356f5deb84d tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.384812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975e65b-5bb9-4760-ab65-736e6414f472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c1c382922145d9ab011b08870b9b16, user_id=9bb6583714f54aefa5ad1c19a4a26ec3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.388491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975e65b-5bb9-4760-ab65-736e6414f472 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.391865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.394741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.396619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.397295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.398172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.402148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.429176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2927/5877] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:13 2022] PUT /identity/v3/projects/73c1c382922145d9ab011b08870b9b16/users/9bb6583714f54aefa5ad1c19a4a26ec3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:13.442626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79389bd9-5c67-406d-8b10-017501f803ea None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.443192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79389bd9-5c67-406d-8b10-017501f803ea None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.443876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79389bd9-5c67-406d-8b10-017501f803ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.464062 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2951/5878] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:19:13 2022] POST /identity/v3/projects => generated 339 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:13.469520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.489187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-79389bd9-5c67-406d-8b10-017501f803ea None None] MFA Rules not processed for user `9bb6583714f54aefa5ad1c19a4a26ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:13.491571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79389bd9-5c67-406d-8b10-017501f803ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:13.623575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7bfb5-a311-46bc-afcd-4d50b322ff02 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.639900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.641622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.642805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.643094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.644585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.658909 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:19:13.658909 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:19:13.658909 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:19:13.663224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.690024 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2928/5879] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:13.737044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79389bd9-5c67-406d-8b10-017501f803ea None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.768001 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2952/5880] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/projects => generated 6913 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:13.772506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.777855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:13.795061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:13.800307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9884f4-4cfa-41ec-976c-6f609e7095b6 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.807436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.808089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.808454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.809687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects/31c6e547947b48069245a6d13c8b6ace` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.810971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:get_project(project_id=31c6e547947b48069245a6d13c8b6ace)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.813083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.815495 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2953/5881] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/projects/31c6e547947b48069245a6d13c8b6ace => generated 387 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:19:13.819271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.843535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066475d-60dd-4335-ad6c-609815ec3332 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.850051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79389bd9-5c67-406d-8b10-017501f803ea None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.855383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.857673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.858049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.858545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] PATH_INFO: `/v3/users/9bb6583714f54aefa5ad1c19a4a26ec3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.859178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.859947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb6583714f54aefa5ad1c19a4a26ec3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.860178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.860604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.861043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects/20b453d19b6943aeb82aaeaaf6a558c7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.862907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:get_project(project_id=20b453d19b6943aeb82aaeaaf6a558c7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.863140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.866177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.875076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2954/5882] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/projects/20b453d19b6943aeb82aaeaaf6a558c7 => generated 339 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:13.881274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.906898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f176062-5128-4971-a1c2-2880dfb81c89 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:13.937196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:13.938010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:13.938643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:13.939113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:13.940514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:13.950854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:13.982922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2955/5883] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/projects => generated 6913 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:19:13.984135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb6583714f54aefa5ad1c19a4a26ec3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:13.986420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb6583714f54aefa5ad1c19a4a26ec3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:13.988071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:13.991408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.004579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb6583714f54aefa5ad1c19a4a26ec3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.018139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a7d80-37ad-40a8-81c5-0874914196a8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.026635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.027136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb6583714f54aefa5ad1c19a4a26ec3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.027514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.028152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.028583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.029393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:14.029890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.034071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.058138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cdf8fcd9-be8e-4c22-98a4-e988ac119efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb6583714f54aefa5ad1c19a4a26ec3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.075455 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2929/5884] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:13 2022] DELETE /identity/v3/users/9bb6583714f54aefa5ad1c19a4a26ec3 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:14.083521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e171-af80-482d-b077-72118847a41f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.083912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e171-af80-482d-b077-72118847a41f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.084355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e171-af80-482d-b077-72118847a41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.086058 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2956/5885] 10.222.0.26 () {58 vars in 1166 bytes} [Tue Sep 27 10:19:13 2022] GET /identity/v3/projects?domain_id=default => generated 6582 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:14.090259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.102985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecc4f3-64f4-484c-9648-a20eb6ad4b3e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.123134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cbc9e171-af80-482d-b077-72118847a41f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:14.124703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cbc9e171-af80-482d-b077-72118847a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.221434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.224649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.226131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.226600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.227994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.230774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.265779 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2957/5886] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:19:14 2022] GET /identity/v3/projects => generated 6913 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:19:14.267292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.269780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.279363 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2930/5887] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:14.280319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9e171-af80-482d-b077-72118847a41f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.291729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a6bdf-adbf-4ca8-89da-9338f8f07dca tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.294229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9e171-af80-482d-b077-72118847a41f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.299168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.299523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.299795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.300073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.300264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.300374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.300551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.300864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] PATH_INFO: `/v3/projects/ccf6dde33c064e69aa8eb8317514de9b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.301038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:14.301405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.302167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf6dde33c064e69aa8eb8317514de9b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.303873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.304841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.342873 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2931/5888] 10.222.0.26 () {58 vars in 1166 bytes} [Tue Sep 27 10:19:14 2022] GET /identity/v3/projects?domain_id=default => generated 6582 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:14.345752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.352080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.480145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:14.494156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf6dde33c064e69aa8eb8317514de9b', 'request_id': 'req-12ef84dd-2871-4c6f-971d-1684b3d85126'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.509823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf6dde33c064e69aa8eb8317514de9b', 'request_id': 'req-12ef84dd-2871-4c6f-971d-1684b3d85126'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.516558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12693fb8-9587-43f2-b2ea-4844664df6e8 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.528186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf6dde33c064e69aa8eb8317514de9b', 'request_id': 'req-12ef84dd-2871-4c6f-971d-1684b3d85126'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.566468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] The token cache is being invalidate because project ccf6dde33c064e69aa8eb8317514de9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:14.566797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.570721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:14.574423 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2958/5889] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:14 2022] DELETE /identity/v3/projects/ccf6dde33c064e69aa8eb8317514de9b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:14.583631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.587383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.657529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b416c-2438-47dd-920c-d08056274f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.658446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b416c-2438-47dd-920c-d08056274f93 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.659032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b416c-2438-47dd-920c-d08056274f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.659568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b416c-2438-47dd-920c-d08056274f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.662675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-753b416c-2438-47dd-920c-d08056274f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.721512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:14.760776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef84dd-2871-4c6f-971d-1684b3d85126 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.767134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.767775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.768058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.768335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.769302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.772461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.790037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b416c-2438-47dd-920c-d08056274f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.791706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b416c-2438-47dd-920c-d08056274f93 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.804907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2932/5890] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:14.806722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dd02c-7c2e-4796-9c4f-78282e2470ad None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.806985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dd02c-7c2e-4796-9c4f-78282e2470ad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.807254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dd02c-7c2e-4796-9c4f-78282e2470ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.818530 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2959/5891] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:19:14 2022] GET /identity/v3/projects => generated 6542 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:14.825058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.837824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-157dd02c-7c2e-4796-9c4f-78282e2470ad None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:14.838936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-157dd02c-7c2e-4796-9c4f-78282e2470ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.856846 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2933/5892] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:14.860584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d391d-d633-46a7-89f2-87c0fca1e9c7 tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.864007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938f6b7-cc9b-4cc6-ab54-423d6621242b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.864007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938f6b7-cc9b-4cc6-ab54-423d6621242b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.864350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938f6b7-cc9b-4cc6-ab54-423d6621242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.876890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.877347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.878178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.878673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.879473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-832634558) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:14.879839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.884571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.895216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8938f6b7-cc9b-4cc6-ab54-423d6621242b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:14.897243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8938f6b7-cc9b-4cc6-ab54-423d6621242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:14.907947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2960/5893] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:19:14 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-832634558 => generated 534 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:14.912331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.913681 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2934/5894] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:14.918823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938f6b7-cc9b-4cc6-ab54-423d6621242b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.930008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938f6b7-cc9b-4cc6-ab54-423d6621242b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.934597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13eb335-4360-4a7e-b505-9f7990ee699a tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.936015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.936985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.937518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.937998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.939015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:14.939550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.942190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.951687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8bb5a48317d4b04993d3edd6b299f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab83b763c59f4dada89c6b4620ffed0b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.952368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.953400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.953785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] PATH_INFO: `/v3/projects/20b453d19b6943aeb82aaeaaf6a558c7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.955898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorizing `identity:delete_project(project_id=20b453d19b6943aeb82aaeaaf6a558c7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:14.959947 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2935/5895] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:14.960980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:14.965202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:14.983036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5299608-098b-4c5c-abc6-3eb01c4d03d6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:14.991750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:14.992643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:14.993084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:14.993435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:14.995123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.000942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.042391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b453d19b6943aeb82aaeaaf6a558c7', 'request_id': 'req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.056573 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2936/5896] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:14 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:15.058089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:15.067645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b453d19b6943aeb82aaeaaf6a558c7', 'request_id': 'req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.076368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b453d19b6943aeb82aaeaaf6a558c7', 'request_id': 'req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.094523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] The token cache is being invalidate because project 20b453d19b6943aeb82aaeaaf6a558c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:15.095532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.096455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.098981 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2961/5897] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:14 2022] DELETE /identity/v3/projects/20b453d19b6943aeb82aaeaaf6a558c7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:15.102944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:15.110199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:15.140962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:15.159923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8d2b3-bdfa-4a0b-8e47-2e4eb12924b6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:15.166652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:15.168203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.169032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.169541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.170771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.174319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.217559 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2937/5898] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:15 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:15.220578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:15.223537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:15.232581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545fe8f-c4df-4e48-9e00-2d18040f9b0e tempest-ListProjectsStaticTestJSON-415093987 tempest-ListProjectsStaticTestJSON-415093987-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:15.238542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ffd5c-e0a8-4321-ba69-f7e7f7b28cec admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:15.241413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:15.242998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.243484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.243890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] PATH_INFO: `/v3/users/4cadaa04d6c745dda814566463731906` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.245097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:15.245723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cadaa04d6c745dda814566463731906)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.246067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.246479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.246908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.247853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.248578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.249786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.261743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2938/5899] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:15.265775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:15.277608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dcadb-bbc4-4887-b0c1-e6463162c144 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:15.283740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:15.285061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.285506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.286375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.287410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.290006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.303808 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2939/5900] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:15.308023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:15.325211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52fd-0f8e-4939-b09a-29b0281a78af admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:15.341078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4c879-87e4-4cbe-8265-6b12c161f640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:15.342832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4c879-87e4-4cbe-8265-6b12c161f640 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.343582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4c879-87e4-4cbe-8265-6b12c161f640 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.344203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4c879-87e4-4cbe-8265-6b12c161f640 admin admin] PATH_INFO: `/v3/projects/0e592e34dc61402f8919ad1b64fbe041/users/0c5bbb6cec7042eda3dfc7a6839dfeda/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.348081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cadaa04d6c745dda814566463731906'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.350794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cadaa04d6c745dda814566463731906'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.361927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4c879-87e4-4cbe-8265-6b12c161f640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e592e34dc61402f8919ad1b64fbe041, user_id=0c5bbb6cec7042eda3dfc7a6839dfeda, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.364799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4c879-87e4-4cbe-8265-6b12c161f640 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.367822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:15.383327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cadaa04d6c745dda814566463731906'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.387561 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2940/5901] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:15 2022] PUT /identity/v3/projects/0e592e34dc61402f8919ad1b64fbe041/users/0c5bbb6cec7042eda3dfc7a6839dfeda/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:15.399066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.399870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.400155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.408233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cadaa04d6c745dda814566463731906'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.414624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-66e7fce0-3880-49d0-adc4-6aae047a1977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cadaa04d6c745dda814566463731906'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:15.451395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2962/5902] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:15 2022] DELETE /identity/v3/users/4cadaa04d6c745dda814566463731906 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:15.458008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.459789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.460173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.493085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] MFA Rules not processed for user `0c5bbb6cec7042eda3dfc7a6839dfeda`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:15.494337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:15.500924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:15.503779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:15.624279 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2963/5903] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:15.677409 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2941/5904] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:15.682660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:15.684419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:15.716509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:15.721623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:15.875175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6253d0-3f3e-4fc9-8ea4-0ec83e9779d0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:15.877948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ba4a-948d-4988-a62d-918d2c7ddebd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:15.885653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:15.887097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.887646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.888685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.903280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.905212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.905355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e82fbe-60e9-46d5-bcf0-0cffca2b2eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:15.907090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e82fbe-60e9-46d5-bcf0-0cffca2b2eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:15.908636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e82fbe-60e9-46d5-bcf0-0cffca2b2eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:15.908955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e82fbe-60e9-46d5-bcf0-0cffca2b2eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:15.918053 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2942/5905] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:15.956983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e82fbe-60e9-46d5-bcf0-0cffca2b2eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:15.958620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e82fbe-60e9-46d5-bcf0-0cffca2b2eb0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:15.970270 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2964/5906] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:16.473050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:16.474880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:16.512467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a38751-444d-425b-a8b5-f7e969d575e2 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:16.525382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:16.526122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:16.526386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:16.526664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] PATH_INFO: `/v3/projects/73c1c382922145d9ab011b08870b9b16` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:16.530378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c1c382922145d9ab011b08870b9b16)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:16.546121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:16.658091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:16.783859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c1c382922145d9ab011b08870b9b16', 'request_id': 'req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:16.806342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c1c382922145d9ab011b08870b9b16', 'request_id': 'req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:16.844814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c1c382922145d9ab011b08870b9b16', 'request_id': 'req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:16.879642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] The token cache is being invalidate because project 73c1c382922145d9ab011b08870b9b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:16.879943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:16.880850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3a91fc19-a9fc-4d1e-ad17-81ebcf99e411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:16.887099 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2943/5907] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:16 2022] DELETE /identity/v3/projects/73c1c382922145d9ab011b08870b9b16 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:16.892080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131df7-e33f-4017-ad80-7887e68b5b6f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:16.892547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131df7-e33f-4017-ad80-7887e68b5b6f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:16.892632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131df7-e33f-4017-ad80-7887e68b5b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:16.931710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-44131df7-e33f-4017-ad80-7887e68b5b6f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:16.932995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-44131df7-e33f-4017-ad80-7887e68b5b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:17.105191 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2965/5908] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:19:17.112930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1a78-5dff-4232-9edc-432d9883953d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.113267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1a78-5dff-4232-9edc-432d9883953d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.113483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1a78-5dff-4232-9edc-432d9883953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.140266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-eaca1a78-5dff-4232-9edc-432d9883953d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:17.142241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eaca1a78-5dff-4232-9edc-432d9883953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:17.153216 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2944/5909] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:17.158235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131df7-e33f-4017-ad80-7887e68b5b6f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:17.172990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131df7-e33f-4017-ad80-7887e68b5b6f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:17.180413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:17.181698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.182086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.182453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.183204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:17.183602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:17.186009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:17.197405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:17.206511 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2966/5910] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:19:17.208556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca1a78-5dff-4232-9edc-432d9883953d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:17.223063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca1a78-5dff-4232-9edc-432d9883953d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:17.235079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:17.235956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.236545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.237169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.238131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:17.241800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:17.304568 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2945/5911] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:17 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:17.304852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:17.321005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ea741-787d-440f-a36d-df5f20835084 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:17.476374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:17.477447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.477447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.477447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.478231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:17.480263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:17.542725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2967/5912] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:17 2022] POST /identity/v3/users => generated 295 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:17.544020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:17.557346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83161cda-a734-43f9-9ee6-cf8a85e4d94b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:17.565895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:17.566908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.568645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.569213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.570130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:17.572382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:17.591793 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2946/5913] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:17.593501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:17.605545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb86ba-6237-4050-9128-ebe49da55d95 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:17.613164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648c5d5-5d11-421c-ad9c-5caf3721150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:17.614192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648c5d5-5d11-421c-ad9c-5caf3721150b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.614581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648c5d5-5d11-421c-ad9c-5caf3721150b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.615011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648c5d5-5d11-421c-ad9c-5caf3721150b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.616079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648c5d5-5d11-421c-ad9c-5caf3721150b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:17.618500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648c5d5-5d11-421c-ad9c-5caf3721150b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:17.633875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2968/5914] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:17.639680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:17.655522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ceae5-1308-4928-a127-f75127853ad3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:17.662506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:17.663371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.663979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.665033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] PATH_INFO: `/v3/projects/b590d2de576e41aab42f79e4daef9c63/users/74878b715fc94b23bb139cb30d8d20c0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.688147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b590d2de576e41aab42f79e4daef9c63, user_id=74878b715fc94b23bb139cb30d8d20c0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:17.693858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:17.722929 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2947/5915] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:17 2022] PUT /identity/v3/projects/b590d2de576e41aab42f79e4daef9c63/users/74878b715fc94b23bb139cb30d8d20c0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:17.723153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:17.723444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:17.723721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:17.744060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:17.745318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:17.894375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2969/5916] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:17.963556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:17.967814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:18.093301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d97d99-1897-4e36-8c1e-1736e9cbbdcc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:18.097756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:18.098380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:18.098650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:18.098933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:18.109816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:18.113208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:18.125238 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2948/5917] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:18.179830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:18.181583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:18.194946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdda3b5-dc34-4521-83f6-e1ef613d6a28 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:18.203398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:18.204397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:18.204730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:18.205207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] PATH_INFO: `/v3/projects/31c6e547947b48069245a6d13c8b6ace` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:18.206433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c6e547947b48069245a6d13c8b6ace)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:18.208388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:18.279762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c6e547947b48069245a6d13c8b6ace', 'request_id': 'req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.293471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c6e547947b48069245a6d13c8b6ace', 'request_id': 'req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.302436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c6e547947b48069245a6d13c8b6ace', 'request_id': 'req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.321281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] The token cache is being invalidate because project 31c6e547947b48069245a6d13c8b6ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:18.321662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.322454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.327333 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2970/5918] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:18 2022] DELETE /identity/v3/projects/31c6e547947b48069245a6d13c8b6ace => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:18.329586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:18.342939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814e40-f3e5-437f-8469-028fed0910de service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:18.422582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:18.443520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:18.444631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:18.445192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:18.445671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] PATH_INFO: `/v3/users/e8bb5a48317d4b04993d3edd6b299f3e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:18.447999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8bb5a48317d4b04993d3edd6b299f3e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:18.450045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:18.564426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8bb5a48317d4b04993d3edd6b299f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.565930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8bb5a48317d4b04993d3edd6b299f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.576043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8bb5a48317d4b04993d3edd6b299f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.588523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8bb5a48317d4b04993d3edd6b299f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.607396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d36b67b-fffd-4829-b853-0a7f20c40193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8bb5a48317d4b04993d3edd6b299f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:18.620478 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2949/5919] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:18 2022] DELETE /identity/v3/users/e8bb5a48317d4b04993d3edd6b299f3e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:19.755477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:19.761095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:19.777378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:19.788772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e9dd7-6ef0-4163-a9b5-cb3d48c468db admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:19.935648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:19.936878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:19.937155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:19.937428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] PATH_INFO: `/v3/users/563ca4b3ccd94ca7a62eea40a55f903a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:19.938776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563ca4b3ccd94ca7a62eea40a55f903a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:19.941443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:20.031307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563ca4b3ccd94ca7a62eea40a55f903a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:20.033661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563ca4b3ccd94ca7a62eea40a55f903a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:20.052818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563ca4b3ccd94ca7a62eea40a55f903a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:20.061520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563ca4b3ccd94ca7a62eea40a55f903a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:20.062639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563ca4b3ccd94ca7a62eea40a55f903a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:20.095673 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2971/5920] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:19 2022] DELETE /identity/v3/users/563ca4b3ccd94ca7a62eea40a55f903a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:20.098599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:20.099197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:20.099526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:20.125850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:20.127759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:20.193357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2950/5921] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:20.217246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:20.220027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:20.319444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f5445-a769-4fe0-9bc4-877033e2c4a7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:20.327063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaebd5-a6d8-4ed3-905e-0c77a51e75df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:20.327762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaebd5-a6d8-4ed3-905e-0c77a51e75df service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:20.328453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaebd5-a6d8-4ed3-905e-0c77a51e75df service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:20.328792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaebd5-a6d8-4ed3-905e-0c77a51e75df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:20.345834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaebd5-a6d8-4ed3-905e-0c77a51e75df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:20.351441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaebd5-a6d8-4ed3-905e-0c77a51e75df service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:20.367931 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2972/5922] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:19:20.915535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:20.918272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:20.943507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ed96-c88c-4193-8e42-b86813eb7ac4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:20.968514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:20.970439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:20.970439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:20.970439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] PATH_INFO: `/v3/projects/ab83b763c59f4dada89c6b4620ffed0b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:20.971386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab83b763c59f4dada89c6b4620ffed0b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:20.973383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:21.074986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab83b763c59f4dada89c6b4620ffed0b', 'request_id': 'req-b093253b-7c89-492b-884d-728d51d7af51'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:21.086889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab83b763c59f4dada89c6b4620ffed0b', 'request_id': 'req-b093253b-7c89-492b-884d-728d51d7af51'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:21.101366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab83b763c59f4dada89c6b4620ffed0b', 'request_id': 'req-b093253b-7c89-492b-884d-728d51d7af51'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:21.109161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:21.126840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] The token cache is being invalidate because project ab83b763c59f4dada89c6b4620ffed0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:21.127653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:21.129204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b093253b-7c89-492b-884d-728d51d7af51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:21.132261 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2951/5923] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:20 2022] DELETE /identity/v3/projects/ab83b763c59f4dada89c6b4620ffed0b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:21.137282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7adb8-9aac-4d4a-8ef2-b94c1777648a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.138555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7adb8-9aac-4d4a-8ef2-b94c1777648a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.138819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7adb8-9aac-4d4a-8ef2-b94c1777648a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.161808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ceb7adb8-9aac-4d4a-8ef2-b94c1777648a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:21.163452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ceb7adb8-9aac-4d4a-8ef2-b94c1777648a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:21.203365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:21.254763 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2973/5924] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:21.264558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26344c-34ae-43b1-87e1-d8ff9f04d541 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.264946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26344c-34ae-43b1-87e1-d8ff9f04d541 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.265305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26344c-34ae-43b1-87e1-d8ff9f04d541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.289235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6e26344c-34ae-43b1-87e1-d8ff9f04d541 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:21.290812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6e26344c-34ae-43b1-87e1-d8ff9f04d541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:21.318163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7adb8-9aac-4d4a-8ef2-b94c1777648a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:21.318904 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2952/5925] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:21.341754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7adb8-9aac-4d4a-8ef2-b94c1777648a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:21.350110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:21.351347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.351897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.352235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.353752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:21.355313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:21.385500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:21.417899 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2974/5926] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:21.422400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26344c-34ae-43b1-87e1-d8ff9f04d541 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:21.437471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26344c-34ae-43b1-87e1-d8ff9f04d541 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:21.449659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:21.450306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.450588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.450859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.451520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:21.453941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:21.558213 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2953/5927] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:21 2022] POST /identity/v3/projects => generated 359 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:19:21.561391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:21.589896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c8821-0671-4811-9084-d78f22aef674 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:21.710248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:21.711016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.711358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.711705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.712454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:21.714769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:21.768775 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2975/5928] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:21 2022] POST /identity/v3/users => generated 280 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:21.773046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:21.791370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54fd0-2123-409e-8a95-23937a606e76 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:21.816582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:21.817395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.817841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.818915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.820798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:21.824325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:21.840709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2954/5929] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:21.844362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:21.858101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ead649-4b17-4d08-89e6-3978b5f144a0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:21.866411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240757c6-b498-49ea-896d-371b2440e145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:21.867840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240757c6-b498-49ea-896d-371b2440e145 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.868325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240757c6-b498-49ea-896d-371b2440e145 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.868670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240757c6-b498-49ea-896d-371b2440e145 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.869489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240757c6-b498-49ea-896d-371b2440e145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:21.871315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240757c6-b498-49ea-896d-371b2440e145 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:21.889720 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2976/5930] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:21.892377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:21.906282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589639-d03e-4c12-a47c-b1457838919e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:21.913341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:21.914328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.914667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.915143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] PATH_INFO: `/v3/projects/3a7e628564cc47e7a584fa4db83a342f/users/f979affd285a4273a03982532d686dfc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:21.939076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a7e628564cc47e7a584fa4db83a342f, user_id=f979affd285a4273a03982532d686dfc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:21.943117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:21.967980 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2955/5931] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:21 2022] PUT /identity/v3/projects/3a7e628564cc47e7a584fa4db83a342f/users/f979affd285a4273a03982532d686dfc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:21.980382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0006d84-1797-4029-bd8d-17d0cce7d918 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:21.981015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0006d84-1797-4029-bd8d-17d0cce7d918 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:21.981300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0006d84-1797-4029-bd8d-17d0cce7d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.027876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b0006d84-1797-4029-bd8d-17d0cce7d918 None None] MFA Rules not processed for user `f979affd285a4273a03982532d686dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:22.028612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b0006d84-1797-4029-bd8d-17d0cce7d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.079978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.082581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.216596 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2977/5932] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:22.221236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d93d-5a8a-4c55-ae1f-2a65a2f897fe None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.221522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d93d-5a8a-4c55-ae1f-2a65a2f897fe None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.221915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404d93d-5a8a-4c55-ae1f-2a65a2f897fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.263142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9404d93d-5a8a-4c55-ae1f-2a65a2f897fe None None] MFA Rules not processed for user `f979affd285a4273a03982532d686dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:22.264582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9404d93d-5a8a-4c55-ae1f-2a65a2f897fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.269245 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2978/5933] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:22 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 10:19:22.275438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ca1c5-e18b-4efa-8d29-467e181df529 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.275857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ca1c5-e18b-4efa-8d29-467e181df529 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.276371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ca1c5-e18b-4efa-8d29-467e181df529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.278515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ce18e-29cf-41fc-b659-3f2d347f272b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.296630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.297477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.298741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.300904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] PATH_INFO: `/v3/projects/c344fff70741430c8b375ff2f2fe0b55` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.304606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c344fff70741430c8b375ff2f2fe0b55)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:22.306745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:22.321855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-492ca1c5-e18b-4efa-8d29-467e181df529 None None] MFA Rules not processed for user `f979affd285a4273a03982532d686dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:22.323240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-492ca1c5-e18b-4efa-8d29-467e181df529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.331600 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2979/5934] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:22 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:19:22.334838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ca1c5-e18b-4efa-8d29-467e181df529 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.360497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ca1c5-e18b-4efa-8d29-467e181df529 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.381390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f979affd285a4273a03982532d686dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a7e628564cc47e7a584fa4db83a342f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.383316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.384466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.385557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.394711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:22.405686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorizing `identity:check_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:22.407666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:22.409703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c344fff70741430c8b375ff2f2fe0b55', 'request_id': 'req-f9c31e55-9495-4451-98ca-b05341c3c9a8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:22.432717 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2980/5935] 10.222.0.26 () {56 vars in 1264 bytes} [Tue Sep 27 10:19:22 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Sep 27 10:19:22.436135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.437081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c344fff70741430c8b375ff2f2fe0b55', 'request_id': 'req-f9c31e55-9495-4451-98ca-b05341c3c9a8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:22.459293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166385a3-90e6-46f6-803b-977e9f08ae9c tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.465660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c344fff70741430c8b375ff2f2fe0b55', 'request_id': 'req-f9c31e55-9495-4451-98ca-b05341c3c9a8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:22.480725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] The token cache is being invalidate because project c344fff70741430c8b375ff2f2fe0b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:22.481161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:22.482439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9c31e55-9495-4451-98ca-b05341c3c9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:22.484291 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2956/5936] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:22 2022] DELETE /identity/v3/projects/c344fff70741430c8b375ff2f2fe0b55 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:22.489496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eba08f-5972-475c-9b10-25007efba75f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.489496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eba08f-5972-475c-9b10-25007efba75f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.490981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eba08f-5972-475c-9b10-25007efba75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.515098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c5eba08f-5972-475c-9b10-25007efba75f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:22.516350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c5eba08f-5972-475c-9b10-25007efba75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.533031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:22.539115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f979affd285a4273a03982532d686dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a7e628564cc47e7a584fa4db83a342f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.540427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.540561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.541398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.543378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.555523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:22.557075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:22.568066 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2981/5937] 10.222.0.26 () {56 vars in 1263 bytes} [Tue Sep 27 10:19:22 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:22.570192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.571674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.590775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943481f-9673-48c9-adb6-dd61cb400c19 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.605697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f979affd285a4273a03982532d686dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a7e628564cc47e7a584fa4db83a342f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.609579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.610480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.610927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.625464 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2957/5938] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:22.628475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74831536-f5a1-44e5-b278-b63bc21c5f56 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.629041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74831536-f5a1-44e5-b278-b63bc21c5f56 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.629557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74831536-f5a1-44e5-b278-b63bc21c5f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.634606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:22.638094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:22.661422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-74831536-f5a1-44e5-b278-b63bc21c5f56 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:22.662858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74831536-f5a1-44e5-b278-b63bc21c5f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.675073 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2982/5939] 10.222.0.26 () {56 vars in 1266 bytes} [Tue Sep 27 10:19:22 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:22.677481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.690937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e7053-eab6-4064-83ab-fde8751bbb6f tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.691381 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2958/5940] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:22.692889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74831536-f5a1-44e5-b278-b63bc21c5f56 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.697594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f979affd285a4273a03982532d686dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a7e628564cc47e7a584fa4db83a342f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.698062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.698390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.698860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.700849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Sep 27 10:19:22.717865 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 27 10:19:22.719017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 27 10:19:22.721391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:22.721391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:22.721391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 27 10:19:22.721391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 27 10:19:22.721391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 27 10:19:22.721391 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:19:22.721391 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2983/5941] 10.222.0.26 () {56 vars in 1264 bytes} [Tue Sep 27 10:19:22 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) Sep 27 10:19:22.726102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74831536-f5a1-44e5-b278-b63bc21c5f56 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.726412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.730364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.732147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d453f64-83e6-47ef-a65e-fe6564e897f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.733501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d453f64-83e6-47ef-a65e-fe6564e897f4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.734002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d453f64-83e6-47ef-a65e-fe6564e897f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.734493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d453f64-83e6-47ef-a65e-fe6564e897f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.735600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d453f64-83e6-47ef-a65e-fe6564e897f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:22.736123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d453f64-83e6-47ef-a65e-fe6564e897f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:22.737998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d453f64-83e6-47ef-a65e-fe6564e897f4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:22.761590 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2959/5942] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:22.761590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04117d7b-9b39-4333-99af-2e95e64645d8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.761590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04117d7b-9b39-4333-99af-2e95e64645d8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.761590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04117d7b-9b39-4333-99af-2e95e64645d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.786251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-04117d7b-9b39-4333-99af-2e95e64645d8 None None] MFA Rules not processed for user `f979affd285a4273a03982532d686dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:22.788941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-04117d7b-9b39-4333-99af-2e95e64645d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:22.796557 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2960/5943] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:22 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:19:22.797113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04117d7b-9b39-4333-99af-2e95e64645d8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.815549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04117d7b-9b39-4333-99af-2e95e64645d8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.823790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.824593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.825000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.825690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.826148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:22.830457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:22.876483 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2961/5944] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:22 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:22.876483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.958050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae927bd-db08-4924-95dd-8fe560537817 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:22.965563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f979affd285a4273a03982532d686dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a7e628564cc47e7a584fa4db83a342f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:22.966351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:22.966877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:22.967226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:22.976057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:22.978208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:22.996426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2962/5945] 10.222.0.26 () {56 vars in 1263 bytes} [Tue Sep 27 10:19:22 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:22.998225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:22.999628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566d487-3bfc-4108-958a-af322f522c39 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.087014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e50ec-ec4d-4a53-a01f-32e36eb583c6 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.093843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.094712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.095012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.095289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.096105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.098078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.101364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:23.115740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:23.125529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.126618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.126926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.127165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.129644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.146836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2963/5946] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:22 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:23.148830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.151950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.156537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.158214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.182352 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2984/5947] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:23.182767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3ffe6-3fe0-4aec-a262-3702455f4c5a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.182896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.191872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.192916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.193569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.195641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.199693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.206678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13849ddf-ae43-4c42-a434-548aaa2b68f8 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.211600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f979affd285a4273a03982532d686dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a7e628564cc47e7a584fa4db83a342f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.212222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.212496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.212795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.223798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.227300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.227731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.229281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.252260 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2964/5948] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:23.253581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.262685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2985/5949] 10.222.0.26 () {56 vars in 1266 bytes} [Tue Sep 27 10:19:23 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:23.263579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.266659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.274475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba67b5-9090-4caa-afc2-a44361ec4741 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.281967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.285506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.286036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.287147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.288965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.294312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.295539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080815be-c53e-471f-a56d-84a558eb3719 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.309344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.309999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.310276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.310551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] PATH_INFO: `/v3/users/dbf876e1c23642578363f9c5995f3d86` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.311718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf876e1c23642578363f9c5995f3d86)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.316225 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2965/5950] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:23.316534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.316639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.335813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd53ee-476a-4c2b-ba79-2fa891e25a13 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.343378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f979affd285a4273a03982532d686dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a7e628564cc47e7a584fa4db83a342f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.343977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.344264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.344544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.348497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 27 10:19:23.371804 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 27 10:19:23.373453 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2966/5951] 10.222.0.26 () {56 vars in 1263 bytes} [Tue Sep 27 10:19:23 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 10:19:23.375262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.399646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36b72f-5cd8-4604-8363-cb26c84a7b92 tempest-TokensV3Test-334699015 tempest-TokensV3Test-334699015-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.406881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.407916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.408345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.408623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.409488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.413689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.427613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf876e1c23642578363f9c5995f3d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.428635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf876e1c23642578363f9c5995f3d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.432156 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2967/5952] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:23.432156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.437818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.457397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf876e1c23642578363f9c5995f3d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.460718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351dbd2-dca9-43f9-b238-2fc10c8f80cb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.467268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.467967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.468292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.468568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] PATH_INFO: `/v3/users/f979affd285a4273a03982532d686dfc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.470562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f979affd285a4273a03982532d686dfc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.473506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.478934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf876e1c23642578363f9c5995f3d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.481954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf876e1c23642578363f9c5995f3d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.544531 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2986/5953] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:23 2022] DELETE /identity/v3/users/dbf876e1c23642578363f9c5995f3d86 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:23.550446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.556345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.580562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f979affd285a4273a03982532d686dfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.581502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f979affd285a4273a03982532d686dfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.608602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f979affd285a4273a03982532d686dfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.637084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f979affd285a4273a03982532d686dfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.679174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b463187-253c-49c5-9da8-c187161dbc2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f979affd285a4273a03982532d686dfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:23.705296 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2968/5954] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:23 2022] DELETE /identity/v3/users/f979affd285a4273a03982532d686dfc => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:23.710170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.713892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.714411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.740547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:23.742978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.756393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4dc98-80a2-44e0-8b79-aeec72fde2ec admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:23.843596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:23.867066 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2969/5955] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:23.882424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362349c-b3a0-466c-84b5-de7b4b1e4138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:23.883739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362349c-b3a0-466c-84b5-de7b4b1e4138 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.884229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362349c-b3a0-466c-84b5-de7b4b1e4138 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.884886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362349c-b3a0-466c-84b5-de7b4b1e4138 admin admin] PATH_INFO: `/v3/projects/e3509b4f3388466a8e31df573dabceee/users/6e080e5097014390a9c5c27f220d60c0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.908792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362349c-b3a0-466c-84b5-de7b4b1e4138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3509b4f3388466a8e31df573dabceee, user_id=6e080e5097014390a9c5c27f220d60c0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:23.914526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362349c-b3a0-466c-84b5-de7b4b1e4138 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:23.929413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:23.931059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:23.948149 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2987/5956] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:23 2022] PUT /identity/v3/projects/e3509b4f3388466a8e31df573dabceee/users/6e080e5097014390a9c5c27f220d60c0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:23.959360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a550c7-375a-46fe-b28f-c2bcb3114811 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:23.959672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a550c7-375a-46fe-b28f-c2bcb3114811 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:23.959945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a550c7-375a-46fe-b28f-c2bcb3114811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:23.998250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-04a550c7-375a-46fe-b28f-c2bcb3114811 None None] MFA Rules not processed for user `6e080e5097014390a9c5c27f220d60c0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:23.999499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-04a550c7-375a-46fe-b28f-c2bcb3114811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:24.001358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:24.104039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a1a42-1d8d-452a-b098-448168b2b0d3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:24.173988 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2988/5957] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:24.243626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:24.244272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:24.244554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:24.244854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:24.346458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a550c7-375a-46fe-b28f-c2bcb3114811 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:24.364493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a550c7-375a-46fe-b28f-c2bcb3114811 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:24.391644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1460f-d19f-4660-8c15-d87a6b5d454b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:24.392299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1460f-d19f-4660-8c15-d87a6b5d454b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:24.398586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1460f-d19f-4660-8c15-d87a6b5d454b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:24.398586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1460f-d19f-4660-8c15-d87a6b5d454b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:24.420708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:24.423604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:24.425874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1460f-d19f-4660-8c15-d87a6b5d454b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:24.428134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1460f-d19f-4660-8c15-d87a6b5d454b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:24.452003 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2970/5958] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:24.455822 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2989/5959] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:24.917671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:24.919479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:24.946200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9b80c-bd70-4d8b-b304-b474b3f0826f service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:24.954005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:24.954670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:24.954949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:24.955360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] PATH_INFO: `/v3/projects/8070262519034021a2b6de1a1e1ec88a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:24.959892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8070262519034021a2b6de1a1e1ec88a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:24.962193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:25.136933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8070262519034021a2b6de1a1e1ec88a', 'request_id': 'req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.157313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8070262519034021a2b6de1a1e1ec88a', 'request_id': 'req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.181848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8070262519034021a2b6de1a1e1ec88a', 'request_id': 'req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.204756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] The token cache is being invalidate because project 8070262519034021a2b6de1a1e1ec88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:25.205056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.210081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.212098 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2971/5960] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:24 2022] DELETE /identity/v3/projects/8070262519034021a2b6de1a1e1ec88a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:25.217201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376e996-59e1-47c7-ab22-6882d7c7e231 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:25.217483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376e996-59e1-47c7-ab22-6882d7c7e231 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:25.218131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376e996-59e1-47c7-ab22-6882d7c7e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:25.251400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2376e996-59e1-47c7-ab22-6882d7c7e231 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:25.252929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2376e996-59e1-47c7-ab22-6882d7c7e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:25.378841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:25.381852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:25.423662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:25.429605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2990/5961] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:25.437891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24def4a-3ea8-4a73-945c-cc50f31dbdb0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:25.438606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24def4a-3ea8-4a73-945c-cc50f31dbdb0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:25.438606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24def4a-3ea8-4a73-945c-cc50f31dbdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:25.473816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f24def4a-3ea8-4a73-945c-cc50f31dbdb0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:25.475295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f24def4a-3ea8-4a73-945c-cc50f31dbdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:25.498938 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2991/5962] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:25.499958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24def4a-3ea8-4a73-945c-cc50f31dbdb0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:25.526988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24def4a-3ea8-4a73-945c-cc50f31dbdb0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:25.538051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:25.539180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:25.539719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:25.540145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:25.543085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:25.543597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:25.545678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:25.591102 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2992/5963] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:25.594593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:25.611802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c70b37-6ce9-4dbb-9818-82f8b7b849b6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:25.625937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf44cc-2f46-416b-a476-9a8601a954f4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:25.629212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:25.634632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:25.634982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:25.635313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] PATH_INFO: `/v3/users/6e080e5097014390a9c5c27f220d60c0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:25.641841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e080e5097014390a9c5c27f220d60c0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:25.645535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:25.659984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:25.660889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:25.661264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:25.661712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:25.663253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:25.665801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:25.724815 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2993/5964] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:25 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:25.726249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:25.734890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:25.789190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e080e5097014390a9c5c27f220d60c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.790809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e080e5097014390a9c5c27f220d60c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.809106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e080e5097014390a9c5c27f220d60c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.830118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e080e5097014390a9c5c27f220d60c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.855614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-39851a71-6aec-4c34-8abe-2ce65bb57003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e080e5097014390a9c5c27f220d60c0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:25.857997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810340cf-2610-4372-a92b-edbb6825b575 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:25.865702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:25.866673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:25.867555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:25.868241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:25.869309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:25.872102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:25.875581 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2972/5965] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:25 2022] DELETE /identity/v3/users/6e080e5097014390a9c5c27f220d60c0 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:25.877310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:25.877605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:25.877997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:25.913663 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2994/5966] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:25 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:25.915087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:25.917515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:25.920933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:25.924904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:25.941831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585490bf-e392-46ba-8e5a-15f3df4f8522 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:26.077622 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2973/5967] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:19:26.109558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:26.111197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:26.121034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:26.123023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.123308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.123582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.124248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:26.131704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:26.157643 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2995/5968] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:26.161727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:26.185790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:26.186015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2060ff-8504-49b0-96d6-0cbadf948e27 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:26.201827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:26.204345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.205535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.205990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.208187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:26.211967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:26.241121 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2996/5969] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:26.241293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada65b5d-8436-4d60-8156-38dad45925aa None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:26.244163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:26.253284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3297610-311c-408e-bb35-8c739b443a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:26.254108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3297610-311c-408e-bb35-8c739b443a3d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.254390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3297610-311c-408e-bb35-8c739b443a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.254663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3297610-311c-408e-bb35-8c739b443a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.273168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2940451-7a0d-406e-874d-7ad7abe80080 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:26.285371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:26.286070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.286352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.286741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] PATH_INFO: `/v3/projects/cb035a91b1f341ce8d8b71d39ff10a0b/users/377e47048a63456793dfc9c125f5a209/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.289495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3297610-311c-408e-bb35-8c739b443a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:26.290980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3297610-311c-408e-bb35-8c739b443a3d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:26.318176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2974/5970] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:19:26.324597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb035a91b1f341ce8d8b71d39ff10a0b, user_id=377e47048a63456793dfc9c125f5a209, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:26.328522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:26.356014 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2997/5971] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:26 2022] PUT /identity/v3/projects/cb035a91b1f341ce8d8b71d39ff10a0b/users/377e47048a63456793dfc9c125f5a209/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:26.361221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.361731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.362344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.391256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:26.396371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:26.447037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:26.450836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:26.534105 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2975/5972] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:26.600096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:26.608118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:26.619607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:26.628911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05250840-5323-44a5-ba4e-65e970b44cc2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:26.636762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:26.637715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.638378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.638800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] PATH_INFO: `/v3/projects/3a7e628564cc47e7a584fa4db83a342f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.640472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a7e628564cc47e7a584fa4db83a342f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:26.644358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:26.754189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a7e628564cc47e7a584fa4db83a342f', 'request_id': 'req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:26.782822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a7e628564cc47e7a584fa4db83a342f', 'request_id': 'req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:26.786102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce74f3-6600-4c10-82ab-62be07bd33cb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:26.818178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a7e628564cc47e7a584fa4db83a342f', 'request_id': 'req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:26.855476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] The token cache is being invalidate because project 3a7e628564cc47e7a584fa4db83a342f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:26.856698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:26.860214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3445e6c6-41d6-4feb-abe1-1b88770bcc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:26.860214 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2998/5973] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:26 2022] DELETE /identity/v3/projects/3a7e628564cc47e7a584fa4db83a342f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:26.868759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948208ed-7530-479b-93ed-4e7ec2bc44ad None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.869562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948208ed-7530-479b-93ed-4e7ec2bc44ad None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.870007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948208ed-7530-479b-93ed-4e7ec2bc44ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.909193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-948208ed-7530-479b-93ed-4e7ec2bc44ad None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:26.910882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-948208ed-7530-479b-93ed-4e7ec2bc44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:26.923146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:26.925113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:26.925402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:26.925678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:26.930464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:27.075390 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 2999/5974] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:27.083886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbfe1d-bdcb-4740-a3e7-1fa74095e750 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.084176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbfe1d-bdcb-4740-a3e7-1fa74095e750 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.084439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbfe1d-bdcb-4740-a3e7-1fa74095e750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.085215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:27.121137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.131930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.133758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-06cbfe1d-bdcb-4740-a3e7-1fa74095e750 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:27.135005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-06cbfe1d-bdcb-4740-a3e7-1fa74095e750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:27.158205 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3000/5975] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:27.159296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbfe1d-bdcb-4740-a3e7-1fa74095e750 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:27.161072 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2976/5976] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:19:27.174660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbfe1d-bdcb-4740-a3e7-1fa74095e750 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:27.180417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:27.181374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.181653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.181927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.182594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:27.182897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.184532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.206655 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3001/5977] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:27.212705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:27.230151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367e140-d81c-40e4-b3c5-e1f527cbeb8f service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:27.239195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:27.241140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.241414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.241678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.247222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.252393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.296484 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2977/5978] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:27 2022] POST /identity/v3/projects => generated 399 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:27.301019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:27.314368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195134-39b7-4025-94f7-21aef6c4b5a5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:27.420456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:27.421238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.421986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.422338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.423779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.426224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.448848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:27.461439 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3002/5979] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:27 2022] POST /identity/v3/users => generated 306 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:27.469513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:27.482300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba983ee-a9c2-4614-8b99-76fb356efa68 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:27.487910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:27.488724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.489199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.489614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.490525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.492350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.498606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:27.500497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:27.517892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55184c5a-dac1-45f5-ac51-4f6be7b71962 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:27.525506 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2978/5980] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:27.527329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:27.533529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:27.537070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.537627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.537627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] PATH_INFO: `/v3/projects/e3509b4f3388466a8e31df573dabceee` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.539745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3509b4f3388466a8e31df573dabceee)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.543503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.549412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182f6c0-802f-4ae2-924d-ef342bf9dc92 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:27.566072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:27.567156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.568032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.568032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] PATH_INFO: `/v3/projects/fba836ddf0a14ee0ae089f3ec0db6a3f/users/f5557679aeb941c3920791ae3657c4cd/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.618325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba836ddf0a14ee0ae089f3ec0db6a3f, user_id=f5557679aeb941c3920791ae3657c4cd, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.622159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.657020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2979/5981] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:27 2022] PUT /identity/v3/projects/fba836ddf0a14ee0ae089f3ec0db6a3f/users/f5557679aeb941c3920791ae3657c4cd/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:27.663994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:27.683657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3509b4f3388466a8e31df573dabceee', 'request_id': 'req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:27.709046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3509b4f3388466a8e31df573dabceee', 'request_id': 'req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:27.718451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3509b4f3388466a8e31df573dabceee', 'request_id': 'req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:27.744684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] The token cache is being invalidate because project e3509b4f3388466a8e31df573dabceee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:27.745070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:27.746224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ff0352-d3d8-4847-8c3b-1eb29a7ac24a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:27.749775 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3003/5982] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:27 2022] DELETE /identity/v3/projects/e3509b4f3388466a8e31df573dabceee => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:27.754772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c259-d869-4b20-bb8e-6665117ce86c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.755208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c259-d869-4b20-bb8e-6665117ce86c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.755633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c259-d869-4b20-bb8e-6665117ce86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.796991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5355c259-d869-4b20-bb8e-6665117ce86c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:27.800354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5355c259-d869-4b20-bb8e-6665117ce86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:27.816641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11b115-1913-4e83-a19a-2b32d417f4ad admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:27.929288 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3004/5983] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:27.935175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bc7de-a4c3-4a5c-8221-3e7d7c15a181 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.936827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bc7de-a4c3-4a5c-8221-3e7d7c15a181 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.938695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bc7de-a4c3-4a5c-8221-3e7d7c15a181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.941916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:27.944294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:27.945081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:27.945889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:27.947309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:27.953745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:27.988326 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2980/5984] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:27.993316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.000239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:28.003803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dc5bc7de-a4c3-4a5c-8221-3e7d7c15a181 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:28.007966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dc5bc7de-a4c3-4a5c-8221-3e7d7c15a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:28.041089 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3005/5985] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:28.042522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfeb16-deea-41a8-af4d-6a619a9d6f67 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.045364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bc7de-a4c3-4a5c-8221-3e7d7c15a181 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.062086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08306e0-ed9a-48eb-a5b4-6498db138d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.064614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08306e0-ed9a-48eb-a5b4-6498db138d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.064929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08306e0-ed9a-48eb-a5b4-6498db138d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.065210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08306e0-ed9a-48eb-a5b4-6498db138d35 admin admin] PATH_INFO: `/v3/projects/fba836ddf0a14ee0ae089f3ec0db6a3f/users/f5557679aeb941c3920791ae3657c4cd/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.075172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bc7de-a4c3-4a5c-8221-3e7d7c15a181 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.075938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08306e0-ed9a-48eb-a5b4-6498db138d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba836ddf0a14ee0ae089f3ec0db6a3f, user_id=f5557679aeb941c3920791ae3657c4cd, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.084924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08306e0-ed9a-48eb-a5b4-6498db138d35 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.094945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.095588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.095878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.098539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.099321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:28.099783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.113676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.126164 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2981/5986] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:27 2022] PUT /identity/v3/projects/fba836ddf0a14ee0ae089f3ec0db6a3f/users/f5557679aeb941c3920791ae3657c4cd/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:28.136474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e211a-2b4a-492e-bb2a-1d0a56990a1b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.136848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e211a-2b4a-492e-bb2a-1d0a56990a1b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.137089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e211a-2b4a-492e-bb2a-1d0a56990a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.143317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3006/5987] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:28.144835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.156180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c11e3a-ac87-492c-9ef5-b6f7cd17088e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.200376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2c5e211a-2b4a-492e-bb2a-1d0a56990a1b None None] MFA Rules not processed for user `f5557679aeb941c3920791ae3657c4cd`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:28.201896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2c5e211a-2b4a-492e-bb2a-1d0a56990a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:28.210826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:28.260122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.260984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.261359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.261724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.262473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.264791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.281058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:28.337415 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3007/5988] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:28 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:28.342067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.358578 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2982/5989] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:28 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:28.364003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e211a-2b4a-492e-bb2a-1d0a56990a1b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.396150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e211a-2b4a-492e-bb2a-1d0a56990a1b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.522777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29844693-8f83-4ffc-b08b-30ca2f3db48a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.534530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.535264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.536816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.537201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.539936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.542993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.550857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.553648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.554034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.554685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.555472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.561227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.596694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3008/5990] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:28 2022] POST /identity/v3/users => generated 282 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:28.598818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.618616 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2983/5991] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:28 2022] POST /identity/v3/projects => generated 399 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:28.620185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.627994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74f011-b214-44fd-9512-c27266706d16 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.720168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.720936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.721347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.721686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.722725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.726209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.747075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3009/5992] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:28.749024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.773727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443b429-5c62-41e6-9d7d-0aeb24dde8e1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.794819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.795465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.796037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.796865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.797527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.798192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa6fc1-a3d6-4b83-8c93-d0543f195866 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.799153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.814387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.816424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.816424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.816424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.816424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.818323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.824638 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3010/5993] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:28.825871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.842221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f91f-8d0f-45f7-aa0f-d0bcd16b1f00 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.849466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248bef-cedd-40cc-887a-025b549e5440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.850861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248bef-cedd-40cc-887a-025b549e5440 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.851200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248bef-cedd-40cc-887a-025b549e5440 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.851580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248bef-cedd-40cc-887a-025b549e5440 admin admin] PATH_INFO: `/v3/projects/0ff624ba07b54ec985a94783d529cdef/users/3d42fa166bb14724a1ddffed4a6c6231/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.864067 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2984/5994] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:28 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:28.870535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.889187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248bef-cedd-40cc-887a-025b549e5440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff624ba07b54ec985a94783d529cdef, user_id=3d42fa166bb14724a1ddffed4a6c6231, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.889664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dad2-1c3a-4c7d-b52d-522dac56db7c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:28.895945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248bef-cedd-40cc-887a-025b549e5440 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.897573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:28.898877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.899161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.899483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.901093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:28.905110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:28.925992 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3011/5995] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:28 2022] PUT /identity/v3/projects/0ff624ba07b54ec985a94783d529cdef/users/3d42fa166bb14724a1ddffed4a6c6231/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:28.938063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e03749-4dbd-497b-8c40-0a0364d8b7b3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:28.938400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e03749-4dbd-497b-8c40-0a0364d8b7b3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:28.938612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e03749-4dbd-497b-8c40-0a0364d8b7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:28.955476 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2985/5996] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:28.961489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:28.979244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e8e03749-4dbd-497b-8c40-0a0364d8b7b3 None None] MFA Rules not processed for user `3d42fa166bb14724a1ddffed4a6c6231`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:28.980595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e8e03749-4dbd-497b-8c40-0a0364d8b7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:28.981248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:28.993663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68ac2f-06e0-4854-b485-8e2c27b15d5a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.011824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:29.103361 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3012/5997] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:19:29.105612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e03749-4dbd-497b-8c40-0a0364d8b7b3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:29.146688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.146688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.146688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.146688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.147412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.148611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:29.181334 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2986/5998] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:29.182495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:29.203462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baca4d-81a7-4d24-aa2b-3657cc63638a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.215152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070a360-4fff-4f6a-a8b4-37ea761ec8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.215843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070a360-4fff-4f6a-a8b4-37ea761ec8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.216124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070a360-4fff-4f6a-a8b4-37ea761ec8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.216401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070a360-4fff-4f6a-a8b4-37ea761ec8a1 admin admin] PATH_INFO: `/v3/projects/81366aa764eb472ea0638bfe9ff90dec/users/073008f8c5434d8086490b017a9b98f1/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.246766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070a360-4fff-4f6a-a8b4-37ea761ec8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81366aa764eb472ea0638bfe9ff90dec, user_id=073008f8c5434d8086490b017a9b98f1, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.249531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070a360-4fff-4f6a-a8b4-37ea761ec8a1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:29.262586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e03749-4dbd-497b-8c40-0a0364d8b7b3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.270853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.272529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.273858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.274271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.275137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.283306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:29.288628 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2987/5999] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:29 2022] PUT /identity/v3/projects/81366aa764eb472ea0638bfe9ff90dec/users/073008f8c5434d8086490b017a9b98f1/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:29.290240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.290524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.290800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.345725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3013/6000] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:29 2022] POST /identity/v3/projects => generated 365 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:29.347163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:29.366518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] MFA Rules not processed for user `073008f8c5434d8086490b017a9b98f1`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:29.368036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:29.466823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34eb61-be72-4ba1-baea-46ed295f6ed3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.473002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.474753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.476873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.482776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.483560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.485581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:29.517670 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2988/6001] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:29.536368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:29.539762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:29.555476 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3014/6002] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:29 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:29.557612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:29.574007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1c04d-803b-4b49-b993-13906f7c6569 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.583110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.585525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.585989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.587205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.588009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.591019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:29.608041 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3015/6003] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:29.614049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:29.645973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72b1d8-f84f-4dfc-9bf5-90dd4548aed4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.661561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.664939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.667017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.667602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] PATH_INFO: `/v3/projects/f510840390bb432ba9e2d2069a16ae5c/users/094a5e4059654755a29deba1a32e7ecc/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.691288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f510840390bb432ba9e2d2069a16ae5c, user_id=094a5e4059654755a29deba1a32e7ecc, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.695232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:29.697728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8eca6-6745-41e3-96f7-ea8227bc4f29 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.702465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.703111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.703764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.704066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.730051 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3016/6004] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:29 2022] PUT /identity/v3/projects/f510840390bb432ba9e2d2069a16ae5c/users/094a5e4059654755a29deba1a32e7ecc/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:29.731911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:29.733920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:29.739917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:29.855068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.856539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:29.904911 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2989/6005] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:19:29.975264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6ebc3-4aad-46bc-ae7f-987a8640d581 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:29.989401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47658774-5c15-4652-85fd-b43d856a0a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:29.992612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47658774-5c15-4652-85fd-b43d856a0a26 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:29.992924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47658774-5c15-4652-85fd-b43d856a0a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:29.993139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47658774-5c15-4652-85fd-b43d856a0a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:29.993909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47658774-5c15-4652-85fd-b43d856a0a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:29.995561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47658774-5c15-4652-85fd-b43d856a0a26 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:30.046719 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3017/6006] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:30.051069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:30.082737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8348-bf6c-4c85-965a-14143b88ab85 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:30.097082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:30.098135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:30.098784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:30.099475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] PATH_INFO: `/v3/projects/f510840390bb432ba9e2d2069a16ae5c/users/094a5e4059654755a29deba1a32e7ecc/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:30.103754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f510840390bb432ba9e2d2069a16ae5c, user_id=094a5e4059654755a29deba1a32e7ecc, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:30.106870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:30.138076 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2990/6007] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:30 2022] PUT /identity/v3/projects/f510840390bb432ba9e2d2069a16ae5c/users/094a5e4059654755a29deba1a32e7ecc/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:30.148633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ff41c-7577-4fa6-8115-ebe65582e36a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:30.149246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ff41c-7577-4fa6-8115-ebe65582e36a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:30.149678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ff41c-7577-4fa6-8115-ebe65582e36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:30.249573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ff5ff41c-7577-4fa6-8115-ebe65582e36a None None] MFA Rules not processed for user `094a5e4059654755a29deba1a32e7ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:30.250984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ff5ff41c-7577-4fa6-8115-ebe65582e36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:30.428507 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3018/6008] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:30.441189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:30.443268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:30.600427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d410218-e7a0-4eea-b1ab-b46b4ce51866 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:30.611616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:30.611962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:30.612794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:30.613084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:30.643628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:30.645480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:30.670562 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2991/6009] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:30.839313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ff41c-7577-4fa6-8115-ebe65582e36a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:30.849615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:30.861114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ff41c-7577-4fa6-8115-ebe65582e36a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:30.870904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88114a90-8518-4e48-83f1-549212374d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:30.871540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88114a90-8518-4e48-83f1-549212374d04 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:30.872229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88114a90-8518-4e48-83f1-549212374d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:30.872507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88114a90-8518-4e48-83f1-549212374d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:30.967926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823b97c-ddac-4c04-a91f-88d9d92b8873 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:30.979150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:30.980572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:30.980892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:30.981195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] PATH_INFO: `/v3/users/f5557679aeb941c3920791ae3657c4cd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:30.985485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5557679aeb941c3920791ae3657c4cd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:30.987212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:31.041307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88114a90-8518-4e48-83f1-549212374d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:31.044463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:31.045523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88114a90-8518-4e48-83f1-549212374d04 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:31.074865 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3019/6010] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:31.105486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5557679aeb941c3920791ae3657c4cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.105486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5557679aeb941c3920791ae3657c4cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.132904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5557679aeb941c3920791ae3657c4cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.153984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5557679aeb941c3920791ae3657c4cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.185279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5557679aeb941c3920791ae3657c4cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.203699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2992/6011] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:30 2022] DELETE /identity/v3/users/f5557679aeb941c3920791ae3657c4cd => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:31.206019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:31.206313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:31.206790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:31.241232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:31.245938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:31.287855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:31.495627 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3020/6012] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:31.522131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:31.522131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:31.617244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:31.619214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:31.659152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fae71-99d8-4e12-aba9-8cafa0c42c18 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:31.683661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:31.684249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:31.684531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:31.684830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] PATH_INFO: `/v3/users/7f35330693e947a6a7021290be4c946f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:31.696704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f35330693e947a6a7021290be4c946f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:31.698784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:31.755636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b167f-b697-472c-8145-e0104d5a671f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:31.769757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:31.770401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:31.771180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:31.771180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:31.814198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:31.815775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:31.831320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f35330693e947a6a7021290be4c946f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.838182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f35330693e947a6a7021290be4c946f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.846519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2993/6013] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:31.854998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f35330693e947a6a7021290be4c946f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.866476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f35330693e947a6a7021290be4c946f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.873337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f35330693e947a6a7021290be4c946f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:31.928943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3021/6014] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:31 2022] DELETE /identity/v3/users/7f35330693e947a6a7021290be4c946f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:32.443303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:32.450469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:32.491042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d733c-764b-4768-aa87-794c0c42a051 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:32.688592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:32.689826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:32.690330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:32.691012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] PATH_INFO: `/v3/users/6efd3d39610444b99c7bb245337c2ac3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:32.704245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efd3d39610444b99c7bb245337c2ac3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:32.711108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:32.742589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:32.744360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:32.780284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533c09a-457b-4236-a80a-006b7d9bbcd5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:32.801632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:32.802720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:32.803166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:32.803613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] PATH_INFO: `/v3/projects/884264769e5b4dd8b661de39c466e76a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:32.806165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884264769e5b4dd8b661de39c466e76a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:32.808443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:32.862012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efd3d39610444b99c7bb245337c2ac3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.863250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efd3d39610444b99c7bb245337c2ac3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.880088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:32.884981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efd3d39610444b99c7bb245337c2ac3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.909947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efd3d39610444b99c7bb245337c2ac3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.923894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884264769e5b4dd8b661de39c466e76a', 'request_id': 'req-27cf6d3e-2c9d-421c-af81-3da5ef502c03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.931245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efd3d39610444b99c7bb245337c2ac3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.945434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884264769e5b4dd8b661de39c466e76a', 'request_id': 'req-27cf6d3e-2c9d-421c-af81-3da5ef502c03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.946673 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2994/6015] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:32 2022] DELETE /identity/v3/users/6efd3d39610444b99c7bb245337c2ac3 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:32.958135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884264769e5b4dd8b661de39c466e76a', 'request_id': 'req-27cf6d3e-2c9d-421c-af81-3da5ef502c03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.985621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] The token cache is being invalidate because project 884264769e5b4dd8b661de39c466e76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:32.985922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.996877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:32.999230 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3022/6016] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:32 2022] DELETE /identity/v3/projects/884264769e5b4dd8b661de39c466e76a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:33.003212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:33.004868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:33.139458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0a72-f657-4db2-a6cd-1adadcdcc298 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:33.156502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:33.157463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:33.158439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:33.158824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] PATH_INFO: `/v3/users/3d0ab16486364c59909350229b72f376` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:33.162221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d0ab16486364c59909350229b72f376)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:33.167576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:33.387049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d0ab16486364c59909350229b72f376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:33.387796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d0ab16486364c59909350229b72f376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:33.397660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d0ab16486364c59909350229b72f376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:33.425823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d0ab16486364c59909350229b72f376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:33.507363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d0ab16486364c59909350229b72f376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:33.530344 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2995/6017] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:33 2022] DELETE /identity/v3/users/3d0ab16486364c59909350229b72f376 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:34.430527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:34.432030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:34.477167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:34.563402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:34.563402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:34.576022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf6d3e-2c9d-421c-af81-3da5ef502c03 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:34.725158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:34.763395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99e418-f476-41cf-a401-6e94456e0f8a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:34.768608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:34.768608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:34.768940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:34.769188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] PATH_INFO: `/v3/projects/cd00595fe8dc4d9594513f900172d7ff` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:34.771070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd00595fe8dc4d9594513f900172d7ff)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:34.776355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:34.776624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:34.777267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:34.777540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:34.778019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] PATH_INFO: `/v3/projects/fba836ddf0a14ee0ae089f3ec0db6a3f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:34.779579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba836ddf0a14ee0ae089f3ec0db6a3f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:34.782091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:34.931208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd00595fe8dc4d9594513f900172d7ff', 'request_id': 'req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:34.945166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba836ddf0a14ee0ae089f3ec0db6a3f', 'request_id': 'req-be7909bd-7fb5-44de-a111-cf2ddec527e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:34.950286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd00595fe8dc4d9594513f900172d7ff', 'request_id': 'req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:34.962443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba836ddf0a14ee0ae089f3ec0db6a3f', 'request_id': 'req-be7909bd-7fb5-44de-a111-cf2ddec527e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:34.969408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd00595fe8dc4d9594513f900172d7ff', 'request_id': 'req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:34.990155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba836ddf0a14ee0ae089f3ec0db6a3f', 'request_id': 'req-be7909bd-7fb5-44de-a111-cf2ddec527e0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:34.997791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] The token cache is being invalidate because project cd00595fe8dc4d9594513f900172d7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:34.998170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.000473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.008419 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3023/6018] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:34 2022] DELETE /identity/v3/projects/cd00595fe8dc4d9594513f900172d7ff => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:35.009381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:35.021082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:35.028764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] The token cache is being invalidate because project fba836ddf0a14ee0ae089f3ec0db6a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:35.029151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.031826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-be7909bd-7fb5-44de-a111-cf2ddec527e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.035487 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2996/6019] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:34 2022] DELETE /identity/v3/projects/fba836ddf0a14ee0ae089f3ec0db6a3f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:35.042187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.042513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.042929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:35.084294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:35.085842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:35.244279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:35.254077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cabe7c-82b5-4e2c-adf4-a6ba41fc3d51 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:35.262117 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2997/6020] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:35.265964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:35.269461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:35.270144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.270422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.270698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] PATH_INFO: `/v3/projects/1097aafb312045ddb9210e60b6af5e60` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:35.271006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:35.276874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1097aafb312045ddb9210e60b6af5e60)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:35.279169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:35.313886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1fd41-6b9d-4180-8983-e39acd9d17e1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:35.323370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:35.324187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.324475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.324794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] PATH_INFO: `/v3/users/073008f8c5434d8086490b017a9b98f1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:35.326382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=073008f8c5434d8086490b017a9b98f1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:35.330648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:35.456700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073008f8c5434d8086490b017a9b98f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.464095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073008f8c5434d8086490b017a9b98f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.478748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1097aafb312045ddb9210e60b6af5e60', 'request_id': 'req-2447e474-a3bc-4959-a624-79bb14121727'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.482911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073008f8c5434d8086490b017a9b98f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.505418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073008f8c5434d8086490b017a9b98f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.514379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1097aafb312045ddb9210e60b6af5e60', 'request_id': 'req-2447e474-a3bc-4959-a624-79bb14121727'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.521376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:35.527149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1097aafb312045ddb9210e60b6af5e60', 'request_id': 'req-2447e474-a3bc-4959-a624-79bb14121727'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.533990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8b459214-18f0-4305-b2a6-d68c7e1531fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073008f8c5434d8086490b017a9b98f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.547849 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2998/6021] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:35 2022] DELETE /identity/v3/users/073008f8c5434d8086490b017a9b98f1 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:35.548214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] The token cache is being invalidate because project 1097aafb312045ddb9210e60b6af5e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:35.549435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.549640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73da0e3-9ccd-4131-8f86-e0c64801d023 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.550215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73da0e3-9ccd-4131-8f86-e0c64801d023 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.550983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73da0e3-9ccd-4131-8f86-e0c64801d023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:35.551378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2447e474-a3bc-4959-a624-79bb14121727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:35.559953 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3024/6022] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:35 2022] DELETE /identity/v3/projects/1097aafb312045ddb9210e60b6af5e60 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:35.563759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80307dc-9b20-4811-b387-8ebe247bf957 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.564195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80307dc-9b20-4811-b387-8ebe247bf957 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.564562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80307dc-9b20-4811-b387-8ebe247bf957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:35.620879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c80307dc-9b20-4811-b387-8ebe247bf957 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:35.622259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c80307dc-9b20-4811-b387-8ebe247bf957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:35.622867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d73da0e3-9ccd-4131-8f86-e0c64801d023 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:35.624248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d73da0e3-9ccd-4131-8f86-e0c64801d023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:35.843005 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 2999/6023] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:35.844396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb77a0-1ba5-457c-a19c-af91cc7014a6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.844708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb77a0-1ba5-457c-a19c-af91cc7014a6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.844982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb77a0-1ba5-457c-a19c-af91cc7014a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:35.923174 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3025/6024] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:35.923174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80307dc-9b20-4811-b387-8ebe247bf957 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:35.934258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b0fb77a0-1ba5-457c-a19c-af91cc7014a6 None None] MFA Rules not processed for user `74878b715fc94b23bb139cb30d8d20c0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:35.935296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b0fb77a0-1ba5-457c-a19c-af91cc7014a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:35.957364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80307dc-9b20-4811-b387-8ebe247bf957 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:35.968917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda315c-fc88-4c4e-90cf-052b419be6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:35.970175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda315c-fc88-4c4e-90cf-052b419be6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.970423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda315c-fc88-4c4e-90cf-052b419be6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.971308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda315c-fc88-4c4e-90cf-052b419be6e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:35.971993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda315c-fc88-4c4e-90cf-052b419be6e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:35.972296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda315c-fc88-4c4e-90cf-052b419be6e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:35.974824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda315c-fc88-4c4e-90cf-052b419be6e6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:35.993509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3026/6025] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:35.995117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611638d1-e809-4c68-8a2f-67898bb9f7cc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:35.996491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611638d1-e809-4c68-8a2f-67898bb9f7cc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:35.996760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611638d1-e809-4c68-8a2f-67898bb9f7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.027294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-611638d1-e809-4c68-8a2f-67898bb9f7cc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:36.028747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-611638d1-e809-4c68-8a2f-67898bb9f7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:36.040246 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3027/6026] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:36.041719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611638d1-e809-4c68-8a2f-67898bb9f7cc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.056946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611638d1-e809-4c68-8a2f-67898bb9f7cc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.064700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.064941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.065598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.065598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.066135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.069439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.099765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3000/6027] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:36.100814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb77a0-1ba5-457c-a19c-af91cc7014a6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.127303 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3028/6028] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:36 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:36.128539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.131606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:36.142339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb77a0-1ba5-457c-a19c-af91cc7014a6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.219263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:36.270065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecba7c-617b-411c-82df-abd2e12c0f91 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.277097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.277973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.278516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.279020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.279687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.281008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.281290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.281590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.282632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:36.282938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.284594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.312433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3001/6029] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:36.317820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.344429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea8b8c-c3dd-44c2-8440-35379cdd93a3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.363773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.363995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.364264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.366159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.366898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.367466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.368925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.369652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.381849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3029/6030] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:36.382970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.399860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b30037-30a8-41e4-b5fe-ceade436c455 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.406440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.406716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.408097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.408097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.408753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.411497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.446986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:36.466532 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3030/6031] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:36 2022] POST /identity/v3/projects => generated 389 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:36.469293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.472735 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3002/6032] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:36 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:36.477153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.515197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f520265-7669-49d0-a93a-7605d1e6ed44 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.597978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f5f40-6c43-4f66-9f2e-a5cd03b87064 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.617492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.619600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.620888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.621165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.621831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.624992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.642272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.642931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.643166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.643619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.644456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.647176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.662723 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3003/6033] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:36.666798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.679211 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3031/6034] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:36 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:19:36.680750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.686997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3284824-8e51-4af4-827c-3e242f6a7fd9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.694434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.695357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.695957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.696386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.697914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.701643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.714781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3004/6035] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:36.718935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.721126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb17c1-b67b-426f-9192-df3cbd16f392 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.738088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.739372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd36efa-4a66-445a-abaa-951ce07a9967 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.740052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.742230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.742856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.744839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.749001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.753042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.754653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.757377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.757962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] PATH_INFO: `/v3/projects/a82b2468ace34a009ca80c159ed44bb8/users/e46a8a1c539e4781afda6aadd5373c8e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.785174 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3032/6036] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:36.786310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.792023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:36.807022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82b2468ace34a009ca80c159ed44bb8, user_id=e46a8a1c539e4781afda6aadd5373c8e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.811654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.825280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4f41-5b3b-408f-bb93-99ec534af53c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:36.840477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:36.841530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:36.845676 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3005/6037] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:36 2022] PUT /identity/v3/projects/a82b2468ace34a009ca80c159ed44bb8/users/e46a8a1c539e4781afda6aadd5373c8e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:36.846369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:36.847338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:36.847728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] PATH_INFO: `/v3/projects/81366aa764eb472ea0638bfe9ff90dec` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:36.850543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81366aa764eb472ea0638bfe9ff90dec)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:36.853097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:36.996061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:37.042569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81366aa764eb472ea0638bfe9ff90dec', 'request_id': 'req-ac0d84cf-5b3f-4457-b45c-19d7004bb875'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:37.069970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81366aa764eb472ea0638bfe9ff90dec', 'request_id': 'req-ac0d84cf-5b3f-4457-b45c-19d7004bb875'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:37.081636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81366aa764eb472ea0638bfe9ff90dec', 'request_id': 'req-ac0d84cf-5b3f-4457-b45c-19d7004bb875'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:37.090117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036b8c-40d8-4dcd-a053-4a418e91a7c6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:37.114985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] The token cache is being invalidate because project 81366aa764eb472ea0638bfe9ff90dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:37.116769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:37.117963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac0d84cf-5b3f-4457-b45c-19d7004bb875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:37.123906 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3033/6038] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:36 2022] DELETE /identity/v3/projects/81366aa764eb472ea0638bfe9ff90dec => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:37.123906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.123906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.123906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.187562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] MFA Rules not processed for user `e46a8a1c539e4781afda6aadd5373c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:37.191234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:37.303030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde746-b13c-481e-a6e9-fe02ba77c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:37.304078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde746-b13c-481e-a6e9-fe02ba77c55c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.304532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde746-b13c-481e-a6e9-fe02ba77c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.304986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde746-b13c-481e-a6e9-fe02ba77c55c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.305883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde746-b13c-481e-a6e9-fe02ba77c55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:37.307512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde746-b13c-481e-a6e9-fe02ba77c55c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:37.324300 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3006/6039] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:37.325715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b4866-b150-41ec-949b-c0c1bbdbff30 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.325998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b4866-b150-41ec-949b-c0c1bbdbff30 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.326268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b4866-b150-41ec-949b-c0c1bbdbff30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.337489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:37.358981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-395b4866-b150-41ec-949b-c0c1bbdbff30 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:37.360584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-395b4866-b150-41ec-949b-c0c1bbdbff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:37.362117 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3034/6040] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:37.363167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:37.365667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:37.384043 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3007/6041] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:37.385521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de8806-146b-4b61-a18b-c3fc770642f9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.388327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de8806-146b-4b61-a18b-c3fc770642f9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.388890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de8806-146b-4b61-a18b-c3fc770642f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.390843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00dbf9-1314-47ea-98ad-a4a7b8f32183 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:37.398347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:37.399593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.400192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.400535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] PATH_INFO: `/v3/projects/932da48df2c44fc6a7dbd6c8daf9fc0d/users/64c9f44e57164b858421b55114acf4cf/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.421890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-00de8806-146b-4b61-a18b-c3fc770642f9 None None] MFA Rules not processed for user `e46a8a1c539e4781afda6aadd5373c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:37.424407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-00de8806-146b-4b61-a18b-c3fc770642f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:37.435928 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3008/6042] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:37.439137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=932da48df2c44fc6a7dbd6c8daf9fc0d, user_id=64c9f44e57164b858421b55114acf4cf, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:37.442831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:37.443055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccb21c-e547-4702-91ea-aca4867e7165 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.443354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccb21c-e547-4702-91ea-aca4867e7165 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.443725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccb21c-e547-4702-91ea-aca4867e7165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.463740 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3035/6043] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:37 2022] PUT /identity/v3/projects/932da48df2c44fc6a7dbd6c8daf9fc0d/users/64c9f44e57164b858421b55114acf4cf/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:37.465213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:37.467467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:37.478359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-42ccb21c-e547-4702-91ea-aca4867e7165 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:37.479663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-42ccb21c-e547-4702-91ea-aca4867e7165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:37.584752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81c27a-d90a-415a-95f5-11d28978b56c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:37.593365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:37.594408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.594836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.595286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.616310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:37.619480 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3009/6044] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:37.623938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67186ff-1bd1-44b2-943c-c1ae47d07931 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.624230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67186ff-1bd1-44b2-943c-c1ae47d07931 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.624512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67186ff-1bd1-44b2-943c-c1ae47d07931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.703221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f67186ff-1bd1-44b2-943c-c1ae47d07931 None None] MFA Rules not processed for user `64c9f44e57164b858421b55114acf4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:37.707359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f67186ff-1bd1-44b2-943c-c1ae47d07931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:37.716983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:37.719125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:37.732975 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3036/6045] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:37.733957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:37.747635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8065f-335b-4053-9ccc-b05205c3080b service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:37.753468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:37.754196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.754545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.754890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.755726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:37.756110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:37.758016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:37.786699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3037/6046] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:37.787836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:37.812073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ce6e6-fae6-4a4a-a6f7-b38b8c65bec2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:37.818650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:37.819391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:37.819673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:37.819947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:37.820593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:37.823512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:37.880392 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3010/6047] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:37.886000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67186ff-1bd1-44b2-943c-c1ae47d07931 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:37.899826 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3038/6048] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:37.900843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:37.915693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67186ff-1bd1-44b2-943c-c1ae47d07931 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.113137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b9d22-5576-45e0-b38c-df223fec7e41 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.131411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.133902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.134179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.134453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.136756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.143139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.157121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.157757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.158078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.158353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.159060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.160946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.247494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3039/6049] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/users => generated 300 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:38.256974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:38.284386 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3011/6050] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:37 2022] POST /identity/v3/projects => generated 389 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:38.285225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:38.307689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:38.308339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55829981-f05e-4126-95ed-0da313760413 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.510435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.511044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.511322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.511596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.512244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.513896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.520078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5db50-fcf9-4008-a5e1-b27f13fc65d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.533237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.534318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.537653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.538988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.544314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3040/6051] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:38.545296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:38.546535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.556061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.570364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5700a88-6446-4579-8634-e4adb6589eb7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.588643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.590565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.591074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.591519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] PATH_INFO: `/v3/projects/1fa62bf8f3e3443692e1ecf8a4ad11ff/users/0910526b1bc14daea492c9d052bb3c6a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.595397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:38.612329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa62bf8f3e3443692e1ecf8a4ad11ff, user_id=0910526b1bc14daea492c9d052bb3c6a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.616314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.648298 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3012/6052] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:38 2022] POST /identity/v3/users => generated 295 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:38.659349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:38.676315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3041/6053] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:38 2022] PUT /identity/v3/projects/1fa62bf8f3e3443692e1ecf8a4ad11ff/users/0910526b1bc14daea492c9d052bb3c6a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:38.680476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:38.726395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1077df8-3906-4b52-bd4d-0b276b189cd2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.801747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa7fb8-947a-4859-88d6-5074043c4a69 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.808278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.809182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.810021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.810382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.811185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.813661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.843990 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3042/6054] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:38.845768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:38.868391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180abcd-fdcd-41aa-be8d-ae1c62149b4a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:38.875945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f658e7-6d74-46bb-82f4-7a29e2952e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.880437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f658e7-6d74-46bb-82f4-7a29e2952e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.880751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f658e7-6d74-46bb-82f4-7a29e2952e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.883255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f658e7-6d74-46bb-82f4-7a29e2952e80 admin admin] PATH_INFO: `/v3/projects/1fa62bf8f3e3443692e1ecf8a4ad11ff/users/0910526b1bc14daea492c9d052bb3c6a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.887021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f658e7-6d74-46bb-82f4-7a29e2952e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa62bf8f3e3443692e1ecf8a4ad11ff, user_id=0910526b1bc14daea492c9d052bb3c6a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.892917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f658e7-6d74-46bb-82f4-7a29e2952e80 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.901503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:38.912699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.912699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.912699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.912699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:38.912699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:38.914402 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3043/6055] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:38 2022] PUT /identity/v3/projects/1fa62bf8f3e3443692e1ecf8a4ad11ff/users/0910526b1bc14daea492c9d052bb3c6a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:38.938502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96f433-e6bb-4bd9-96a1-149277319c05 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:38.938772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96f433-e6bb-4bd9-96a1-149277319c05 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:38.939129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96f433-e6bb-4bd9-96a1-149277319c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:38.940398 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3013/6056] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:38.945591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:38.968824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5c371-3940-4215-8b9e-617d7d077a39 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.042106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0f96f433-e6bb-4bd9-96a1-149277319c05 None None] MFA Rules not processed for user `0910526b1bc14daea492c9d052bb3c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:39.046704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0f96f433-e6bb-4bd9-96a1-149277319c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:39.249990 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3044/6057] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:39.251044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96f433-e6bb-4bd9-96a1-149277319c05 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:39.283124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.286452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.287859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.292182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.293475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:39.303816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:39.356432 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3014/6058] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:39.363934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:39.399873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:39.414288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e187db-0323-4f76-b851-f3e7750091a5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.432671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e021b2b-13b0-4bb7-88a1-9263612953da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.433462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e021b2b-13b0-4bb7-88a1-9263612953da admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.433741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e021b2b-13b0-4bb7-88a1-9263612953da admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.434016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e021b2b-13b0-4bb7-88a1-9263612953da admin admin] PATH_INFO: `/v3/projects/430fffd9dbbb49ab90d73cafa9b64fe3/users/f0f8930eeda742ea9f3483e3314f1a88/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.434331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96f433-e6bb-4bd9-96a1-149277319c05 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.450142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.451050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.451695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.452459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.454507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:39.461222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:39.488961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e021b2b-13b0-4bb7-88a1-9263612953da admin admin] RBAC: Authorizing `identity:create_grant(project_id=430fffd9dbbb49ab90d73cafa9b64fe3, user_id=f0f8930eeda742ea9f3483e3314f1a88, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:39.497029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e021b2b-13b0-4bb7-88a1-9263612953da admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:39.525242 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3015/6059] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:39 2022] PUT /identity/v3/projects/430fffd9dbbb49ab90d73cafa9b64fe3/users/f0f8930eeda742ea9f3483e3314f1a88/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:19:39.539600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44926ca-117b-483e-b0f5-570fad450a03 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.540308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44926ca-117b-483e-b0f5-570fad450a03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.543489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44926ca-117b-483e-b0f5-570fad450a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.547184 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3045/6060] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:39 2022] POST /identity/v3/projects => generated 387 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:39.551866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:39.564099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:39.622695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f44926ca-117b-483e-b0f5-570fad450a03 None None] MFA Rules not processed for user `f0f8930eeda742ea9f3483e3314f1a88`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:39.624157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f44926ca-117b-483e-b0f5-570fad450a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:39.696633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405fa99-6e38-4798-8ad3-87400abf471e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.703148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.704239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.704583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.705579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.706346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:39.708865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:39.750969 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3046/6061] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:39 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:19:39.755027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:39.775547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412a494-3539-4a6c-8117-57ec5872a678 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.788209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.788942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.789521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.789735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.791006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:39.793721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:39.809083 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3016/6062] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:39.810787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44926ca-117b-483e-b0f5-570fad450a03 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:39.824353 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3047/6063] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:39.825321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:39.847044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44926ca-117b-483e-b0f5-570fad450a03 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.853866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5b463-115f-4ccf-8617-f345df224a80 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.854033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.855526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.855800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.856150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.856835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:39.859265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:39.859360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.859992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.862244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.862536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.863194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:39.864888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:39.903748 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3048/6064] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:39.924137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:39.949822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8846f7-3761-42cf-9e53-dcc154cabf91 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:39.970352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619fbb1-67e2-4470-83e6-b391b8f24dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:39.978135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fbb1-67e2-4470-83e6-b391b8f24dea admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:39.978684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fbb1-67e2-4470-83e6-b391b8f24dea admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:39.978966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fbb1-67e2-4470-83e6-b391b8f24dea admin admin] PATH_INFO: `/v3/projects/f0f67ba3b16246f7a1ced6806a022c48/users/472418bd0aab47028a5dc6240e7c4f3b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:39.983489 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3017/6065] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:39 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:39.988179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:40.037237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619fbb1-67e2-4470-83e6-b391b8f24dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f67ba3b16246f7a1ced6806a022c48, user_id=472418bd0aab47028a5dc6240e7c4f3b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:40.044835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619fbb1-67e2-4470-83e6-b391b8f24dea admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:40.082508 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3049/6066] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:39 2022] PUT /identity/v3/projects/f0f67ba3b16246f7a1ced6806a022c48/users/472418bd0aab47028a5dc6240e7c4f3b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:40.085542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:40.085831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:40.086138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:40.163622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] MFA Rules not processed for user `472418bd0aab47028a5dc6240e7c4f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:40.167164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:40.182447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d464cf3-3b64-4fa7-9887-c344cd3b4f9e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:40.344353 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3050/6067] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:40.367955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:40.370202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:40.385277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:40.385918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:40.389969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:40.389969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:40.394252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:40.402691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:40.413295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:40.479670 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3018/6068] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:39 2022] POST /identity/v3/users => generated 300 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:40.481841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:40.512491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e9b7f-8dd6-451d-8ee7-ad3fcfbec9f6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:40.544581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:40.546645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:40.549758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:40.550086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:40.555418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:40.560466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:40.570054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:40.609309 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3019/6069] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:40.616436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6594e-dd0b-4bc3-a0a8-cba33dda321e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:40.617275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:40.632399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:40.633082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:40.636333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:40.636745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:40.649215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d8a29-ae58-4dff-818e-3ffe23a101c1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:40.654974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:40.657018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:40.657424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:40.657779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] PATH_INFO: `/v3/projects/e7b9670ac0de4e028fe2eb0d7636090c/users/7499cab28aae45eabab969fcaf60cea3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:40.697155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b9670ac0de4e028fe2eb0d7636090c, user_id=7499cab28aae45eabab969fcaf60cea3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:40.710793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:40.736402 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3020/6070] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:40 2022] PUT /identity/v3/projects/e7b9670ac0de4e028fe2eb0d7636090c/users/7499cab28aae45eabab969fcaf60cea3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:40.737542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:40.797516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:40.800638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:40.832244 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3051/6071] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:19:40.877424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfadce-01de-4c94-b57b-ec63841eefa4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:40.898195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92909ede-300a-4f6d-a501-e3214ed960ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:40.898897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909ede-300a-4f6d-a501-e3214ed960ad admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:40.899240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909ede-300a-4f6d-a501-e3214ed960ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:40.899541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909ede-300a-4f6d-a501-e3214ed960ad admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:40.900494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909ede-300a-4f6d-a501-e3214ed960ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:40.902205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909ede-300a-4f6d-a501-e3214ed960ad admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:40.937737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:40.938653 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3021/6072] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:40.955895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e4ac-3a1d-4fe6-b745-52eb7700fe5f service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:40.969349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:40.970041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:40.970314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:40.970584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] PATH_INFO: `/v3/projects/e7b9670ac0de4e028fe2eb0d7636090c/users/7499cab28aae45eabab969fcaf60cea3/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:40.977604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b9670ac0de4e028fe2eb0d7636090c, user_id=7499cab28aae45eabab969fcaf60cea3, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:40.981317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:41.010043 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3052/6073] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:40 2022] PUT /identity/v3/projects/e7b9670ac0de4e028fe2eb0d7636090c/users/7499cab28aae45eabab969fcaf60cea3/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:41.025720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1207731-91b4-48a9-86f3-c70fbbd003d9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:41.026102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1207731-91b4-48a9-86f3-c70fbbd003d9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:41.026448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1207731-91b4-48a9-86f3-c70fbbd003d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:41.093352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f1207731-91b4-48a9-86f3-c70fbbd003d9 None None] MFA Rules not processed for user `7499cab28aae45eabab969fcaf60cea3`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:41.097129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f1207731-91b4-48a9-86f3-c70fbbd003d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:41.338934 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3022/6074] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:41.357289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:41.521048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7d833-ad3d-475c-89f8-44cfa79b1bd4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:41.529280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ddb27-b593-4b27-bce5-6c409ff4852f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:41.531198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ddb27-b593-4b27-bce5-6c409ff4852f service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:41.531792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ddb27-b593-4b27-bce5-6c409ff4852f service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:41.536235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ddb27-b593-4b27-bce5-6c409ff4852f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:41.666640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:41.701610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ddb27-b593-4b27-bce5-6c409ff4852f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:41.703321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ddb27-b593-4b27-bce5-6c409ff4852f service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:41.722445 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3053/6075] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:42.102438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1207731-91b4-48a9-86f3-c70fbbd003d9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:42.198700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:42.216505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1207731-91b4-48a9-86f3-c70fbbd003d9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:42.228207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:42.228690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:42.229177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:42.229531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] PATH_INFO: `/v3/users/0910526b1bc14daea492c9d052bb3c6a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:42.232427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0910526b1bc14daea492c9d052bb3c6a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:42.235755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:42.353989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0910526b1bc14daea492c9d052bb3c6a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:42.357047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0910526b1bc14daea492c9d052bb3c6a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:42.375438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0910526b1bc14daea492c9d052bb3c6a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:42.400169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0910526b1bc14daea492c9d052bb3c6a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:42.430802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0910526b1bc14daea492c9d052bb3c6a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:42.442022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3023/6076] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:42 2022] DELETE /identity/v3/users/0910526b1bc14daea492c9d052bb3c6a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:42.448242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:42.448806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:42.449012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:42.473094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:42.474434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:42.555993 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3054/6077] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:42.581393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:42.583304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:42.723875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ea2a-8244-46af-a789-b9cee6f9be15 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:42.735074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29af95-6e9d-4720-b651-f96e5330d5af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:42.735731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29af95-6e9d-4720-b651-f96e5330d5af service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:42.736017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29af95-6e9d-4720-b651-f96e5330d5af service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:42.736261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29af95-6e9d-4720-b651-f96e5330d5af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:42.757953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29af95-6e9d-4720-b651-f96e5330d5af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:42.759439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29af95-6e9d-4720-b651-f96e5330d5af service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:42.793108 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3024/6078] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:19:43.527233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:43.529325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:43.558793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1244c1f-8983-46da-a2fc-27d60bf02b69 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:43.574475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:43.575424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:43.575805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:43.577157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] PATH_INFO: `/v3/users/3d42fa166bb14724a1ddffed4a6c6231` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:43.583029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d42fa166bb14724a1ddffed4a6c6231)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:43.589170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:43.703276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d42fa166bb14724a1ddffed4a6c6231'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:43.704346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d42fa166bb14724a1ddffed4a6c6231'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:43.733426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d42fa166bb14724a1ddffed4a6c6231'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:43.749032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d42fa166bb14724a1ddffed4a6c6231'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:43.749817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e5c7502-a30c-4887-9205-a9fc40d485da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d42fa166bb14724a1ddffed4a6c6231'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:43.800756 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3055/6079] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:43 2022] DELETE /identity/v3/users/3d42fa166bb14724a1ddffed4a6c6231 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:43.805188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:43.805721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:43.805721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:43.845016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:43.851216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:44.009465 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3025/6080] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:44.064466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:44.064811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:44.065136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:44.079258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:44.081856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:44.100685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] MFA Rules not processed for user `74878b715fc94b23bb139cb30d8d20c0`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:44.101960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:44.141564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:44.235859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:44.252877 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3056/6081] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:44.254546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483cf58-eac3-4b38-8bd4-a9631e723079 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:44.263860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:44.265569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:44.268138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:44.268976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:44.296863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:44.300567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:44.319613 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3026/6082] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:19:45.249666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:45.251289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:45.282379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0048c6-17db-476d-b188-8ffcd475915e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:45.308397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:45.311063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:45.311591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:45.312156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] PATH_INFO: `/v3/projects/0ff624ba07b54ec985a94783d529cdef` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:45.315457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff624ba07b54ec985a94783d529cdef)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:45.318226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:45.422100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff624ba07b54ec985a94783d529cdef', 'request_id': 'req-0f5144b6-d419-4a6b-b9de-f01a185e5619'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.443900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff624ba07b54ec985a94783d529cdef', 'request_id': 'req-0f5144b6-d419-4a6b-b9de-f01a185e5619'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.453243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:45.455659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:45.459750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff624ba07b54ec985a94783d529cdef', 'request_id': 'req-0f5144b6-d419-4a6b-b9de-f01a185e5619'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.477655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] The token cache is being invalidate because project 0ff624ba07b54ec985a94783d529cdef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:45.477872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.478523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.480887 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3057/6083] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:45 2022] DELETE /identity/v3/projects/0ff624ba07b54ec985a94783d529cdef => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:45.484841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb2795-4c4e-478c-a585-10ae7eca9eeb service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:45.485318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:45.494526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:45.627160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5144b6-d419-4a6b-b9de-f01a185e5619 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:45.642762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:45.643652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:45.643999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:45.644337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] PATH_INFO: `/v3/users/094a5e4059654755a29deba1a32e7ecc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:45.645649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=094a5e4059654755a29deba1a32e7ecc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:45.648305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:45.655487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:45.660376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:45.660932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:45.661324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] PATH_INFO: `/v3/projects/1fa62bf8f3e3443692e1ecf8a4ad11ff` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:45.668052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa62bf8f3e3443692e1ecf8a4ad11ff)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:45.673458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:45.801426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa62bf8f3e3443692e1ecf8a4ad11ff', 'request_id': 'req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.808848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '094a5e4059654755a29deba1a32e7ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.810262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '094a5e4059654755a29deba1a32e7ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.818066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa62bf8f3e3443692e1ecf8a4ad11ff', 'request_id': 'req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.826485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '094a5e4059654755a29deba1a32e7ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.844470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa62bf8f3e3443692e1ecf8a4ad11ff', 'request_id': 'req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.860260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '094a5e4059654755a29deba1a32e7ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.862613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '094a5e4059654755a29deba1a32e7ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.867577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] The token cache is being invalidate because project 1fa62bf8f3e3443692e1ecf8a4ad11ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:45.867874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.870502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:45.874059 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3027/6084] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:45 2022] DELETE /identity/v3/projects/1fa62bf8f3e3443692e1ecf8a4ad11ff => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:45.877869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:45.881810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:45.898556 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3058/6085] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:45 2022] DELETE /identity/v3/users/094a5e4059654755a29deba1a32e7ecc => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:45.959131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:45.997646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269a29-902c-4948-a9f7-ceb7cf6b4c20 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:46.092165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:46.092871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:46.093157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:46.093437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] PATH_INFO: `/v3/users/472418bd0aab47028a5dc6240e7c4f3b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:46.094692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] RBAC: Authorizing `identity:delete_user(user_id=472418bd0aab47028a5dc6240e7c4f3b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:46.097394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:46.241693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '472418bd0aab47028a5dc6240e7c4f3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:46.244579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '472418bd0aab47028a5dc6240e7c4f3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:46.267895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '472418bd0aab47028a5dc6240e7c4f3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:46.292048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '472418bd0aab47028a5dc6240e7c4f3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:46.317672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7dce441f-01b2-4909-a66f-c183dab0e89c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '472418bd0aab47028a5dc6240e7c4f3b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:46.329800 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3028/6086] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:45 2022] DELETE /identity/v3/users/472418bd0aab47028a5dc6240e7c4f3b => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:47.321859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:47.323545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:47.340240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:47.348048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286fb8b-b67c-4954-bf8e-0cc4dbf0aebf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:47.437800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:47.438473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:47.438919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:47.439198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] PATH_INFO: `/v3/projects/f510840390bb432ba9e2d2069a16ae5c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:47.441818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f510840390bb432ba9e2d2069a16ae5c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:47.444682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:47.545510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f510840390bb432ba9e2d2069a16ae5c', 'request_id': 'req-a6e2473e-20ae-45e1-9b9a-084bb7931706'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:47.558840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f510840390bb432ba9e2d2069a16ae5c', 'request_id': 'req-a6e2473e-20ae-45e1-9b9a-084bb7931706'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:47.567066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f510840390bb432ba9e2d2069a16ae5c', 'request_id': 'req-a6e2473e-20ae-45e1-9b9a-084bb7931706'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:47.582568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] The token cache is being invalidate because project f510840390bb432ba9e2d2069a16ae5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:47.583299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:47.584875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a6e2473e-20ae-45e1-9b9a-084bb7931706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:47.587820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3059/6087] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:47 2022] DELETE /identity/v3/projects/f510840390bb432ba9e2d2069a16ae5c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:47.592918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e236-9b45-411a-b03d-c3893b65d841 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:47.594022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e236-9b45-411a-b03d-c3893b65d841 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:47.594604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e236-9b45-411a-b03d-c3893b65d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:47.642707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e8b7e236-9b45-411a-b03d-c3893b65d841 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:47.648387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e8b7e236-9b45-411a-b03d-c3893b65d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:47.795084 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3029/6088] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:47.804183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232300a6-c9af-4d92-9e2e-b2da7ba8b483 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:47.805540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232300a6-c9af-4d92-9e2e-b2da7ba8b483 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:47.806070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232300a6-c9af-4d92-9e2e-b2da7ba8b483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:47.867088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-232300a6-c9af-4d92-9e2e-b2da7ba8b483 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:47.871489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-232300a6-c9af-4d92-9e2e-b2da7ba8b483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:47.886755 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3060/6089] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:47.892368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7e236-9b45-411a-b03d-c3893b65d841 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:47.906012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7e236-9b45-411a-b03d-c3893b65d841 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:47.921716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:47.922374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:47.922647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:47.922914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:47.924509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:47.924915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:47.926805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:47.947122 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3030/6090] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:47.951717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232300a6-c9af-4d92-9e2e-b2da7ba8b483 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:47.962661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232300a6-c9af-4d92-9e2e-b2da7ba8b483 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:47.979513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:47.980129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:47.984182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:47.984503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:47.985223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:47.987313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.053378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3061/6091] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:47 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:48.057541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.063095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:48.070178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2736f7b-b018-4dff-b07e-5b9169184f1b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.091798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.093796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:48.230520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.232869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.233154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.233429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.234205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.240180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.256277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2445-fb2e-495d-b2fe-1c6adcb01bf5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.264293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.265072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.265323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.265749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] PATH_INFO: `/v3/projects/f0f67ba3b16246f7a1ced6806a022c48` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.269595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f67ba3b16246f7a1ced6806a022c48)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.287241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.296703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:48.305323 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3031/6092] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:48.317018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.339940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7c2b-307d-4eb7-b81b-cfc0e1218f89 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.349275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.350302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.350471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.351319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.352117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.353560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.372316 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3032/6093] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:48.376268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.376422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f67ba3b16246f7a1ced6806a022c48', 'request_id': 'req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:48.389996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f67ba3b16246f7a1ced6806a022c48', 'request_id': 'req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:48.396857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f67ba3b16246f7a1ced6806a022c48', 'request_id': 'req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:48.405226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee532c-6227-451c-8b9e-1ff6fa419fc7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.414228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] The token cache is being invalidate because project f0f67ba3b16246f7a1ced6806a022c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:48.414785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:48.415655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-48b7ccfe-b0a1-4ef7-a44d-4dc1c4982f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:48.418210 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3062/6094] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:48 2022] DELETE /identity/v3/projects/f0f67ba3b16246f7a1ced6806a022c48 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:48.423308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc3b1d-f094-4bc9-b7b4-8f16ff9ed0a4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.424316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc3b1d-f094-4bc9-b7b4-8f16ff9ed0a4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.424777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc3b1d-f094-4bc9-b7b4-8f16ff9ed0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.464362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cacc3b1d-f094-4bc9-b7b4-8f16ff9ed0a4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:48.465609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cacc3b1d-f094-4bc9-b7b4-8f16ff9ed0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:48.553261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.555621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.555910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.556263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.557642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.559786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.577072 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3033/6095] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:48.578017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.580141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:48.593986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddc97b-83e2-49cf-9031-f8db19bedb26 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.606938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bbfff-5681-4cab-ba6e-cb0eaf6e3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.607917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bbfff-5681-4cab-ba6e-cb0eaf6e3e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.608355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bbfff-5681-4cab-ba6e-cb0eaf6e3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.608813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bbfff-5681-4cab-ba6e-cb0eaf6e3e54 admin admin] PATH_INFO: `/v3/projects/1d576d7a3ba44e14ba171a3c8ee6a18a/users/46a22f58246647e9a990b3865fa5c4c1/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.627233 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3063/6096] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:48.633794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bbfff-5681-4cab-ba6e-cb0eaf6e3e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d576d7a3ba44e14ba171a3c8ee6a18a, user_id=46a22f58246647e9a990b3865fa5c4c1, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.635117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6279de-9f00-4c26-8286-9a9e8cf3b3dc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.635678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6279de-9f00-4c26-8286-9a9e8cf3b3dc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.636050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6279de-9f00-4c26-8286-9a9e8cf3b3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.636583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bbfff-5681-4cab-ba6e-cb0eaf6e3e54 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.670114 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3034/6097] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:48 2022] PUT /identity/v3/projects/1d576d7a3ba44e14ba171a3c8ee6a18a/users/46a22f58246647e9a990b3865fa5c4c1/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:48.671941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5b2b-26f1-4dcc-b57c-2435e3258e3b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.672125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5b2b-26f1-4dcc-b57c-2435e3258e3b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.672400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5b2b-26f1-4dcc-b57c-2435e3258e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.675364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dc6279de-9f00-4c26-8286-9a9e8cf3b3dc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:48.677293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dc6279de-9f00-4c26-8286-9a9e8cf3b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:48.713866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a2fe5b2b-26f1-4dcc-b57c-2435e3258e3b None None] MFA Rules not processed for user `46a22f58246647e9a990b3865fa5c4c1`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:48.715237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a2fe5b2b-26f1-4dcc-b57c-2435e3258e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:48.773864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:48.775527 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3064/6098] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:19:48.779701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6279de-9f00-4c26-8286-9a9e8cf3b3dc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.803088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6279de-9f00-4c26-8286-9a9e8cf3b3dc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.809032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.809592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.809870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.810146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.810800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:48.811104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.812752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.829321 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3035/6099] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:48.830540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5b2b-26f1-4dcc-b57c-2435e3258e3b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.844937 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3065/6100] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:19:48.845233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5b2b-26f1-4dcc-b57c-2435e3258e3b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.848973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.863784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a8931-0f22-4092-83a7-62291d44f465 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:48.865007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.866969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.867363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.867742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.868497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.870561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.876741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:48.877984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:48.878458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:48.878984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:48.879826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:48.882085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:48.923849 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3036/6101] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:48.925068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:48.935788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:48.953068 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3066/6102] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:48.955372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.051745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3ade9-33c7-46d1-b0af-6d317d042adc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.124109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae15c7-feda-4035-9615-fc6d101a2388 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.132608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.133304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.134309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.134589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.135236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.138754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.178127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.179686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.180017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.180300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.182387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.186102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.211356 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3067/6103] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:49.216009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.223055 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3037/6104] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:48 2022] POST /identity/v3/users => generated 294 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:19:49.224144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.238874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d5bce-3167-4958-a0b8-e05502534c94 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.247126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.247763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.248044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.248371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.249268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.251461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.261249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35788248-1771-45b0-a133-67467f30ca1d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.271992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3038/6105] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:49.273065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.281888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.282551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.286726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.287033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.287727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.289590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8fd9f-b071-4d6c-9281-5ed2e725362f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.294400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.296544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.297702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.298091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.298471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] PATH_INFO: `/v3/projects/cdb214195364462d817e8cabeac1f866/users/1dc61d03273242b0987a6f10b59849b2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.326015 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3068/6106] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:49.337044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.358105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb214195364462d817e8cabeac1f866, user_id=1dc61d03273242b0987a6f10b59849b2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.371696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.377359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f0c22-7ee4-4ed0-a9a9-a4195c765fc9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.397626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.401033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.402144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.402429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.403156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.413264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.420162 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3039/6107] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:49 2022] PUT /identity/v3/projects/cdb214195364462d817e8cabeac1f866/users/1dc61d03273242b0987a6f10b59849b2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:49.420976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.452768 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3069/6108] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:49.465782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.473249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:49.504596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fda20f-3725-4ced-86a7-8d5bd3a51f6c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.605893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561869d-bb1c-4637-83ac-d534c87d814d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.622216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0052ab1-4f54-4639-a972-822fea13239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.623052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0052ab1-4f54-4639-a972-822fea13239a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.625582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0052ab1-4f54-4639-a972-822fea13239a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.625988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0052ab1-4f54-4639-a972-822fea13239a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.626775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0052ab1-4f54-4639-a972-822fea13239a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.628504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0052ab1-4f54-4639-a972-822fea13239a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.652873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.653888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.654386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.654823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] PATH_INFO: `/v3/projects/28c44d86cc114acaa440a74785de4ae6/users/5739c1d476a14938a3915fa4bf6607b8/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.660006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3040/6109] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:19:49.661040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8822c-f443-4c05-b468-ddb06012e90b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.662772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8822c-f443-4c05-b468-ddb06012e90b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.663156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8822c-f443-4c05-b468-ddb06012e90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.677834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c44d86cc114acaa440a74785de4ae6, user_id=5739c1d476a14938a3915fa4bf6607b8, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.681458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.708548 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3070/6110] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:49 2022] PUT /identity/v3/projects/28c44d86cc114acaa440a74785de4ae6/users/5739c1d476a14938a3915fa4bf6607b8/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:49.710694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:49.731425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7ea8822c-f443-4c05-b468-ddb06012e90b None None] MFA Rules not processed for user `377e47048a63456793dfc9c125f5a209`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:49.735767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7ea8822c-f443-4c05-b468-ddb06012e90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:49.911170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:49.941806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c6be-8210-4150-b823-3233e9e5ce4a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:49.953386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057392d7-3d5c-4f5c-972a-d98f58eaf57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:49.954488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057392d7-3d5c-4f5c-972a-d98f58eaf57c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.954771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057392d7-3d5c-4f5c-972a-d98f58eaf57c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.955048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057392d7-3d5c-4f5c-972a-d98f58eaf57c admin admin] PATH_INFO: `/v3/projects/cdb214195364462d817e8cabeac1f866/users/1dc61d03273242b0987a6f10b59849b2/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:49.962202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057392d7-3d5c-4f5c-972a-d98f58eaf57c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb214195364462d817e8cabeac1f866, user_id=1dc61d03273242b0987a6f10b59849b2, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:49.972070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3041/6111] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:49.972302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057392d7-3d5c-4f5c-972a-d98f58eaf57c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:49.982872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:49.983291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:49.984252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:50.021584 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3071/6112] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:19:49 2022] PUT /identity/v3/projects/cdb214195364462d817e8cabeac1f866/users/1dc61d03273242b0987a6f10b59849b2/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:50.026373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d373b9-79f6-4380-b2ac-63e6b69e348f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:50.026373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d373b9-79f6-4380-b2ac-63e6b69e348f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:50.026669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d373b9-79f6-4380-b2ac-63e6b69e348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:50.074070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] MFA Rules not processed for user `5739c1d476a14938a3915fa4bf6607b8`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:50.080235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.080478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-55d373b9-79f6-4380-b2ac-63e6b69e348f None None] MFA Rules not processed for user `377e47048a63456793dfc9c125f5a209`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:50.081298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-55d373b9-79f6-4380-b2ac-63e6b69e348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.197373 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3072/6113] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:50.199603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:50.200101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:50.200599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:50.209909 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3042/6114] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:50.211005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:50.217631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.252111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] MFA Rules not processed for user `1dc61d03273242b0987a6f10b59849b2`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:50.255601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.325251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:50.388430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7c719-8755-44ed-b053-f51c47080bf7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:50.388925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3073/6115] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:50.388925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:50.388925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.400041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:50.400825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:50.401134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:50.401379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:50.410914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:50.412602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:50.422847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3043/6116] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:50.423837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:50.425349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.528702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c94cde-16a4-4508-8bc3-bfd5fc0f7b8b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:50.537345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:50.538205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:50.538853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:50.539200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:50.562405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:50.564634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:50.585784 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3074/6117] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:50.600640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:50.605840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.614313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575dca-634f-408d-b352-c0fd23583153 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:50.632082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4e79a-0644-4af7-b5aa-9965b36bb3d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:50.637715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4e79a-0644-4af7-b5aa-9965b36bb3d1 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:50.639463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4e79a-0644-4af7-b5aa-9965b36bb3d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:50.640757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4e79a-0644-4af7-b5aa-9965b36bb3d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:50.684340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:50.770808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4e79a-0644-4af7-b5aa-9965b36bb3d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:50.772783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4e79a-0644-4af7-b5aa-9965b36bb3d1 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:50.798015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3044/6118] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:50.845027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63037a-89e1-4eab-beca-ff0fad490e25 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:50.859658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:50.860292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:50.861206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:50.861607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:50.865009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:50.995789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8092591-6a46-4760-86c0-181624679294 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:50.996196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8092591-6a46-4760-86c0-181624679294 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:50.996987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8092591-6a46-4760-86c0-181624679294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:51.086927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:51.096781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:51.131787 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3075/6119] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:51.168082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e8092591-6a46-4760-86c0-181624679294 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:51.170347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e8092591-6a46-4760-86c0-181624679294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:51.289990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:51.344895 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3045/6120] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:19:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:51.433323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:51.435487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:51.770767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed63e-105d-48e1-9f9d-266ebf97095a service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:51.807941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:51.808594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:51.808916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:51.809192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:51.835310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:51.838132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:51.869560 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3076/6121] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:19:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:19:52.699347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be40d-d15e-4153-bf1a-2dbd4cb06a26 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:52.699855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be40d-d15e-4153-bf1a-2dbd4cb06a26 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:52.699946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be40d-d15e-4153-bf1a-2dbd4cb06a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:52.779803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5b7be40d-d15e-4153-bf1a-2dbd4cb06a26 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:52.788886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5b7be40d-d15e-4153-bf1a-2dbd4cb06a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:52.830624 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3046/6122] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:19:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:52.849046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:52.866779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e2acc-b241-48a7-9564-51563e991e68 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:52.881060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:52.881691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:52.881967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:52.882240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:52.901517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:52.903143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:52.923121 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3077/6123] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:19:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:53.771867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f3639-774c-4213-96c4-4f4be034a854 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:53.773021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f3639-774c-4213-96c4-4f4be034a854 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:53.773471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f3639-774c-4213-96c4-4f4be034a854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:53.818195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5d4f3639-774c-4213-96c4-4f4be034a854 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:53.819539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5d4f3639-774c-4213-96c4-4f4be034a854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:53.827631 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3047/6124] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:19:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:53.841467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:53.857260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c316d-7c0d-47f8-87e0-49010067578b service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:53.865464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:53.866427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:53.866912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:53.867410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:53.906559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:53.908315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:53.920468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3078/6125] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:19:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:54.264417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad19b2d-69b0-44dc-8efe-a2f94ceeb249 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:54.265288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad19b2d-69b0-44dc-8efe-a2f94ceeb249 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:54.266105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad19b2d-69b0-44dc-8efe-a2f94ceeb249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:54.312695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cad19b2d-69b0-44dc-8efe-a2f94ceeb249 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:54.316752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cad19b2d-69b0-44dc-8efe-a2f94ceeb249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:54.328998 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3048/6126] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:19:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:54.360854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:54.378414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14c0ca-233f-48b0-a0f5-3036b73f38be service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:54.402230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:54.402889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:54.404995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:54.405283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:54.440511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:54.442365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:54.448986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:54.462803 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3079/6127] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:19:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:19:55.122785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:55.123346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:55.123346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:55.172367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:55.173653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:55.185194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3049/6128] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:19:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:55.211177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:55.231778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e93afb-4ce4-4f18-be40-5d75cccc0aef service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:55.245179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6500d-2e77-4cfe-8c54-b58cc9fb915b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:55.248851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6500d-2e77-4cfe-8c54-b58cc9fb915b service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:55.249420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6500d-2e77-4cfe-8c54-b58cc9fb915b service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:55.251095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6500d-2e77-4cfe-8c54-b58cc9fb915b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:55.274707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6500d-2e77-4cfe-8c54-b58cc9fb915b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:55.276427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6500d-2e77-4cfe-8c54-b58cc9fb915b service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:55.290881 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3080/6129] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:55.313003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:55.315766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:55.336966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320143c-85ce-4df8-93ae-97aafd31f67e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:55.349091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:55.351682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:55.352179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:55.352600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:55.367405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:55.376856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:55.395901 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3050/6130] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:55.672858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6292d0a-d760-40ca-affa-c25fd048fe31 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:55.673167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6292d0a-d760-40ca-affa-c25fd048fe31 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:55.673448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6292d0a-d760-40ca-affa-c25fd048fe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:55.712969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c6292d0a-d760-40ca-affa-c25fd048fe31 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:55.714347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c6292d0a-d760-40ca-affa-c25fd048fe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:55.739841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3081/6131] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:19:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:19:55.749640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:55.770404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac0c30-70bd-49e4-85d1-a111c004db1e service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:55.788988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae64efa-4b11-4395-a058-fd61f40d4385 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:55.790093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64efa-4b11-4395-a058-fd61f40d4385 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:55.790627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64efa-4b11-4395-a058-fd61f40d4385 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:55.791125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64efa-4b11-4395-a058-fd61f40d4385 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:55.826247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae64efa-4b11-4395-a058-fd61f40d4385 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:55.828033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae64efa-4b11-4395-a058-fd61f40d4385 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:55.859604 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3051/6132] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:56.082117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6292d0a-d760-40ca-affa-c25fd048fe31 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:56.283712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6292d0a-d760-40ca-affa-c25fd048fe31 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:56.301242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:56.302008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:56.303702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:56.304039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] PATH_INFO: `/v3/users/5739c1d476a14938a3915fa4bf6607b8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:56.305633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5739c1d476a14938a3915fa4bf6607b8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:56.309959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:56.425045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5739c1d476a14938a3915fa4bf6607b8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:56.426117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5739c1d476a14938a3915fa4bf6607b8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:56.447673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5739c1d476a14938a3915fa4bf6607b8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:56.463417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5739c1d476a14938a3915fa4bf6607b8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:56.465805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5739c1d476a14938a3915fa4bf6607b8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:56.510665 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3082/6133] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:19:56 2022] DELETE /identity/v3/users/5739c1d476a14938a3915fa4bf6607b8 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:56.517593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:56.518167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:56.518167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:56.557744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:56.559568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:56.702261 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3052/6134] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:56.784670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:56.787064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:56.930033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2013-4275-4e10-b314-ee3906665776 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:56.978336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0ba96-1090-4aa3-aec6-dd8c5e1b6772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:56.978983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ba96-1090-4aa3-aec6-dd8c5e1b6772 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:56.979259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ba96-1090-4aa3-aec6-dd8c5e1b6772 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:56.979939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ba96-1090-4aa3-aec6-dd8c5e1b6772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:56.990487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0ba96-1090-4aa3-aec6-dd8c5e1b6772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:56.992072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0ba96-1090-4aa3-aec6-dd8c5e1b6772 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:57.000369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:57.018320 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3083/6135] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:19:59.075933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:59.077602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:59.112415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34843b9-4ded-47a7-8763-3d181754ca03 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:59.130742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:59.132178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:59.132771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:59.134154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] PATH_INFO: `/v3/projects/28c44d86cc114acaa440a74785de4ae6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:59.138405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c44d86cc114acaa440a74785de4ae6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:59.141174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:59.288281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c44d86cc114acaa440a74785de4ae6', 'request_id': 'req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:59.306968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c44d86cc114acaa440a74785de4ae6', 'request_id': 'req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:59.331069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c44d86cc114acaa440a74785de4ae6', 'request_id': 'req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:59.352014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] The token cache is being invalidate because project 28c44d86cc114acaa440a74785de4ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:19:59.352650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:59.353506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ffd8df20-c35b-40b2-b829-ea9d4a7951d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:19:59.355217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3053/6136] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:19:59 2022] DELETE /identity/v3/projects/28c44d86cc114acaa440a74785de4ae6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:19:59.360130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab60b6-9052-4f83-b714-47e69ec13db8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:59.360810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab60b6-9052-4f83-b714-47e69ec13db8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:59.360810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab60b6-9052-4f83-b714-47e69ec13db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:59.402238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-34ab60b6-9052-4f83-b714-47e69ec13db8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:59.406043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34ab60b6-9052-4f83-b714-47e69ec13db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:59.584437 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3084/6137] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:59.592485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e5489-c253-4806-8493-c16c4a1a657a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:59.593076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e5489-c253-4806-8493-c16c4a1a657a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:59.593203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e5489-c253-4806-8493-c16c4a1a657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:59.636537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-234e5489-c253-4806-8493-c16c4a1a657a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:19:59.639018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-234e5489-c253-4806-8493-c16c4a1a657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:19:59.676091 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3054/6138] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:19:59.681147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab60b6-9052-4f83-b714-47e69ec13db8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:59.705629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab60b6-9052-4f83-b714-47e69ec13db8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:59.738291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:59.739168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:59.739824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:59.741307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:59.742229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:19:59.742741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:59.757029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:59.791816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3085/6139] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:19:59.797562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e5489-c253-4806-8493-c16c4a1a657a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:59.824625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e5489-c253-4806-8493-c16c4a1a657a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:19:59.836599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:19:59.838468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:19:59.839173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:19:59.839690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:19:59.840584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:19:59.842988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:19:59.876741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:59.934111 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3055/6140] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:19:59 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:19:59.936008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:19:59.947075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c4744-19d3-4487-93e6-612299a8c4a2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.080902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.081988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.082439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.082891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.083743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.086092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:00.129440 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3086/6141] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:19:59 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:00.130897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:00.144387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6a272-74d7-46ef-adf7-84bedf74038e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.155607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.156951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.157557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.158361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.159288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.162366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:00.188302 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3056/6142] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:00.192870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:00.209126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865ecd4-d0e6-45b5-bc93-c8f81b175b91 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.216575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff978de-2375-407c-bbf1-96d5316034d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.219797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff978de-2375-407c-bbf1-96d5316034d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.219797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff978de-2375-407c-bbf1-96d5316034d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.220041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff978de-2375-407c-bbf1-96d5316034d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.220736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff978de-2375-407c-bbf1-96d5316034d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.223038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff978de-2375-407c-bbf1-96d5316034d0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:00.244237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:00.253479 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3087/6143] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:00.259892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab8c79-fadb-4380-a546-70a4b59d83f5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.269143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.269844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.270313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.270797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] PATH_INFO: `/v3/projects/5e6569ded30f43689fd5cf9a3fcdb2a3/users/5253c76068cc469487d68500da94030d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.293844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6569ded30f43689fd5cf9a3fcdb2a3, user_id=5253c76068cc469487d68500da94030d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.298732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:00.336552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad0101-e48b-4612-b294-8b023994c31e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.336819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad0101-e48b-4612-b294-8b023994c31e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.337041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad0101-e48b-4612-b294-8b023994c31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.352103 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3057/6144] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:00 2022] PUT /identity/v3/projects/5e6569ded30f43689fd5cf9a3fcdb2a3/users/5253c76068cc469487d68500da94030d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:00.396646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-52ad0101-e48b-4612-b294-8b023994c31e None None] MFA Rules not processed for user `5253c76068cc469487d68500da94030d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:00.398668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-52ad0101-e48b-4612-b294-8b023994c31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:00.499709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:00.516827 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3088/6145] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:00.521446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:00.633205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d58efd-b9d9-46a7-ab18-144416783c31 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.647097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.648677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.650321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.650776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.654852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.659285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:00.714732 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3058/6146] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:00 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:00.715791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad0101-e48b-4612-b294-8b023994c31e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:00.733877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad0101-e48b-4612-b294-8b023994c31e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.831722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.832331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.832849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.833333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.834358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.836560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:00.881067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3089/6147] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:00 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:00.885610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:00.903421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309ae33-f06b-4fce-beeb-a78f96b88e7a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.917714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.919146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.919472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.920190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.921707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.925906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:00.941337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3059/6148] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:00.945627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:00.967199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840c0f9-903e-4fae-b7a7-ae83a1f8ee50 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:00.994475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacd59a-49d5-42f8-a67b-8d799b0fbc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:00.995220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd59a-49d5-42f8-a67b-8d799b0fbc8b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:00.995582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd59a-49d5-42f8-a67b-8d799b0fbc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:00.995863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd59a-49d5-42f8-a67b-8d799b0fbc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:00.996721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacd59a-49d5-42f8-a67b-8d799b0fbc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:00.998493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacd59a-49d5-42f8-a67b-8d799b0fbc8b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:01.014666 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3090/6149] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:01.016556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:01.029251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345913-8546-4d1e-8969-5432ca33ecfb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:01.035182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0d4f-8340-4958-8cf6-481a77883eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:01.036398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0d4f-8340-4958-8cf6-481a77883eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:01.037254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0d4f-8340-4958-8cf6-481a77883eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:01.037773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0d4f-8340-4958-8cf6-481a77883eb4 admin admin] PATH_INFO: `/v3/projects/7a2d89c9a12f45ea8d7518333dab2732/users/61275b0a75ea4293b4faf2606ec74e17/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:01.058558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b0d4f-8340-4958-8cf6-481a77883eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a2d89c9a12f45ea8d7518333dab2732, user_id=61275b0a75ea4293b4faf2606ec74e17, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:01.063478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b0d4f-8340-4958-8cf6-481a77883eb4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:01.102032 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3060/6150] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:01 2022] PUT /identity/v3/projects/7a2d89c9a12f45ea8d7518333dab2732/users/61275b0a75ea4293b4faf2606ec74e17/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:01.155646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:01.155927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:01.156237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:01.195377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c9d7-5687-441d-9c39-c58fbe45ab4c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:01.195954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c9d7-5687-441d-9c39-c58fbe45ab4c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:01.195954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c9d7-5687-441d-9c39-c58fbe45ab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:01.213106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] MFA Rules not processed for user `64c9f44e57164b858421b55114acf4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:01.215059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:01.236773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e866c9d7-5687-441d-9c39-c58fbe45ab4c None None] MFA Rules not processed for user `61275b0a75ea4293b4faf2606ec74e17`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:01.238716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e866c9d7-5687-441d-9c39-c58fbe45ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:01.310005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3091/6151] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:01.325037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:01.346004 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3061/6152] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:01.365865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:01.369939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:01.513437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccdc7b-2709-4048-8502-128a3711df84 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:01.520411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:01.521817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:01.522357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:01.522815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:01.611644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:01.648266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:01.650116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:01.680912 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3092/6153] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:01.887105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:01.887910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:01.888688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:01.920309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:01.926931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:02.091155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:02.091811 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3062/6154] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:20:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:02.093128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:02.247525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec56cd-ff89-4b73-a115-3b6781dcc9ea service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:02.265512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c12ef-072a-48ce-9d59-d8fbc4244461 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:02.266156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c12ef-072a-48ce-9d59-d8fbc4244461 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:02.266432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c12ef-072a-48ce-9d59-d8fbc4244461 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:02.266706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c12ef-072a-48ce-9d59-d8fbc4244461 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:02.276907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c12ef-072a-48ce-9d59-d8fbc4244461 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:02.278504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c12ef-072a-48ce-9d59-d8fbc4244461 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:02.290323 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3093/6155] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:20:02.802623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:02.805199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:02.839198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9242147-06ed-43cd-bb37-4018252077c5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:02.857477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:02.858766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:02.860984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:02.864889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:02.906263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:02.910452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:02.946429 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3063/6156] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:03.161432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed678713-92c6-4fd8-8e9d-c29cc2d66804 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:03.162210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed678713-92c6-4fd8-8e9d-c29cc2d66804 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:03.162210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed678713-92c6-4fd8-8e9d-c29cc2d66804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:03.201004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ed678713-92c6-4fd8-8e9d-c29cc2d66804 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:03.203180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed678713-92c6-4fd8-8e9d-c29cc2d66804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:03.214806 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3094/6157] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:20:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:03.229171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:03.247838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b550366-fe32-4442-8a75-7a63b160e252 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:03.292492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5e37a-f3d3-4e85-b6af-303e45b42e1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:03.293073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5e37a-f3d3-4e85-b6af-303e45b42e1b service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:03.293333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5e37a-f3d3-4e85-b6af-303e45b42e1b service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:03.295097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5e37a-f3d3-4e85-b6af-303e45b42e1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:03.305638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5e37a-f3d3-4e85-b6af-303e45b42e1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:03.307994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5e37a-f3d3-4e85-b6af-303e45b42e1b service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:03.337000 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3064/6158] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:03.647788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed678713-92c6-4fd8-8e9d-c29cc2d66804 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:03.838621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed678713-92c6-4fd8-8e9d-c29cc2d66804 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:03.854672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:03.855552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:03.856993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:03.857271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] PATH_INFO: `/v3/users/5253c76068cc469487d68500da94030d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:03.864390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5253c76068cc469487d68500da94030d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:03.866963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:04.008238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5253c76068cc469487d68500da94030d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:04.009002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5253c76068cc469487d68500da94030d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:04.043543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5253c76068cc469487d68500da94030d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:04.058382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:04.065142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5253c76068cc469487d68500da94030d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:04.067176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5253c76068cc469487d68500da94030d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:04.112100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3095/6159] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:03 2022] DELETE /identity/v3/users/5253c76068cc469487d68500da94030d => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:04.113632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:04.114139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:04.114260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:04.180391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:04.188625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:04.368611 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3065/6160] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:04.461185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:04.462993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:04.730592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1ba89-a480-430b-8c61-d3568fdbd2e9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:04.736138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:04.737288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:04.737745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:04.738229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:04.753340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:04.756575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:04.770417 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3096/6161] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:05.737403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:05.740436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:05.776417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7229f7-dfe3-42b2-b0d3-f44ee3e6299e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:05.795511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:05.796193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:05.796476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:05.797491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] PATH_INFO: `/v3/projects/5e6569ded30f43689fd5cf9a3fcdb2a3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:05.803081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6569ded30f43689fd5cf9a3fcdb2a3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:05.805416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:05.932376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6569ded30f43689fd5cf9a3fcdb2a3', 'request_id': 'req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:05.943903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6569ded30f43689fd5cf9a3fcdb2a3', 'request_id': 'req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:05.967087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6569ded30f43689fd5cf9a3fcdb2a3', 'request_id': 'req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:05.980234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:05.989194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] The token cache is being invalidate because project 5e6569ded30f43689fd5cf9a3fcdb2a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:05.989603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:05.991859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:05.995051 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3066/6162] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:05 2022] DELETE /identity/v3/projects/5e6569ded30f43689fd5cf9a3fcdb2a3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:05.999489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:06.014219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1777da-4c3d-4d1c-acea-8d274763bb51 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:06.177761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:06.179880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:06.180225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:06.181255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] PATH_INFO: `/v3/users/61275b0a75ea4293b4faf2606ec74e17` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:06.185622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61275b0a75ea4293b4faf2606ec74e17)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:06.189483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:06.311465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61275b0a75ea4293b4faf2606ec74e17'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:06.315429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61275b0a75ea4293b4faf2606ec74e17'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:06.346698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61275b0a75ea4293b4faf2606ec74e17'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:06.360085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61275b0a75ea4293b4faf2606ec74e17'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:06.361925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a3661365-a239-4469-ac99-4b09d1e1df36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61275b0a75ea4293b4faf2606ec74e17'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:06.391273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:06.403472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3097/6163] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:05 2022] DELETE /identity/v3/users/61275b0a75ea4293b4faf2606ec74e17 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:07.752652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:07.755177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:07.778950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3cbd1-21c0-46c3-ae6e-ea6e9c3b8a8e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:07.985406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:07.986070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:07.987246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:07.987246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] PATH_INFO: `/v3/projects/7a2d89c9a12f45ea8d7518333dab2732` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:07.988041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a2d89c9a12f45ea8d7518333dab2732)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:07.992305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:08.072366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a2d89c9a12f45ea8d7518333dab2732', 'request_id': 'req-9abbd84a-5285-45a7-9002-0c4bda8fb400'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:08.081475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a2d89c9a12f45ea8d7518333dab2732', 'request_id': 'req-9abbd84a-5285-45a7-9002-0c4bda8fb400'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:08.095759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a2d89c9a12f45ea8d7518333dab2732', 'request_id': 'req-9abbd84a-5285-45a7-9002-0c4bda8fb400'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:08.111921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] The token cache is being invalidate because project 7a2d89c9a12f45ea8d7518333dab2732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:08.112330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:08.113097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9abbd84a-5285-45a7-9002-0c4bda8fb400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:08.115007 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3067/6164] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:07 2022] DELETE /identity/v3/projects/7a2d89c9a12f45ea8d7518333dab2732 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:08.120726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef97107-9249-4d4d-81ba-775f164cb0dd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.121726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef97107-9249-4d4d-81ba-775f164cb0dd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.122068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef97107-9249-4d4d-81ba-775f164cb0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.144723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7ef97107-9249-4d4d-81ba-775f164cb0dd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:08.146147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7ef97107-9249-4d4d-81ba-775f164cb0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:08.229130 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3098/6165] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:08.237662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3474-9df7-40b2-9e84-1175bd7c846b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.237954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3474-9df7-40b2-9e84-1175bd7c846b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.238237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3474-9df7-40b2-9e84-1175bd7c846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.263422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cb6c3474-9df7-40b2-9e84-1175bd7c846b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:08.264881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb6c3474-9df7-40b2-9e84-1175bd7c846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:08.278894 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3068/6166] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:08.297493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef97107-9249-4d4d-81ba-775f164cb0dd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:08.313005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef97107-9249-4d4d-81ba-775f164cb0dd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:08.321599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:08.322507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.323004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.323650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.324862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:08.325578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:08.327653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:08.343272 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3099/6167] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:20:08.348047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c3474-9df7-40b2-9e84-1175bd7c846b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:08.358315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c3474-9df7-40b2-9e84-1175bd7c846b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:08.367137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:08.367468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.368374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.368374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.368918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:08.370839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:08.410472 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3069/6168] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:08 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:08.414772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:08.430551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02068ebd-e158-45bb-bacf-2f29cbf82aad admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:08.503244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:08.503869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.504142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.504461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.505447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:08.507826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:08.566620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:08.568238 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3100/6169] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:08 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:08.582429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7b43-a35a-45fc-b1c7-41ff4b034528 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:08.591361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:08.593424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.593610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.597249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.597972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:08.616943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:08.631995 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3070/6170] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:08.633762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:08.648514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6fe98-1721-4f2d-a738-335c08531d87 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:08.656350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c702-f788-43a6-9fc3-cfa2f0de793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:08.657381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c702-f788-43a6-9fc3-cfa2f0de793a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.657894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c702-f788-43a6-9fc3-cfa2f0de793a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.659507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c702-f788-43a6-9fc3-cfa2f0de793a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.660397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c702-f788-43a6-9fc3-cfa2f0de793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:08.662743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c702-f788-43a6-9fc3-cfa2f0de793a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:08.680728 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3101/6171] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:08.685167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:08.708320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327ec3e-9577-4f11-b1a6-8bc15cc0cfce admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:08.713859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:08.714788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.715723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.716122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] PATH_INFO: `/v3/projects/e4584bc83ee14991a77466fa38ea77b7/users/827f070dff124e89aff2ba0601c45f3e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.745100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4584bc83ee14991a77466fa38ea77b7, user_id=827f070dff124e89aff2ba0601c45f3e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:08.749501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:08.781924 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3071/6172] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:08 2022] PUT /identity/v3/projects/e4584bc83ee14991a77466fa38ea77b7/users/827f070dff124e89aff2ba0601c45f3e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:08.797458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57f34c-918f-42e7-a5ec-392ef58f1d44 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:08.797458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57f34c-918f-42e7-a5ec-392ef58f1d44 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:08.797458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57f34c-918f-42e7-a5ec-392ef58f1d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:08.881938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dc57f34c-918f-42e7-a5ec-392ef58f1d44 None None] MFA Rules not processed for user `827f070dff124e89aff2ba0601c45f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:08.883409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dc57f34c-918f-42e7-a5ec-392ef58f1d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:08.994589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:09.010688 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3102/6173] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:09.034275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:09.036900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:09.091751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:09.142713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ed3f-aa77-4214-9b16-6d2d6686254e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:09.148042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:09.148873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:09.149568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:09.150189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:09.160422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:09.162501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:09.173371 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3072/6174] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:09.399224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2053e-ef7a-4be5-8f0f-85060d5d92a9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:09.400208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2053e-ef7a-4be5-8f0f-85060d5d92a9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:09.401237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2053e-ef7a-4be5-8f0f-85060d5d92a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:09.425610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c0d2053e-ef7a-4be5-8f0f-85060d5d92a9 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:09.427416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c0d2053e-ef7a-4be5-8f0f-85060d5d92a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:09.527288 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3103/6175] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:20:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:09.545948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:09.548865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:09.713060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11ae7d-d223-4814-8f8c-a630ec18b910 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:09.719847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:09.720464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:09.720871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:09.721330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:09.731984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:09.733994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:09.745467 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3073/6176] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:10.267109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904441-b599-4623-a11b-7c80eff1b30e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:10.267416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904441-b599-4623-a11b-7c80eff1b30e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:10.267646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904441-b599-4623-a11b-7c80eff1b30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:10.336282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-69904441-b599-4623-a11b-7c80eff1b30e None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:10.337652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69904441-b599-4623-a11b-7c80eff1b30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:10.350329 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3104/6177] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:20:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:10.377280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:10.398260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ce0ca-4b75-44df-9091-2d959b3b5bc3 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:10.407899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:10.408811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:10.409935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:10.410581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:10.427066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:10.429034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:10.439886 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3074/6178] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:10.858156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899b19-7813-4b5c-9c18-524d332db121 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:10.858747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899b19-7813-4b5c-9c18-524d332db121 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:10.859070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899b19-7813-4b5c-9c18-524d332db121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:10.915624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-69899b19-7813-4b5c-9c18-524d332db121 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:10.917216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69899b19-7813-4b5c-9c18-524d332db121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:10.937195 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3105/6179] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:20:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:10.958414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:10.978589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bdfff-769f-4ae9-a745-46f30c407ce6 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:10.987570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:10.988226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:10.988711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:10.989051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:11.000874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:11.001795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:11.022903 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3075/6180] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:11.470765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc3f65-c6c1-4ea4-8f53-d99af1d794ad None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:11.471339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc3f65-c6c1-4ea4-8f53-d99af1d794ad None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:11.472363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc3f65-c6c1-4ea4-8f53-d99af1d794ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:11.513330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b8bc3f65-c6c1-4ea4-8f53-d99af1d794ad None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:11.515044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8bc3f65-c6c1-4ea4-8f53-d99af1d794ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:11.525358 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3106/6181] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:11.544218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:11.563538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03c146-b970-4b4e-a597-a16dc462a3e3 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:11.571156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:11.572171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:11.572721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:11.573229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:11.585400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:11.587902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:11.619970 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3076/6182] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:12.175852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87344d8-bf82-4758-afbf-da410acb4d32 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:12.177453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87344d8-bf82-4758-afbf-da410acb4d32 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:12.177767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87344d8-bf82-4758-afbf-da410acb4d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:12.218607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c87344d8-bf82-4758-afbf-da410acb4d32 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:12.224787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c87344d8-bf82-4758-afbf-da410acb4d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:12.242193 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3107/6183] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:20:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:12.256395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:12.271788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed660-c935-4d1b-b84b-233f54456dd5 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:12.280499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:12.281606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:12.282150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:12.282637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:12.301374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:12.303359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:12.314658 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3077/6184] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:12.757700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734887db-8224-4428-a34a-74b7327a5377 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:12.758366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734887db-8224-4428-a34a-74b7327a5377 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:12.758822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734887db-8224-4428-a34a-74b7327a5377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:12.813769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-734887db-8224-4428-a34a-74b7327a5377 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:12.816512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-734887db-8224-4428-a34a-74b7327a5377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:12.845034 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3108/6185] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:20:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:12.852441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:12.868297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2c668-29a9-4602-bb51-d8685541b9b8 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:12.884020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:12.884956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:12.885236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:12.885529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:12.904844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:12.906707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:12.930183 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3078/6186] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:20:14.616411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb6a91-18f3-4735-91dd-32fe2859aecf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:14.616918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb6a91-18f3-4735-91dd-32fe2859aecf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:14.617046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb6a91-18f3-4735-91dd-32fe2859aecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:14.647665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:14.649599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-10eb6a91-18f3-4735-91dd-32fe2859aecf None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:14.649737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:14.651078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-10eb6a91-18f3-4735-91dd-32fe2859aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:14.673711 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3109/6187] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:20:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:14.692957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb6a91-18f3-4735-91dd-32fe2859aecf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:14.718964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb6a91-18f3-4735-91dd-32fe2859aecf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:14.727875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:14.736470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddab319-4444-403d-8f87-e35ea699ef70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:14.737871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddab319-4444-403d-8f87-e35ea699ef70 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:14.738307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddab319-4444-403d-8f87-e35ea699ef70 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:14.738754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddab319-4444-403d-8f87-e35ea699ef70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:14.758740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7236b-75a8-4ecb-bb66-ebbac3e24c2b service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:14.765074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:14.765725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:14.766142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:14.766424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] PATH_INFO: `/v3/users/46a22f58246647e9a990b3865fa5c4c1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:14.767552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a22f58246647e9a990b3865fa5c4c1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:14.769527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:14.771955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddab319-4444-403d-8f87-e35ea699ef70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:14.773716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddab319-4444-403d-8f87-e35ea699ef70 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:14.784683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3110/6188] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:20:14.838259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a22f58246647e9a990b3865fa5c4c1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:14.839019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a22f58246647e9a990b3865fa5c4c1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:14.846295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a22f58246647e9a990b3865fa5c4c1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:14.857405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a22f58246647e9a990b3865fa5c4c1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:14.873934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3722dd34-4ec7-4edf-a2a6-c045c47cfd51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a22f58246647e9a990b3865fa5c4c1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:14.881135 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3079/6189] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:14 2022] DELETE /identity/v3/users/46a22f58246647e9a990b3865fa5c4c1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:14.886403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:14.886748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:14.887067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:14.913706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:14.914942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:14.962676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:14.962957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:14.963540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:14.987395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:14.988708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:14.991089 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3111/6190] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:15.002807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:15.004221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:15.067765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3080/6191] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:20:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:15.076309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:15.078952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:15.081191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:15.089304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c6f30-85e8-406e-bc16-87a26dc20bc1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:15.093784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605b087-87d8-4b13-b5a0-c607d61a00ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:15.094524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605b087-87d8-4b13-b5a0-c607d61a00ad service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:15.094918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605b087-87d8-4b13-b5a0-c607d61a00ad service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:15.095187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605b087-87d8-4b13-b5a0-c607d61a00ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:15.106882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605b087-87d8-4b13-b5a0-c607d61a00ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:15.108558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605b087-87d8-4b13-b5a0-c607d61a00ad service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:15.121329 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3112/6192] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:15.167494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e96ae-f4be-4d9a-929f-2fad6e96608b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:15.175116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:15.175897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:15.176593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:15.176988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:15.191138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:15.195017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:15.210204 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3081/6193] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:20:15.455870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:15.456153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:15.456626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:15.504480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:15.507746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:15.528648 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3113/6194] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:20:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:15.549285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:15.562325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:15.575041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1946e4-fd54-4dcb-a810-16545496748e service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:15.593235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdeed68-05ae-4749-aed5-0261b601956b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:15.593939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeed68-05ae-4749-aed5-0261b601956b service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:15.594361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeed68-05ae-4749-aed5-0261b601956b service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:15.594730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeed68-05ae-4749-aed5-0261b601956b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:15.606629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeed68-05ae-4749-aed5-0261b601956b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:15.612542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeed68-05ae-4749-aed5-0261b601956b service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:15.636139 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3082/6195] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:15.731685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:15.733417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:15.762982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7812-63fe-48dc-b778-51ada0ac7866 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:15.770655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:15.771859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:15.774608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:15.775388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] PATH_INFO: `/v3/projects/1d576d7a3ba44e14ba171a3c8ee6a18a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:15.777213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d576d7a3ba44e14ba171a3c8ee6a18a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:15.790088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:15.860713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:15.862061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:15.862625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:15.878074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d576d7a3ba44e14ba171a3c8ee6a18a', 'request_id': 'req-548e185b-493e-458d-8a14-6fa4301e9dc1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:15.889583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d576d7a3ba44e14ba171a3c8ee6a18a', 'request_id': 'req-548e185b-493e-458d-8a14-6fa4301e9dc1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:15.897088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d576d7a3ba44e14ba171a3c8ee6a18a', 'request_id': 'req-548e185b-493e-458d-8a14-6fa4301e9dc1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:15.898041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:15.899759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:15.919819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] The token cache is being invalidate because project 1d576d7a3ba44e14ba171a3c8ee6a18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:15.920108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:15.920993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:15.925616 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3114/6196] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:15 2022] DELETE /identity/v3/projects/1d576d7a3ba44e14ba171a3c8ee6a18a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:15.926784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:15.930686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:15.996644 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3083/6197] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:20:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:15.997885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:16.009814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:16.056325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e185b-493e-458d-8a14-6fa4301e9dc1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:16.062832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:16.063944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.064462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.064932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] PATH_INFO: `/v3/users/1dc61d03273242b0987a6f10b59849b2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.066343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dc61d03273242b0987a6f10b59849b2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:16.069393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:16.126628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c0110-d72d-4725-a7d1-2a72c85e8af2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:16.132547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e0635-faa5-4b54-8f30-c96b82cefe7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:16.133247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e0635-faa5-4b54-8f30-c96b82cefe7c service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.133481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e0635-faa5-4b54-8f30-c96b82cefe7c service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.133758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e0635-faa5-4b54-8f30-c96b82cefe7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.138183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dc61d03273242b0987a6f10b59849b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.138974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dc61d03273242b0987a6f10b59849b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.144368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e0635-faa5-4b54-8f30-c96b82cefe7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:16.146427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e0635-faa5-4b54-8f30-c96b82cefe7c service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:16.148212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dc61d03273242b0987a6f10b59849b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.155517 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3084/6198] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:16.163428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dc61d03273242b0987a6f10b59849b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.164256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dc61d03273242b0987a6f10b59849b2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.169858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:16.189506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3115/6199] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:15 2022] DELETE /identity/v3/users/1dc61d03273242b0987a6f10b59849b2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:16.352717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.353195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.353367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.386741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:16.388380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:16.475369 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3085/6200] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:20:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:16.494375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:16.496009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:16.557513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:16.559677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:16.600082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d228-5a14-45bd-9b88-19e3486b5704 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:16.621064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736f87d-b222-4754-a01f-6095da331759 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:16.621650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f87d-b222-4754-a01f-6095da331759 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.621931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f87d-b222-4754-a01f-6095da331759 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.622201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f87d-b222-4754-a01f-6095da331759 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.628743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:16.638586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736f87d-b222-4754-a01f-6095da331759 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:16.642571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736f87d-b222-4754-a01f-6095da331759 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:16.658950 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3116/6201] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:16.676329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc50c8-70f9-4e46-bef3-307918f519e7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:16.686967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:16.688375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.688851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.689611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] PATH_INFO: `/v3/projects/cdb214195364462d817e8cabeac1f866` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.692044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdb214195364462d817e8cabeac1f866)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:16.695575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:16.790138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb214195364462d817e8cabeac1f866', 'request_id': 'req-cc61f527-4a5b-4055-82b0-2109f2a3c87c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.800517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb214195364462d817e8cabeac1f866', 'request_id': 'req-cc61f527-4a5b-4055-82b0-2109f2a3c87c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.813001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb214195364462d817e8cabeac1f866', 'request_id': 'req-cc61f527-4a5b-4055-82b0-2109f2a3c87c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.838476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] The token cache is being invalidate because project cdb214195364462d817e8cabeac1f866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:16.838774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.839463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cc61f527-4a5b-4055-82b0-2109f2a3c87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:16.843325 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3086/6202] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:16 2022] DELETE /identity/v3/projects/cdb214195364462d817e8cabeac1f866 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:16.846287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cef4a1-129b-4961-8be5-209e0c5b4108 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.846287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cef4a1-129b-4961-8be5-209e0c5b4108 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.847149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cef4a1-129b-4961-8be5-209e0c5b4108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.862410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.863726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.864007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.874593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d9cef4a1-129b-4961-8be5-209e0c5b4108 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:16.876058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d9cef4a1-129b-4961-8be5-209e0c5b4108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:16.887772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:16.890528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:16.976686 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3117/6203] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:16.978728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2390e-f17f-4b82-b525-e8fbd4e400e2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:16.979218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2390e-f17f-4b82-b525-e8fbd4e400e2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:16.979651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2390e-f17f-4b82-b525-e8fbd4e400e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:16.981262 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3087/6204] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:20:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:16.989026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:16.990629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:17.016396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bab2390e-f17f-4b82-b525-e8fbd4e400e2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:17.017845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bab2390e-f17f-4b82-b525-e8fbd4e400e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:17.044676 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3118/6205] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:17.049432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2390e-f17f-4b82-b525-e8fbd4e400e2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.067139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2390e-f17f-4b82-b525-e8fbd4e400e2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.076673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.077310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.077590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.077863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.081262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:17.081587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.084329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.110072 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3119/6206] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:17.112798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153537f2-b1ed-41aa-8322-ca05eb7c469d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.113965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.119070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.119881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.120437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.120885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.126811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3cc43-ed80-495a-b5db-b7783d4952e0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.128234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:17.131830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.133864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.135263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.135420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.135708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.136100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.136918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.143918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.167333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3088/6207] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:20:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:20:17.198500 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3120/6208] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:17 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:17.202560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.222217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579193-5c6f-40f9-b249-81dfb96b14ca service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.336192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.337489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.337955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.338441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.339253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.341475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.379579 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3089/6209] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:17 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:17.384950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.396810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5447a-55af-43c6-bc74-935244354419 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.401939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.402775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.403261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.403738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.404546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.406665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.419315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.422045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:17.423056 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3121/6210] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:17.431482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.442584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b987b-53ab-4ce7-aa13-517001f11bce admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.443476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2161-49a3-4357-b58b-e61575fd7d7e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.447949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.449061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.449442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.449819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.450923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.451288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.452054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.452494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.452945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] PATH_INFO: `/v3/users/827f070dff124e89aff2ba0601c45f3e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.453540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.454958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=827f070dff124e89aff2ba0601c45f3e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.458196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.492924 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3122/6211] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:17.501500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.518734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea631d-41f3-4ec0-a858-a99aab8c3ac7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.524787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913b09c-dd7d-44ea-9ba1-2c532b8b0eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.525481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913b09c-dd7d-44ea-9ba1-2c532b8b0eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.525760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913b09c-dd7d-44ea-9ba1-2c532b8b0eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.526041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913b09c-dd7d-44ea-9ba1-2c532b8b0eb3 admin admin] PATH_INFO: `/v3/projects/e7360fe11754459695933974bea9a0de/users/02a491391a244279896401b95fd3b36d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.532535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '827f070dff124e89aff2ba0601c45f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:17.533376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '827f070dff124e89aff2ba0601c45f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:17.536392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:17.546256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '827f070dff124e89aff2ba0601c45f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:17.546489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913b09c-dd7d-44ea-9ba1-2c532b8b0eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7360fe11754459695933974bea9a0de, user_id=02a491391a244279896401b95fd3b36d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.549897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913b09c-dd7d-44ea-9ba1-2c532b8b0eb3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.564296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '827f070dff124e89aff2ba0601c45f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:17.573506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3123/6212] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:17 2022] PUT /identity/v3/projects/e7360fe11754459695933974bea9a0de/users/02a491391a244279896401b95fd3b36d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:17.585386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.585766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.585948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.604152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9cf077b9-99bb-47ef-8212-18dd85c36772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '827f070dff124e89aff2ba0601c45f3e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:17.616440 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3090/6213] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:17 2022] DELETE /identity/v3/users/827f070dff124e89aff2ba0601c45f3e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:17.622187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.622476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.622763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.625790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] MFA Rules not processed for user `02a491391a244279896401b95fd3b36d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:17.627841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:17.650339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:17.652085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:17.702494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3124/6214] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:17.707630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.709550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:17.719580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:17.737125 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3091/6215] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:17.745178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.746811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:17.796891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab43a20-d7bf-4b26-8794-7c7d31c37ebc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.802676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.803422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.803704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.804190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.804605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.806604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.817454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642108d-f64b-4451-9073-e46caf396fb0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.822095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.822720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.823004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.823279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.834417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.836746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.839921 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3125/6216] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:17 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:17.843516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:17.856686 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3092/6217] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:20:17.940969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0a65-9e38-43d5-b55e-110a077eea1b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:17.946926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:17.948898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:17.951367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:17.951866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:17.953562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:17.960557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:17.981868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:18.017036 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3126/6218] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:17 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:18.020896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.042555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b5d4a-b0b4-4af7-86d1-deb9b8200569 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.057518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.059171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.061344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.063191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.065370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:18.071358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:18.086427 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3093/6219] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:18.087643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.104719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc714479-89f1-403d-abab-03529dd4e979 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.113014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.114116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.114678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.115471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] PATH_INFO: `/v3/projects/c69594d750fe475b86503c42144e362f/users/b767bc725f454d63a727b4590ac5d869/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.141169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69594d750fe475b86503c42144e362f, user_id=b767bc725f454d63a727b4590ac5d869, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:18.146448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:18.175949 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3127/6220] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:18 2022] PUT /identity/v3/projects/c69594d750fe475b86503c42144e362f/users/b767bc725f454d63a727b4590ac5d869/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:18.179963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.197895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dd63a-25b9-4085-818f-23c0db7838b1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.285043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4be5e-26ff-48bc-aa52-0883e047b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.285695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4be5e-26ff-48bc-aa52-0883e047b1cf admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.286125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4be5e-26ff-48bc-aa52-0883e047b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.286740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4be5e-26ff-48bc-aa52-0883e047b1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.287414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4be5e-26ff-48bc-aa52-0883e047b1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:18.291439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4be5e-26ff-48bc-aa52-0883e047b1cf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:18.327006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3094/6221] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:18.328806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.340412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d4cd4-cd0e-4f37-975e-2da939e62790 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.346673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.347494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.347844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.348198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] PATH_INFO: `/v3/projects/c69594d750fe475b86503c42144e362f/users/b767bc725f454d63a727b4590ac5d869/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.350531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69594d750fe475b86503c42144e362f, user_id=b767bc725f454d63a727b4590ac5d869, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:18.353565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:18.367648 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3128/6222] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:18 2022] PUT /identity/v3/projects/c69594d750fe475b86503c42144e362f/users/b767bc725f454d63a727b4590ac5d869/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:18.378167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.378455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.378752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.418401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] MFA Rules not processed for user `b767bc725f454d63a727b4590ac5d869`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:18.420262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:18.503477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.506174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:18.578329 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3095/6223] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:18.595884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.598427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:18.627835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:18.686955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebc7cf-e70f-453f-96b0-9a149fe1c1f5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.721704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.722355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.722618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.722924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.725823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:18.753078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:18.755668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:18.772524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da292c1-498e-4318-814d-a5ce90c5ead0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.777872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.778790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.779889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.780491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.788921 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3129/6224] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:20:18.789986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.791698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:18.799348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:18.803886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:18.832768 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3096/6225] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:18.893875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca145d-6567-45a4-9263-d3ba7d173779 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.903303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.903958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.904228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.904805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] PATH_INFO: `/v3/projects/e4584bc83ee14991a77466fa38ea77b7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.906145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4584bc83ee14991a77466fa38ea77b7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:18.906886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:18.908371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:18.909032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:18.925351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0fd36-f00a-4663-b3d4-178f1af30fc4 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:18.933053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeba3c-043c-4d5b-9697-17013a0c38d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:18.933836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeba3c-043c-4d5b-9697-17013a0c38d5 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:18.934198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeba3c-043c-4d5b-9697-17013a0c38d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:18.934540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeba3c-043c-4d5b-9697-17013a0c38d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:18.966366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=4,keystone:SELECT=93,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:18.972792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4584bc83ee14991a77466fa38ea77b7', 'request_id': 'req-33e999cb-b222-4d59-b413-442f7087cd32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:18.987928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4584bc83ee14991a77466fa38ea77b7', 'request_id': 'req-33e999cb-b222-4d59-b413-442f7087cd32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:18.999025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4584bc83ee14991a77466fa38ea77b7', 'request_id': 'req-33e999cb-b222-4d59-b413-442f7087cd32'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:19.017448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] The token cache is being invalidate because project e4584bc83ee14991a77466fa38ea77b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:19.018064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:19.018929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeba3c-043c-4d5b-9697-17013a0c38d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.020031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-33e999cb-b222-4d59-b413-442f7087cd32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:19.020552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeba3c-043c-4d5b-9697-17013a0c38d5 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.022087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3130/6226] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:18 2022] DELETE /identity/v3/projects/e4584bc83ee14991a77466fa38ea77b7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:19.026978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30980e57-1abb-435a-beb0-953890844673 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.027348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30980e57-1abb-435a-beb0-953890844673 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.027622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30980e57-1abb-435a-beb0-953890844673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.036382 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3097/6227] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:20:19.054417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-30980e57-1abb-435a-beb0-953890844673 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:19.055779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-30980e57-1abb-435a-beb0-953890844673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:19.128261 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3131/6228] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:19.140369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45e9d2-1305-4fe1-8339-167fc7eadac5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.140369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45e9d2-1305-4fe1-8339-167fc7eadac5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.140762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45e9d2-1305-4fe1-8339-167fc7eadac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.163052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4d45e9d2-1305-4fe1-8339-167fc7eadac5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:19.164418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d45e9d2-1305-4fe1-8339-167fc7eadac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:19.192537 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3098/6229] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:19.195161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30980e57-1abb-435a-beb0-953890844673 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.211828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30980e57-1abb-435a-beb0-953890844673 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:19.222402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:19.223131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.223402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.223678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.224701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:19.225197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.227549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.248721 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3132/6230] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:19.253601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45e9d2-1305-4fe1-8339-167fc7eadac5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.265877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45e9d2-1305-4fe1-8339-167fc7eadac5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:19.273450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:19.274613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.275036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.275372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.277128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.278206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.315466 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3099/6231] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:19 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:19.319601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.331555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d954ac-1478-4a0a-961c-72a96cbc4e79 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:19.406142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:19.406896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.407416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.407695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.408575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.411282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.445078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.451863 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3133/6232] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:19 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:19.460431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219c445-a8b8-4260-885d-6f8ac346fbb4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:19.467617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:19.468130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.468495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.469038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.470267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.471834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.489582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.500326 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3100/6233] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:19.501293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd889e30-f63e-43cd-b74c-717ace31bfd6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:19.506734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467399d-fef1-44a3-9364-c1975bfd36b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:19.507351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467399d-fef1-44a3-9364-c1975bfd36b0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.507737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467399d-fef1-44a3-9364-c1975bfd36b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.508009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467399d-fef1-44a3-9364-c1975bfd36b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.509197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467399d-fef1-44a3-9364-c1975bfd36b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.510942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467399d-fef1-44a3-9364-c1975bfd36b0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.529972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.535696 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3134/6234] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:19.541332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536da938-882c-4790-ac9f-3b6c02b559d1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:19.546755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:19.547725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.547984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.548139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] PATH_INFO: `/v3/projects/dc44549a5bba49639cb279171df82c3f/users/02fbd57819f04cbfa9780ec4823cb77c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.571856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc44549a5bba49639cb279171df82c3f, user_id=02fbd57819f04cbfa9780ec4823cb77c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.575061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.624455 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3101/6235] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:19 2022] PUT /identity/v3/projects/dc44549a5bba49639cb279171df82c3f/users/02fbd57819f04cbfa9780ec4823cb77c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:19.635625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be319d02-1e57-4866-9d2a-86f0e73e9925 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.636797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be319d02-1e57-4866-9d2a-86f0e73e9925 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.636797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be319d02-1e57-4866-9d2a-86f0e73e9925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.694724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-be319d02-1e57-4866-9d2a-86f0e73e9925 None None] MFA Rules not processed for user `02fbd57819f04cbfa9780ec4823cb77c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:19.695920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-be319d02-1e57-4866-9d2a-86f0e73e9925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:19.733596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:19.803532 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3135/6236] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:19.804683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.863183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:19.883987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6b33-6c38-49f2-97f6-4631655d0619 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:19.889889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:19.890574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:19.891045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:19.891329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:19.891972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:19.894009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:19.932709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3102/6237] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:19 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:19.936702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be319d02-1e57-4866-9d2a-86f0e73e9925 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:19.950367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be319d02-1e57-4866-9d2a-86f0e73e9925 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:20.035945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:20.036811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:20.036979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:20.037240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:20.038316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:20.040550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:20.075193 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3136/6238] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:19 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:20.075657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:20.087857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e3fb-69f5-46e6-a971-3f0db66374d7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:20.094578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:20.095230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:20.095511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:20.095788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:20.096460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:20.098141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:20.112720 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3103/6239] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:20.115342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:20.126507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53770d-9e9c-483f-983b-2db1f12a1a9c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:20.132914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877de764-1e49-4c92-a757-fb8ad6e89b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:20.133659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877de764-1e49-4c92-a757-fb8ad6e89b60 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:20.134100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877de764-1e49-4c92-a757-fb8ad6e89b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:20.134363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877de764-1e49-4c92-a757-fb8ad6e89b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:20.135151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877de764-1e49-4c92-a757-fb8ad6e89b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:20.137055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877de764-1e49-4c92-a757-fb8ad6e89b60 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:20.148583 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3137/6240] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:20.152768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:20.163461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52e429-ec1d-4e93-b34d-1b9f874c5282 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:20.168823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:20.169576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:20.169946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:20.170285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] PATH_INFO: `/v3/projects/48f460ae10c945228cb1a12becf13fe3/users/4c113cb55dc643e4afe4f279ce4e5eb1/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:20.190007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f460ae10c945228cb1a12becf13fe3, user_id=4c113cb55dc643e4afe4f279ce4e5eb1, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:20.193321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:20.214421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3104/6241] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:20 2022] PUT /identity/v3/projects/48f460ae10c945228cb1a12becf13fe3/users/4c113cb55dc643e4afe4f279ce4e5eb1/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:20.226264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30945915-1e57-454e-b2d9-af65e2181b19 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:20.226553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30945915-1e57-454e-b2d9-af65e2181b19 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:20.226833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30945915-1e57-454e-b2d9-af65e2181b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:20.282842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-30945915-1e57-454e-b2d9-af65e2181b19 None None] MFA Rules not processed for user `4c113cb55dc643e4afe4f279ce4e5eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:20.284227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-30945915-1e57-454e-b2d9-af65e2181b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:20.410180 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3138/6242] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:20.427523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:20.429781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:20.508037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c64dc3-ed2f-4c3f-9460-32f9c50d4cb8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:20.513096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:20.513845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:20.514137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:20.514414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:20.614898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:20.618009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:20.632423 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3105/6243] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:20.863460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30945915-1e57-454e-b2d9-af65e2181b19 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:20.887521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30945915-1e57-454e-b2d9-af65e2181b19 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:20.897318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983efac-1b91-46c5-8005-43177433e799 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:20.898145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983efac-1b91-46c5-8005-43177433e799 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:20.898575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983efac-1b91-46c5-8005-43177433e799 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:20.902884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983efac-1b91-46c5-8005-43177433e799 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:20.940526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983efac-1b91-46c5-8005-43177433e799 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:20.942558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983efac-1b91-46c5-8005-43177433e799 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:20.955031 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3139/6244] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:21.549175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:21.634875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:21.649600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d40d-973d-4f11-a9fd-47d7349ac5f0 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:21.655154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:21.655999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:21.656385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:21.656800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] PATH_INFO: `/v3/users/02fbd57819f04cbfa9780ec4823cb77c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:21.658152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02fbd57819f04cbfa9780ec4823cb77c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:21.660202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:21.717654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02fbd57819f04cbfa9780ec4823cb77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:21.718624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02fbd57819f04cbfa9780ec4823cb77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:21.727988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02fbd57819f04cbfa9780ec4823cb77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:21.739392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02fbd57819f04cbfa9780ec4823cb77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:21.753283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02fbd57819f04cbfa9780ec4823cb77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:21.760212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3106/6245] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:21 2022] DELETE /identity/v3/users/02fbd57819f04cbfa9780ec4823cb77c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:21.764949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:21.765468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:21.765468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:21.787474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:21.788309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:21.826796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:21.919308 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3140/6246] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:21.953832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:21.955691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:22.030026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ded16-1972-4d53-9b3e-dec8ac70e985 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:22.034302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:22.035056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:22.035403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:22.035680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:22.046159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:22.047701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:22.060406 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3107/6247] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:23.202490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:23.205004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:23.229807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a15131-5984-4aeb-a034-47fb78165a58 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:23.238298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:23.241996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:23.244891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:23.245031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] PATH_INFO: `/v3/projects/dc44549a5bba49639cb279171df82c3f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:23.249493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc44549a5bba49639cb279171df82c3f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:23.265400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:23.338865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc44549a5bba49639cb279171df82c3f', 'request_id': 'req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.351932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc44549a5bba49639cb279171df82c3f', 'request_id': 'req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.367705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc44549a5bba49639cb279171df82c3f', 'request_id': 'req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.392909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] The token cache is being invalidate because project dc44549a5bba49639cb279171df82c3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:23.393312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.394132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.398856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3141/6248] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:23 2022] DELETE /identity/v3/projects/dc44549a5bba49639cb279171df82c3f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:23.399556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:23.401089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:23.402251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:23.403642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:23.415315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48651724-ea04-4e77-89bc-22772f88430d service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:23.532504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:23.533389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:23.533552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:23.533728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] PATH_INFO: `/v3/users/02a491391a244279896401b95fd3b36d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:23.535658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e50c49-bf1f-4057-ae0e-406ba20ee92f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:23.537362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a491391a244279896401b95fd3b36d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:23.539477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:23.555974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:23.558762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:23.559231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:23.559517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] PATH_INFO: `/v3/users/4c113cb55dc643e4afe4f279ce4e5eb1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:23.560911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c113cb55dc643e4afe4f279ce4e5eb1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:23.563258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:23.573531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:23.638548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a491391a244279896401b95fd3b36d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.639356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a491391a244279896401b95fd3b36d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.649095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a491391a244279896401b95fd3b36d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.659280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c113cb55dc643e4afe4f279ce4e5eb1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.660155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c113cb55dc643e4afe4f279ce4e5eb1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.662417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a491391a244279896401b95fd3b36d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.665812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:23.677700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c113cb55dc643e4afe4f279ce4e5eb1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.688036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c113cb55dc643e4afe4f279ce4e5eb1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.688854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cde685a7-771f-4dab-8bfb-01c4d49135a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a491391a244279896401b95fd3b36d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.689343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c113cb55dc643e4afe4f279ce4e5eb1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:23.702636 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3108/6249] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:23 2022] DELETE /identity/v3/users/02a491391a244279896401b95fd3b36d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:23.705563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:23.705961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:23.706835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:23.717275 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3142/6250] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:23 2022] DELETE /identity/v3/users/4c113cb55dc643e4afe4f279ce4e5eb1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:23.737320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:23.738886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:23.822542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3109/6251] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:23.839059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:23.840515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:23.927430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce9e70-34d0-4d76-aca4-1dbd17cfbabf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:23.934875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423947f-5bcb-452d-98f6-90308453d13e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:23.936276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423947f-5bcb-452d-98f6-90308453d13e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:23.937584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423947f-5bcb-452d-98f6-90308453d13e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:23.938476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423947f-5bcb-452d-98f6-90308453d13e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:23.986576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423947f-5bcb-452d-98f6-90308453d13e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:23.989688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423947f-5bcb-452d-98f6-90308453d13e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:24.002768 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3143/6252] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:24.992784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:24.994378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:25.015461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0360e-7c78-4590-b8cd-e5f080b968f3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:25.034690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:25.035831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:25.036362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:25.036841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] PATH_INFO: `/v3/projects/48f460ae10c945228cb1a12becf13fe3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:25.038229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=48f460ae10c945228cb1a12becf13fe3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:25.051600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:25.163140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48f460ae10c945228cb1a12becf13fe3', 'request_id': 'req-a57c2868-6738-473a-bf9a-fef36ae3b5ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.177457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48f460ae10c945228cb1a12becf13fe3', 'request_id': 'req-a57c2868-6738-473a-bf9a-fef36ae3b5ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.214498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48f460ae10c945228cb1a12becf13fe3', 'request_id': 'req-a57c2868-6738-473a-bf9a-fef36ae3b5ef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.235217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] The token cache is being invalidate because project 48f460ae10c945228cb1a12becf13fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:25.235696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.236296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a57c2868-6738-473a-bf9a-fef36ae3b5ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.237851 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3110/6253] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:24 2022] DELETE /identity/v3/projects/48f460ae10c945228cb1a12becf13fe3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:25.244192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:25.246389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:25.247164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:25.287820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:25.297370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:25.396216 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3144/6254] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:25.399796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016abe95-a887-44ca-89f6-7ae1cb592422 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:25.400954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016abe95-a887-44ca-89f6-7ae1cb592422 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:25.401113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016abe95-a887-44ca-89f6-7ae1cb592422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:25.418087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:25.420083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:25.456050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-016abe95-a887-44ca-89f6-7ae1cb592422 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:25.458696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-016abe95-a887-44ca-89f6-7ae1cb592422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:25.459606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e431e52-7f78-4877-adf3-0b0bff4fbd61 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:25.469422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:25.470074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:25.470348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:25.470616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] PATH_INFO: `/v3/projects/e7360fe11754459695933974bea9a0de` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:25.475416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7360fe11754459695933974bea9a0de)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:25.483219 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3111/6255] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:25.483447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:25.484584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016abe95-a887-44ca-89f6-7ae1cb592422 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:25.511329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016abe95-a887-44ca-89f6-7ae1cb592422 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:25.527210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:25.527836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:25.529042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:25.529042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:25.529042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:25.529338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:25.536625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:25.563329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:25.583206 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3112/6256] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:20:25.593753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:25.650993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83fc92-f0e9-4a34-a8d2-d48b1971d0db admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:25.668043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7360fe11754459695933974bea9a0de', 'request_id': 'req-b92e36b5-b00f-4202-b51b-8d0c8add3db3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.700703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7360fe11754459695933974bea9a0de', 'request_id': 'req-b92e36b5-b00f-4202-b51b-8d0c8add3db3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.709571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7360fe11754459695933974bea9a0de', 'request_id': 'req-b92e36b5-b00f-4202-b51b-8d0c8add3db3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.711157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:25.737974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] The token cache is being invalidate because project e7360fe11754459695933974bea9a0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:25.738686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.739703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:25.741927 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3145/6257] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:25 2022] DELETE /identity/v3/projects/e7360fe11754459695933974bea9a0de => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:25.746691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:25.748433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:25.822354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:25.825094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:25.825739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:25.827493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:25.828460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:25.835059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:25.902777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e36b5-b00f-4202-b51b-8d0c8add3db3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:25.908438 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3113/6258] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:25 2022] POST /identity/v3/projects => generated 373 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:25.912965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:25.914783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:26.053299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a9a0-192b-47d7-841a-856d76a66cb0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:26.053963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:26.057585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.057877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.058151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] PATH_INFO: `/v3/users/b767bc725f454d63a727b4590ac5d869` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.059157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:26.059808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b767bc725f454d63a727b4590ac5d869)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:26.059955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.060356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.060717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.061698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:26.063714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:26.066469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:26.097997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3114/6259] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:25 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:26.099038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:26.113283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3c5f1-f177-4172-8b09-1d63eada1a5b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:26.118963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:26.119696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.120429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.120835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.122919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:26.127120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:26.143196 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3115/6260] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:26.150282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:26.172512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d49422-b01c-48aa-9432-ce823eda366e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:26.180440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b767bc725f454d63a727b4590ac5d869'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.181813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b767bc725f454d63a727b4590ac5d869'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.183183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:26.183829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.184117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.184395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.185146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:26.186998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:26.206100 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3116/6261] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:26.210163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b767bc725f454d63a727b4590ac5d869'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.210423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:26.212948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:26.220838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b767bc725f454d63a727b4590ac5d869'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.221799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b767bc725f454d63a727b4590ac5d869'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.231165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b5a0-e010-4c3e-9e86-8a876d1df063 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:26.244804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d29af6-b518-40b9-bf69-8d85370ce9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:26.248291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d29af6-b518-40b9-bf69-8d85370ce9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.248885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d29af6-b518-40b9-bf69-8d85370ce9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.250902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d29af6-b518-40b9-bf69-8d85370ce9f4 admin admin] PATH_INFO: `/v3/projects/ef7491906c7f484483794b676660346e/users/bf619d61cc4f40dc9e2d65071f67e3e0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.255729 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3146/6262] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:25 2022] DELETE /identity/v3/users/b767bc725f454d63a727b4590ac5d869 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:26.281034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:26.283314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:26.296341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d29af6-b518-40b9-bf69-8d85370ce9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7491906c7f484483794b676660346e, user_id=bf619d61cc4f40dc9e2d65071f67e3e0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:26.301318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d29af6-b518-40b9-bf69-8d85370ce9f4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:26.344839 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3117/6263] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:26 2022] PUT /identity/v3/projects/ef7491906c7f484483794b676660346e/users/bf619d61cc4f40dc9e2d65071f67e3e0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:26.346501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.346797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.347076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.350471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:26.408497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] MFA Rules not processed for user `bf619d61cc4f40dc9e2d65071f67e3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:26.412628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:26.444050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d29c74-efef-48c2-ab8e-1907431887a5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:26.453625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:26.545142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:26.545903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.547819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.548193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] PATH_INFO: `/v3/users/e46a8a1c539e4781afda6aadd5373c8e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.549906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46a8a1c539e4781afda6aadd5373c8e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:26.552188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:26.621568 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3118/6264] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:26.633768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:26.636400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:26.664185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46a8a1c539e4781afda6aadd5373c8e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.665894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46a8a1c539e4781afda6aadd5373c8e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.696521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46a8a1c539e4781afda6aadd5373c8e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.717330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46a8a1c539e4781afda6aadd5373c8e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.718439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-45e24c76-6801-4d75-8216-89ba0d4939d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46a8a1c539e4781afda6aadd5373c8e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:26.766065 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3147/6265] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:26 2022] DELETE /identity/v3/users/e46a8a1c539e4781afda6aadd5373c8e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:26.767475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01801-231c-478e-9835-d2372d463164 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.767765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01801-231c-478e-9835-d2372d463164 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.768041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01801-231c-478e-9835-d2372d463164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.805440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461a1fb-7a38-4ef6-a940-50c21f143459 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:26.825076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f8d01801-231c-478e-9835-d2372d463164 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:26.830368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8d01801-231c-478e-9835-d2372d463164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:26.937717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:26.938914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:26.939419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:26.939845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:26.940356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:26.951045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:26.990763 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3148/6266] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:27.051654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d01801-231c-478e-9835-d2372d463164 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:27.054383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8d01801-231c-478e-9835-d2372d463164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:27.091969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:27.095602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:27.124971 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3119/6267] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:20:27.197131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d01801-231c-478e-9835-d2372d463164 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:27.202722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851fedc-a00a-4dfd-ac34-4e96ca117c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:27.203583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851fedc-a00a-4dfd-ac34-4e96ca117c4c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:27.204015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851fedc-a00a-4dfd-ac34-4e96ca117c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:27.205463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851fedc-a00a-4dfd-ac34-4e96ca117c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:27.239036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851fedc-a00a-4dfd-ac34-4e96ca117c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:27.243271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851fedc-a00a-4dfd-ac34-4e96ca117c4c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:27.286690 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3149/6268] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:20:27.730680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:27.734122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:27.765380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b980495-b18c-464f-bfcf-6ae388688e68 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:27.789418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:27.790515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:27.791170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:27.791614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] PATH_INFO: `/v3/projects/c69594d750fe475b86503c42144e362f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:27.793855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69594d750fe475b86503c42144e362f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:27.796756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:27.853499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:27.903608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69594d750fe475b86503c42144e362f', 'request_id': 'req-0a66da19-517f-4e30-a73f-9622411a4323'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:27.915388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69594d750fe475b86503c42144e362f', 'request_id': 'req-0a66da19-517f-4e30-a73f-9622411a4323'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:27.933461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69594d750fe475b86503c42144e362f', 'request_id': 'req-0a66da19-517f-4e30-a73f-9622411a4323'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:27.958195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] The token cache is being invalidate because project c69594d750fe475b86503c42144e362f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:27.958597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:27.959805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a66da19-517f-4e30-a73f-9622411a4323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:27.961727 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3120/6269] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:27 2022] DELETE /identity/v3/projects/c69594d750fe475b86503c42144e362f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:27.967299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2fe00-9a3c-45ae-8b66-20acd119f75d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:27.967528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2fe00-9a3c-45ae-8b66-20acd119f75d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:27.968016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2fe00-9a3c-45ae-8b66-20acd119f75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.022679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a5f2fe00-9a3c-45ae-8b66-20acd119f75d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:28.024427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a5f2fe00-9a3c-45ae-8b66-20acd119f75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:28.145644 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3150/6270] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:28.148714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8bffd-4b9f-4627-ac58-b14efa59f908 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.149239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8bffd-4b9f-4627-ac58-b14efa59f908 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.150576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8bffd-4b9f-4627-ac58-b14efa59f908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.179568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-99d8bffd-4b9f-4627-ac58-b14efa59f908 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:28.182843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99d8bffd-4b9f-4627-ac58-b14efa59f908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:28.199104 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3121/6271] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:28.200114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2fe00-9a3c-45ae-8b66-20acd119f75d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.215166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2fe00-9a3c-45ae-8b66-20acd119f75d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.230729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:28.231360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.231686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.232007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.232684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:28.233104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:28.234918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:28.266862 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3151/6272] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:28.272437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8bffd-4b9f-4627-ac58-b14efa59f908 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.293932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8bffd-4b9f-4627-ac58-b14efa59f908 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.309492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:28.310148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.310696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.311120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.311787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:28.316233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:28.382099 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3122/6273] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:28 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:28.386790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.401328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761896d9-9189-409b-b8de-d132e2ce15a7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.558679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:28.558679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.558679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.558679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.559514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:28.565890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:28.596601 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3152/6274] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:28 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:28.600576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.613414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3181b-79bc-4438-8114-480b105d93d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.634321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:28.639567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.641850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.642468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.643187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:28.649214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:28.664416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.666774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:28.668460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3123/6275] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:28.670047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.682512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d87dd-2c69-4874-812f-db534809ba51 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.684421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cfe7b-e559-4d1a-aaeb-cf25cd410a69 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.687781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:28.688235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.688617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.689221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] PATH_INFO: `/v3/users/bf619d61cc4f40dc9e2d65071f67e3e0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.691370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf619d61cc4f40dc9e2d65071f67e3e0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:28.694221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:28.696416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:28.696601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:28.697510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.698035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.698563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.699665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:28.702520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:28.715196 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3124/6276] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:28.719310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.730418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198443d-6d27-40e0-92aa-bf2b47a36b81 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.740024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:28.741087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.742031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.742427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] PATH_INFO: `/v3/projects/e7c268d4e8144356bbd53273e847653e/users/53e8c8b1a46e4d0badb04fdc15765505/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.786596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf619d61cc4f40dc9e2d65071f67e3e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:28.787872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf619d61cc4f40dc9e2d65071f67e3e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:28.789080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c268d4e8144356bbd53273e847653e, user_id=53e8c8b1a46e4d0badb04fdc15765505, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:28.793344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:28.809135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf619d61cc4f40dc9e2d65071f67e3e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:28.818905 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3125/6277] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:28 2022] PUT /identity/v3/projects/e7c268d4e8144356bbd53273e847653e/users/53e8c8b1a46e4d0badb04fdc15765505/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:28.819855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:28.822419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf619d61cc4f40dc9e2d65071f67e3e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:28.823722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:28.828499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fcd31203-2ac0-41f6-a6a6-c5e190351cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf619d61cc4f40dc9e2d65071f67e3e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:28.851628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3153/6278] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:28 2022] DELETE /identity/v3/users/bf619d61cc4f40dc9e2d65071f67e3e0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:28.851628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7d73-60d4-48aa-9af4-bfda0a8d8e2c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.851628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7d73-60d4-48aa-9af4-bfda0a8d8e2c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.851628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7d73-60d4-48aa-9af4-bfda0a8d8e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.885456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c55a7d73-60d4-48aa-9af4-bfda0a8d8e2c None None] MFA Rules not processed for user `53e8c8b1a46e4d0badb04fdc15765505`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:28.885456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c55a7d73-60d4-48aa-9af4-bfda0a8d8e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:28.971642 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3154/6279] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:28.973614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:28.974122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:28.974567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:28.975145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21b630-4b08-4e5f-8517-5db319e8eb2f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:28.997908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:28.999776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:29.038736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:29.069629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.070700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.070981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.071297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] PATH_INFO: `/v3/projects/a82b2468ace34a009ca80c159ed44bb8` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.073607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82b2468ace34a009ca80c159ed44bb8)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.076519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.096542 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3155/6280] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:29.098272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.099270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:29.120264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e8fe7-0647-4080-8cee-f419eb1116fc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.128063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.129614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.130221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.130654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.131467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.139178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.154480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:29.168800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82b2468ace34a009ca80c159ed44bb8', 'request_id': 'req-cba411c2-4fa9-40fc-80ac-2bb1956b513d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:29.176821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82b2468ace34a009ca80c159ed44bb8', 'request_id': 'req-cba411c2-4fa9-40fc-80ac-2bb1956b513d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:29.188364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82b2468ace34a009ca80c159ed44bb8', 'request_id': 'req-cba411c2-4fa9-40fc-80ac-2bb1956b513d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:29.200001 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3156/6281] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:29 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:29.201147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.202909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:29.207615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] The token cache is being invalidate because project a82b2468ace34a009ca80c159ed44bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:29.207914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:29.208668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:29.212766 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3126/6282] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:28 2022] DELETE /identity/v3/projects/a82b2468ace34a009ca80c159ed44bb8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:29.212766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.212766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:29.281796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba79e30-5732-4c9f-9292-e079e0161a94 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.285500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629b80b-b0b4-49bb-be7c-7a86d5bd7c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.286344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629b80b-b0b4-49bb-be7c-7a86d5bd7c0c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.286677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629b80b-b0b4-49bb-be7c-7a86d5bd7c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.286945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629b80b-b0b4-49bb-be7c-7a86d5bd7c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.289445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3629b80b-b0b4-49bb-be7c-7a86d5bd7c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:29.367624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba411c2-4fa9-40fc-80ac-2bb1956b513d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.373674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.373897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629b80b-b0b4-49bb-be7c-7a86d5bd7c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.374745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.375026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.375302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.375951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.377474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629b80b-b0b4-49bb-be7c-7a86d5bd7c0c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.378840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.390816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3157/6283] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:29.392779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377c194-f6de-485e-9b39-2fc88a56c471 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.393259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377c194-f6de-485e-9b39-2fc88a56c471 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.393732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377c194-f6de-485e-9b39-2fc88a56c471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.413633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3127/6284] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:29 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:29.417479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.417855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6377c194-f6de-485e-9b39-2fc88a56c471 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:29.419245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6377c194-f6de-485e-9b39-2fc88a56c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:29.428230 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3158/6285] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:29.435315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685119e8-a031-4e36-9931-df55a2924af2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.437660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf65e3-b458-4b6b-b54c-17a0a204a30b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.437951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf65e3-b458-4b6b-b54c-17a0a204a30b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.438247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf65e3-b458-4b6b-b54c-17a0a204a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.444519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.445437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.445874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.446333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.448219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.450639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.467810 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3128/6286] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:29.472046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.490112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032ebd9-6ea3-449e-89a3-9b81a1d68848 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.495669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-13cf65e3-b458-4b6b-b54c-17a0a204a30b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:29.496862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.497342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-13cf65e3-b458-4b6b-b54c-17a0a204a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:29.497732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.498102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.498467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] PATH_INFO: `/v3/projects/a2013d57752240ef928961f5f9b947ee/users/fd7daa7175cb47b8aac2720a89273d1a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.512526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3159/6287] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:29.513505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf65e3-b458-4b6b-b54c-17a0a204a30b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.519456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2013d57752240ef928961f5f9b947ee, user_id=fd7daa7175cb47b8aac2720a89273d1a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.522936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.524206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf65e3-b458-4b6b-b54c-17a0a204a30b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.531877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.531877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.531877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.534546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.535525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:29.535879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.538092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.555252 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3160/6288] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:29.559987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.568193 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3129/6289] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:29 2022] PUT /identity/v3/projects/a2013d57752240ef928961f5f9b947ee/users/fd7daa7175cb47b8aac2720a89273d1a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:29.571121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.583213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd0656-054f-495f-b148-bd382843401c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.652433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:29.687341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.690945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.691233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.691508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.693772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.699018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.719159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ced844-8884-4439-a39c-3271679c344c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.729329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.729979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.730301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.730586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.731632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.732314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:29.734064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.753378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3161/6290] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:29 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:29.759502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.778153 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3130/6291] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:29.797420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.815731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8e14b-c0ab-4a82-9bbd-cd1b36382159 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.884302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af22d64-9db7-42d5-bc87-4228bccecedd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.902376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.902376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.902376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.902376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.902966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.905178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.948170 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3162/6292] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:29 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:29.952454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:29.957058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5e47e-fc70-41de-bbc9-d8c718e71b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.958328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5e47e-fc70-41de-bbc9-d8c718e71b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.958694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5e47e-fc70-41de-bbc9-d8c718e71b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.959070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5e47e-fc70-41de-bbc9-d8c718e71b76 admin admin] PATH_INFO: `/v3/projects/a2013d57752240ef928961f5f9b947ee/users/fd7daa7175cb47b8aac2720a89273d1a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.961867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5e47e-fc70-41de-bbc9-d8c718e71b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2013d57752240ef928961f5f9b947ee, user_id=fd7daa7175cb47b8aac2720a89273d1a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.964863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5e47e-fc70-41de-bbc9-d8c718e71b76 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.967030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdd96-f898-4491-bc9c-843666d6e651 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:29.973630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:29.976153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.976617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.977109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.978030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3131/6293] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:29 2022] PUT /identity/v3/projects/a2013d57752240ef928961f5f9b947ee/users/fd7daa7175cb47b8aac2720a89273d1a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:29.978642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:29.980463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:29.989506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:29.989853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:29.990134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:29.996734 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3163/6294] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:29.998229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:30.011111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903f73c-54f6-4b1d-a2cf-4fcc2deb2fe7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:30.025258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] MFA Rules not processed for user `fd7daa7175cb47b8aac2720a89273d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:30.026394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.085418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:30.086503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.086890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.087267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.088006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:30.089897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:30.101911 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3164/6295] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:30.106218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:30.118552 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3132/6296] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:30.120797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9f739-e40e-4361-b006-b626c8f1db75 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:30.130884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:30.132212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.132841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8334c77-2842-496e-ac09-1e9b0d460ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:30.133537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334c77-2842-496e-ac09-1e9b0d460ece admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.133812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334c77-2842-496e-ac09-1e9b0d460ece admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.134082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334c77-2842-496e-ac09-1e9b0d460ece admin admin] PATH_INFO: `/v3/projects/4f80f3a9dcfe43c6868d300dd1495253/users/4c6f77ddc6a245db9d9b236145c7647f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.152334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334c77-2842-496e-ac09-1e9b0d460ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f80f3a9dcfe43c6868d300dd1495253, user_id=4c6f77ddc6a245db9d9b236145c7647f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:30.155116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334c77-2842-496e-ac09-1e9b0d460ece admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:30.171771 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3165/6297] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:30 2022] PUT /identity/v3/projects/4f80f3a9dcfe43c6868d300dd1495253/users/4c6f77ddc6a245db9d9b236145c7647f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:30.182767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944c40-770f-4e55-8526-a71211403de8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.183152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944c40-770f-4e55-8526-a71211403de8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.183423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944c40-770f-4e55-8526-a71211403de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.185342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:30.202133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0baab-cee4-458c-aa40-b8f59f5374c5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:30.216456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-03944c40-770f-4e55-8526-a71211403de8 None None] MFA Rules not processed for user `4c6f77ddc6a245db9d9b236145c7647f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:30.217943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-03944c40-770f-4e55-8526-a71211403de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.256553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:30.265145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:30.265852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.266131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.266450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.300725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3166/6298] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:30.313091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944c40-770f-4e55-8526-a71211403de8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:30.315217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-03944c40-770f-4e55-8526-a71211403de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.339285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:30.340862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:30.353212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3133/6299] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:20:30.419017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944c40-770f-4e55-8526-a71211403de8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:30.425358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:30.426000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.426281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.426548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.449698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:30.451819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:30.481822 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3167/6300] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:30.512651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:30.606003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930658f-b97d-4c78-b240-9e0ef3656583 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:30.612087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:30.612816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.613201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.613499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] PATH_INFO: `/v3/users/53e8c8b1a46e4d0badb04fdc15765505` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.615064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e8c8b1a46e4d0badb04fdc15765505)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:30.621311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:30.640233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:30.641712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.666729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668529bd-58b2-44fa-a8ad-d07747d94678 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:30.673940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:30.674651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.674932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.675348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] PATH_INFO: `/v3/projects/ef7491906c7f484483794b676660346e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.677623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7491906c7f484483794b676660346e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:30.679723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:30.701482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e8c8b1a46e4d0badb04fdc15765505'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.702300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e8c8b1a46e4d0badb04fdc15765505'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.705284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:30.713908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e8c8b1a46e4d0badb04fdc15765505'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.728346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e8c8b1a46e4d0badb04fdc15765505'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.751690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c70f98a0-55e8-4286-a496-1861457cda1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e8c8b1a46e4d0badb04fdc15765505'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.757008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7491906c7f484483794b676660346e', 'request_id': 'req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.767215 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3134/6301] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:30 2022] DELETE /identity/v3/users/53e8c8b1a46e4d0badb04fdc15765505 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:30.767795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7491906c7f484483794b676660346e', 'request_id': 'req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.769400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.771308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.771608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.780577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7491906c7f484483794b676660346e', 'request_id': 'req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.805537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:30.807152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.808489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] The token cache is being invalidate because project ef7491906c7f484483794b676660346e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:30.809041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.809523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2ce4f8d-db7d-4302-a7b2-8bb24fe25406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:30.814820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3168/6302] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:30 2022] DELETE /identity/v3/projects/ef7491906c7f484483794b676660346e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:30.817214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec512f1-f81e-457e-9633-71300a8bed65 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.817546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec512f1-f81e-457e-9633-71300a8bed65 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.817956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec512f1-f81e-457e-9633-71300a8bed65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:30.845983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1ec512f1-f81e-457e-9633-71300a8bed65 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:30.846854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ec512f1-f81e-457e-9633-71300a8bed65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.902266 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3135/6303] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:30.916120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:30.917404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:30.921348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:30.975605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3169/6304] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:30.981668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42e09f-2bf7-41af-b132-04c82cc3b5e4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:30.981668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42e09f-2bf7-41af-b132-04c82cc3b5e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:30.981668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42e09f-2bf7-41af-b132-04c82cc3b5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.043800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ae42e09f-2bf7-41af-b132-04c82cc3b5e4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:31.052946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ae42e09f-2bf7-41af-b132-04c82cc3b5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:31.080618 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3170/6305] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:31.081897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42e09f-2bf7-41af-b132-04c82cc3b5e4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:31.103766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42e09f-2bf7-41af-b132-04c82cc3b5e4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.108152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289651c-1905-4485-94fd-1315b3264cb8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.116956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.117930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.118727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.120519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.121068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.121307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.121701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.122549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:31.122935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.123728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.128241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:31.146879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.148813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:31.153677 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3171/6306] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:31.159351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:31.175025 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3136/6307] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:31.182350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aabc51-15f0-4ee7-b81d-db72d5668e57 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.197901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.198862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.199237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.199604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.200348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.203231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:31.252314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3172/6308] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:31 2022] POST /identity/v3/projects => generated 361 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:31.257773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:31.272836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4af4f-75b1-447d-88f1-4a44af1814a7 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.373024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.373663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.373947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.374223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.374866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.377011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:31.410250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:31.422629 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3137/6309] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:31 2022] POST /identity/v3/users => generated 281 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:31.424220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:31.440174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145a530-8cce-414a-831a-b794a77d39b9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.447944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.448963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.449486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.449969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.450846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.452714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:31.468841 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3173/6310] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:31.469861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:31.482844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63606a52-08e5-4a9d-97bc-a8a56b2748a9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.488037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71012b-53fe-4502-9308-0776387e6875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.488713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71012b-53fe-4502-9308-0776387e6875 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.488998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71012b-53fe-4502-9308-0776387e6875 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.489456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71012b-53fe-4502-9308-0776387e6875 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.490415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71012b-53fe-4502-9308-0776387e6875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.492014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71012b-53fe-4502-9308-0776387e6875 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:31.508378 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3138/6311] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:31.512224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:31.524510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b525e-2d01-45ed-8404-e0565875f3ec admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.531835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.533114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.533944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.533944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] PATH_INFO: `/v3/projects/536ee6a97d9a49c6b3963bb02e2b0ede/users/8014f80a35df4ba3b8fa8ac9bc803e34/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.555384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536ee6a97d9a49c6b3963bb02e2b0ede, user_id=8014f80a35df4ba3b8fa8ac9bc803e34, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.560702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:31.584791 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3174/6312] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:31 2022] PUT /identity/v3/projects/536ee6a97d9a49c6b3963bb02e2b0ede/users/8014f80a35df4ba3b8fa8ac9bc803e34/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:31.595937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb8e08-44d4-4ec7-9f60-49d7361607c8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.596238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb8e08-44d4-4ec7-9f60-49d7361607c8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.598045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb8e08-44d4-4ec7-9f60-49d7361607c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.655764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a8cb8e08-44d4-4ec7-9f60-49d7361607c8 None None] MFA Rules not processed for user `8014f80a35df4ba3b8fa8ac9bc803e34`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:31.659273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8cb8e08-44d4-4ec7-9f60-49d7361607c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:31.776724 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3139/6313] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:31.788271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:31.900125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544988c-1660-4751-bf4a-573a66d23199 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:31.905963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:31.907038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:31.907476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:31.907920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] PATH_INFO: `/v3/users/8014f80a35df4ba3b8fa8ac9bc803e34` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:31.909276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8014f80a35df4ba3b8fa8ac9bc803e34)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:31.911577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:32.012499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8014f80a35df4ba3b8fa8ac9bc803e34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.013274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8014f80a35df4ba3b8fa8ac9bc803e34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.028489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8014f80a35df4ba3b8fa8ac9bc803e34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.040303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8014f80a35df4ba3b8fa8ac9bc803e34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.041522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8014f80a35df4ba3b8fa8ac9bc803e34'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.048456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:32.071215 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3175/6314] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:31 2022] DELETE /identity/v3/users/8014f80a35df4ba3b8fa8ac9bc803e34 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:32.077686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:32.077936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:32.078177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:32.103379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:32.104650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:32.206874 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3140/6315] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:32.222257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:32.224298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:32.297666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd351ae0-b419-440f-a1ed-0e9b613c953c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:32.303930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:32.304757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:32.305085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:32.306225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:32.317654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:32.319246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:32.330759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3176/6316] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:32.648213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:32.657844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:32.687030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cdb67-60d7-44a8-a431-c41b6f27fe17 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:32.703616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:32.705146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:32.706130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:32.706654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] PATH_INFO: `/v3/projects/e7c268d4e8144356bbd53273e847653e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:32.709480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c268d4e8144356bbd53273e847653e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:32.712395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:32.800935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c268d4e8144356bbd53273e847653e', 'request_id': 'req-0660f576-8617-40f1-aab9-b5de0bc32d2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.808594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c268d4e8144356bbd53273e847653e', 'request_id': 'req-0660f576-8617-40f1-aab9-b5de0bc32d2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.825541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c268d4e8144356bbd53273e847653e', 'request_id': 'req-0660f576-8617-40f1-aab9-b5de0bc32d2d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.848034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] The token cache is being invalidate because project e7c268d4e8144356bbd53273e847653e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:32.848851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.849791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:32.855160 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3141/6317] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:32 2022] DELETE /identity/v3/projects/e7c268d4e8144356bbd53273e847653e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:32.856522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:32.892868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19c924-bd9c-4d7f-9113-9e4673a43803 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:32.998245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:32.998902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:32.999282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:33.000067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] PATH_INFO: `/v3/users/fd7daa7175cb47b8aac2720a89273d1a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:33.002490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7daa7175cb47b8aac2720a89273d1a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:33.005311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:33.088626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7daa7175cb47b8aac2720a89273d1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.090368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7daa7175cb47b8aac2720a89273d1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.104763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7daa7175cb47b8aac2720a89273d1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.117095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7daa7175cb47b8aac2720a89273d1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.118056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f289df4-0ee2-48be-bd62-39ef6234966a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7daa7175cb47b8aac2720a89273d1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.146825 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3177/6318] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:32 2022] DELETE /identity/v3/users/fd7daa7175cb47b8aac2720a89273d1a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:33.183103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:33.184726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:33.194062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:33.205534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660f576-8617-40f1-aab9-b5de0bc32d2d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:33.370688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:33.371836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:33.372361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:33.372891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] PATH_INFO: `/v3/projects/536ee6a97d9a49c6b3963bb02e2b0ede` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:33.374355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] RBAC: Authorizing `identity:delete_project(project_id=536ee6a97d9a49c6b3963bb02e2b0ede)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:33.377142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:33.467792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536ee6a97d9a49c6b3963bb02e2b0ede', 'request_id': 'req-501f7f70-60bc-4bf0-b542-a0149f81734c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.476376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536ee6a97d9a49c6b3963bb02e2b0ede', 'request_id': 'req-501f7f70-60bc-4bf0-b542-a0149f81734c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.497460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536ee6a97d9a49c6b3963bb02e2b0ede', 'request_id': 'req-501f7f70-60bc-4bf0-b542-a0149f81734c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.515402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] The token cache is being invalidate because project 536ee6a97d9a49c6b3963bb02e2b0ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:33.516018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.516960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-501f7f70-60bc-4bf0-b542-a0149f81734c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.521149 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3142/6319] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:33 2022] DELETE /identity/v3/projects/536ee6a97d9a49c6b3963bb02e2b0ede => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:33.524591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:33.525182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:33.525689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:33.578006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:33.580487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:33.601567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:33.723758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3178/6320] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:33.732016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57921418-2699-4051-b79f-027ca9de81e5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:33.732509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57921418-2699-4051-b79f-027ca9de81e5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:33.732788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57921418-2699-4051-b79f-027ca9de81e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:33.762725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:33.764324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:33.775199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-57921418-2699-4051-b79f-027ca9de81e5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:33.776741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-57921418-2699-4051-b79f-027ca9de81e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:33.790563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a1340-612c-494f-8d18-7c85ae52b9ae None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:33.799765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3143/6321] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:33.803963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57921418-2699-4051-b79f-027ca9de81e5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:33.805521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:33.806394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:33.806677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:33.806971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] PATH_INFO: `/v3/projects/a2013d57752240ef928961f5f9b947ee` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:33.808734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2013d57752240ef928961f5f9b947ee)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:33.810685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:33.837035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57921418-2699-4051-b79f-027ca9de81e5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:33.851669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:33.852318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:33.852597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:33.853164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:33.853960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:33.854269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:33.861187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:33.891920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2013d57752240ef928961f5f9b947ee', 'request_id': 'req-fbe816b9-ee4d-438e-ac5b-db84894833ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.904200 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3144/6322] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:20:33.904200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:33.906520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2013d57752240ef928961f5f9b947ee', 'request_id': 'req-fbe816b9-ee4d-438e-ac5b-db84894833ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.908802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:33.916725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2013d57752240ef928961f5f9b947ee', 'request_id': 'req-fbe816b9-ee4d-438e-ac5b-db84894833ba'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.935095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] The token cache is being invalidate because project a2013d57752240ef928961f5f9b947ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:33.935492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.936301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:33.942122 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3179/6323] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:33 2022] DELETE /identity/v3/projects/a2013d57752240ef928961f5f9b947ee => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:33.942122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:33.942122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:33.945435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f4879-6c58-4468-9964-888309a2f16e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.103826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.104493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.104804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.105080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] PATH_INFO: `/v3/users/4c6f77ddc6a245db9d9b236145c7647f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.108958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6f77ddc6a245db9d9b236145c7647f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.112678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.118022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe816b9-ee4d-438e-ac5b-db84894833ba admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.127278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f97639-6cdb-4873-a932-5b0d4169ee25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.128198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f97639-6cdb-4873-a932-5b0d4169ee25 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.128360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f97639-6cdb-4873-a932-5b0d4169ee25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.128543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f97639-6cdb-4873-a932-5b0d4169ee25 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.132820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f97639-6cdb-4873-a932-5b0d4169ee25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.137077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f97639-6cdb-4873-a932-5b0d4169ee25 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.149063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:34.221361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6f77ddc6a245db9d9b236145c7647f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:34.222389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6f77ddc6a245db9d9b236145c7647f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:34.222808 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3180/6324] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:33 2022] POST /identity/v3/projects => generated 373 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:34.229641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94112-cf19-405d-88f1-9d1a36b8acf6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.229933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94112-cf19-405d-88f1-9d1a36b8acf6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.231742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94112-cf19-405d-88f1-9d1a36b8acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.236328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6f77ddc6a245db9d9b236145c7647f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:34.255503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6f77ddc6a245db9d9b236145c7647f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:34.278853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7dd94112-cf19-405d-88f1-9d1a36b8acf6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:34.285133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7dd94112-cf19-405d-88f1-9d1a36b8acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:34.294840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6f77ddc6a245db9d9b236145c7647f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:34.312314 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3145/6325] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:33 2022] DELETE /identity/v3/users/4c6f77ddc6a245db9d9b236145c7647f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:34.318252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.322409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:34.327151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:34.460162 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3181/6326] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:34.466344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5c6ba-ef44-4d3a-be19-9b6f56dd7a96 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.466634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5c6ba-ef44-4d3a-be19-9b6f56dd7a96 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.466911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5c6ba-ef44-4d3a-be19-9b6f56dd7a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.502147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f84b6b-e82c-47c3-a295-c8c255414d26 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.510472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c3b5c6ba-ef44-4d3a-be19-9b6f56dd7a96 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:34.511717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3b5c6ba-ef44-4d3a-be19-9b6f56dd7a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:34.515504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.516143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.516423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.516719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.517505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.519748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.539321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3182/6327] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:34.544570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437571-4173-4316-92db-c0b02b6ccd47 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.546870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437571-4173-4316-92db-c0b02b6ccd47 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.547209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437571-4173-4316-92db-c0b02b6ccd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.587933 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3146/6328] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:34 2022] POST /identity/v3/users => generated 287 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:34.588865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.593255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:34.623739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d1437571-4173-4316-92db-c0b02b6ccd47 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:34.624743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d1437571-4173-4316-92db-c0b02b6ccd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:34.650051 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3183/6329] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:34.659065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1437571-4173-4316-92db-c0b02b6ccd47 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.670204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1437571-4173-4316-92db-c0b02b6ccd47 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.675342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.675975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.676251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.676521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.677350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.679161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.692798 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3184/6330] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:34.693637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.705933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153f363-0e4d-4b4f-89b6-a960a52aaf1e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.716134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.716964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.717249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.717533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.718238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:34.718538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.726915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.754463 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3185/6331] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:34.755728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.779748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a22a0-86a6-410a-9acc-95589ec90ba8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.804093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697301b-352c-45fd-b3fb-21a28d64e4be admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.809682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.810864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.811468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.812867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.813402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.813770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.814809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.814809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.815045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.816066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.832114 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3186/6332] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:34.833946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.848508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.850092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a53de9-c88c-4c98-9865-4565c72d92df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.853393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.857549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:34.858324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:34.858708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:34.859086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:34.859858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:34.862441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:34.883888 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3147/6333] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:34.887128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.905228 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3187/6334] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:34 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:34.909859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:34.919070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974ab0-41cd-4c4b-a3fb-b4787eb64886 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:34.984211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:35.059073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5235e-60f0-4c96-8f5c-1130aceb4db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.060104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5235e-60f0-4c96-8f5c-1130aceb4db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.060235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5235e-60f0-4c96-8f5c-1130aceb4db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.060853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5235e-60f0-4c96-8f5c-1130aceb4db8 admin admin] PATH_INFO: `/v3/projects/53602f1965754e138539682fefb56233/users/2e6e274484624e6bb6cb7d556b43840b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.080136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5235e-60f0-4c96-8f5c-1130aceb4db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53602f1965754e138539682fefb56233, user_id=2e6e274484624e6bb6cb7d556b43840b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.083423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5235e-60f0-4c96-8f5c-1130aceb4db8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.107342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3148/6335] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:34 2022] PUT /identity/v3/projects/53602f1965754e138539682fefb56233/users/2e6e274484624e6bb6cb7d556b43840b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:35.120687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589291ea-84db-4538-8a06-3642a228d42d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.121226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589291ea-84db-4538-8a06-3642a228d42d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.121377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1b4f5-6099-4a9a-930e-bf29cf1bbb2f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.121696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589291ea-84db-4538-8a06-3642a228d42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.181278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-589291ea-84db-4538-8a06-3642a228d42d None None] MFA Rules not processed for user `2e6e274484624e6bb6cb7d556b43840b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:35.182881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-589291ea-84db-4538-8a06-3642a228d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:35.239802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:35.264703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.266723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.267079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.268277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.269447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.287894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.347460 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3188/6336] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:34 2022] POST /identity/v3/users => generated 293 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:35.348536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:35.372389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bf13e-2c06-412a-9032-c313f925ba7d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.374425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3149/6337] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:35.378406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589291ea-84db-4538-8a06-3642a228d42d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:35.393122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.394530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.395076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.395802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.396942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.402715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.403683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589291ea-84db-4538-8a06-3642a228d42d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.410711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.411744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.412116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.412420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.413252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.415186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.446502 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3189/6338] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:35.449560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:35.465439 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3150/6339] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:35 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:35.470632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:35.476278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411214e9-56b8-4846-977c-a74c220cf247 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.559307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669b6a8-03f7-489a-b1ef-8e9382622a05 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.568127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.568842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.569120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.569680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.570489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.573412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.628430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3151/6340] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:35 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:35.629738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:35.630424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.631154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.631531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.631945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.632718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.634514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.653423 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3190/6341] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:35.657650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:35.670995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49955ed1-f0b7-4023-9e56-a7044e60b8a3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.673277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:35.687979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d48b3-d38e-468d-a48b-f8f97077f062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.688876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f451ed-e356-4705-ad2b-c0baac6cfcee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.691008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d48b3-d38e-468d-a48b-f8f97077f062 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.695289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d48b3-d38e-468d-a48b-f8f97077f062 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.695289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d48b3-d38e-468d-a48b-f8f97077f062 admin admin] PATH_INFO: `/v3/projects/02a11eb5a0054a6d9e9ab49b2a46c3cd/users/a2f9da3822fe4334baeb233659613c9d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.705658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.707502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.707800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.708093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.708818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.710475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.743942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d48b3-d38e-468d-a48b-f8f97077f062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a11eb5a0054a6d9e9ab49b2a46c3cd, user_id=a2f9da3822fe4334baeb233659613c9d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:35.748108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d48b3-d38e-468d-a48b-f8f97077f062 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:35.749798 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3152/6342] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:35.759779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:35.791262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3c95a-9fa2-4231-bbe8-9d8c152333fd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:35.798189 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3191/6343] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:35 2022] PUT /identity/v3/projects/02a11eb5a0054a6d9e9ab49b2a46c3cd/users/a2f9da3822fe4334baeb233659613c9d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:35.810682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f9ee-f963-4c8c-8d18-671f7e29123f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.811070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f9ee-f963-4c8c-8d18-671f7e29123f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.811564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f9ee-f963-4c8c-8d18-671f7e29123f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:35.873257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b488f9ee-f963-4c8c-8d18-671f7e29123f None None] MFA Rules not processed for user `a2f9da3822fe4334baeb233659613c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:35.876073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b488f9ee-f963-4c8c-8d18-671f7e29123f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:35.982035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:35.982876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:35.983313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:35.983746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] PATH_INFO: `/v3/projects/8e9be0604dda4dfb9fb981fecab8676b/users/d74331a37a0f4f64872e47639631c69a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.025634 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3192/6344] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:36.029458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9be0604dda4dfb9fb981fecab8676b, user_id=d74331a37a0f4f64872e47639631c69a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.030343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488f9ee-f963-4c8c-8d18-671f7e29123f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.033317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.047168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488f9ee-f963-4c8c-8d18-671f7e29123f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.055612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.057961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.057961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.058262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.059715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.063262 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3153/6345] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:35 2022] PUT /identity/v3/projects/8e9be0604dda4dfb9fb981fecab8676b/users/d74331a37a0f4f64872e47639631c69a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:36.069134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.069600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.121991 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3193/6346] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:36 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:36.126637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.159953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:36.233398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdedcd8-0565-48f1-9639-ceedc1cc18a5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.298807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97663267-a65a-4ce5-a7c2-c80504b62148 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.313445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.314093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.314093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.314093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.315442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.317589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.374654 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3194/6347] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:36 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:36.378623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.413367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c835-06b8-4918-9862-a0cb7d911deb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.415031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.415963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.416796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.417489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.422461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.424398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.431142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.431439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.431716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.433174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.435128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.436680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.460191 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3195/6348] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:36.462802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.473987 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3154/6349] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:36.475503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.496714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:36.510058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a96ada-3938-4f40-92ba-badeca2ceb0e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.513819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15f438-688e-4362-9c88-fb46ddf02546 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.521490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.522327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.522703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.523079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] PATH_INFO: `/v3/projects/4f80f3a9dcfe43c6868d300dd1495253` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.523544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.524508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f80f3a9dcfe43c6868d300dd1495253)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.524868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.525151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.525426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] PATH_INFO: `/v3/projects/9e2f89c7118a44789b369ccd1e08e2e1/users/dd424ff515b04f9f826a358f5fb4f59e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.527523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.568669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2f89c7118a44789b369ccd1e08e2e1, user_id=dd424ff515b04f9f826a358f5fb4f59e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.572890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.584140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:36.614125 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3155/6350] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:36 2022] PUT /identity/v3/projects/9e2f89c7118a44789b369ccd1e08e2e1/users/dd424ff515b04f9f826a358f5fb4f59e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:36.615528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.647493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f80f3a9dcfe43c6868d300dd1495253', 'request_id': 'req-d0287275-c585-4716-b733-f34fe30f885a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:36.680225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f80f3a9dcfe43c6868d300dd1495253', 'request_id': 'req-d0287275-c585-4716-b733-f34fe30f885a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:36.694345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f80f3a9dcfe43c6868d300dd1495253', 'request_id': 'req-d0287275-c585-4716-b733-f34fe30f885a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:36.721375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] The token cache is being invalidate because project 4f80f3a9dcfe43c6868d300dd1495253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:36.721686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:36.722423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:36.729742 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3196/6351] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:36 2022] DELETE /identity/v3/projects/4f80f3a9dcfe43c6868d300dd1495253 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:36.735379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.737494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:36.789958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872796a-934c-42ef-8f37-fe2b63e873b4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.878603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0287275-c585-4716-b733-f34fe30f885a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:36.886046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a98488-cfcd-49c5-98a2-210a49665252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.887891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a98488-cfcd-49c5-98a2-210a49665252 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.887891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a98488-cfcd-49c5-98a2-210a49665252 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.888153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a98488-cfcd-49c5-98a2-210a49665252 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.888715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a98488-cfcd-49c5-98a2-210a49665252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.890777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a98488-cfcd-49c5-98a2-210a49665252 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.912990 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3197/6352] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:36.912990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e0b01-7b76-4102-b657-ef7a0aea95d3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.912990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e0b01-7b76-4102-b657-ef7a0aea95d3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.912990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e0b01-7b76-4102-b657-ef7a0aea95d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.943725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86692d6b-dd8f-4fd8-a772-bf89090e8a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:36.944782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86692d6b-dd8f-4fd8-a772-bf89090e8a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:36.945043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86692d6b-dd8f-4fd8-a772-bf89090e8a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:36.945317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86692d6b-dd8f-4fd8-a772-bf89090e8a0b admin admin] PATH_INFO: `/v3/projects/8e9be0604dda4dfb9fb981fecab8676b/users/d74331a37a0f4f64872e47639631c69a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:36.948061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86692d6b-dd8f-4fd8-a772-bf89090e8a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9be0604dda4dfb9fb981fecab8676b, user_id=d74331a37a0f4f64872e47639631c69a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:36.951178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5d9e0b01-7b76-4102-b657-ef7a0aea95d3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:36.951518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86692d6b-dd8f-4fd8-a772-bf89090e8a0b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:36.952900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5d9e0b01-7b76-4102-b657-ef7a0aea95d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:36.964716 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3198/6353] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:36.966447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e0b01-7b76-4102-b657-ef7a0aea95d3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:36.974727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:36.997620 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3156/6354] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:36 2022] PUT /identity/v3/projects/8e9be0604dda4dfb9fb981fecab8676b/users/d74331a37a0f4f64872e47639631c69a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:36.999390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a7d0c-260a-41a1-9d3b-31d3956c942d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.001836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a7d0c-260a-41a1-9d3b-31d3956c942d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.002216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a7d0c-260a-41a1-9d3b-31d3956c942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.012451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e0b01-7b76-4102-b657-ef7a0aea95d3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:37.053715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f94a7d0c-260a-41a1-9d3b-31d3956c942d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:37.060556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f94a7d0c-260a-41a1-9d3b-31d3956c942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:37.143930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:37.145168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.145168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.145506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] PATH_INFO: `/v3/projects/9e2f89c7118a44789b369ccd1e08e2e1/users/dd424ff515b04f9f826a358f5fb4f59e/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.147531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2f89c7118a44789b369ccd1e08e2e1, user_id=dd424ff515b04f9f826a358f5fb4f59e, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:37.150778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:37.151166 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3157/6355] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:37.152889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020d14b-50a3-418a-bf89-220fd076b61a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.153175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020d14b-50a3-418a-bf89-220fd076b61a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.153449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020d14b-50a3-418a-bf89-220fd076b61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.170730 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3199/6356] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:36 2022] PUT /identity/v3/projects/9e2f89c7118a44789b369ccd1e08e2e1/users/dd424ff515b04f9f826a358f5fb4f59e/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:37.171852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:37.214189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1020d14b-50a3-418a-bf89-220fd076b61a None None] MFA Rules not processed for user `d74331a37a0f4f64872e47639631c69a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:37.216510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1020d14b-50a3-418a-bf89-220fd076b61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:37.263584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:37.296736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436f85a-f798-43dc-a28d-74ddfe314fdd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:37.302557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3b1d6-220f-4287-bac6-1d80ebf574bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:37.303979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3b1d6-220f-4287-bac6-1d80ebf574bb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.305039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3b1d6-220f-4287-bac6-1d80ebf574bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.305914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3b1d6-220f-4287-bac6-1d80ebf574bb admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.306770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3b1d6-220f-4287-bac6-1d80ebf574bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:37.307247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3b1d6-220f-4287-bac6-1d80ebf574bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:37.309056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3b1d6-220f-4287-bac6-1d80ebf574bb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:37.324153 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3200/6357] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:37.331353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9ab8d-77c6-4501-91eb-b53b9acb4cfd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.331637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9ab8d-77c6-4501-91eb-b53b9acb4cfd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.331904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9ab8d-77c6-4501-91eb-b53b9acb4cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.336275 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3158/6358] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:37.337494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020d14b-50a3-418a-bf89-220fd076b61a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:37.353286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020d14b-50a3-418a-bf89-220fd076b61a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:37.360617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:37.361866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.362475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.363421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.364368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:37.367094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:37.383278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-94f9ab8d-77c6-4501-91eb-b53b9acb4cfd None None] MFA Rules not processed for user `dd424ff515b04f9f826a358f5fb4f59e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:37.385014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-94f9ab8d-77c6-4501-91eb-b53b9acb4cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:37.420519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3159/6359] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:37 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:37.422642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:37.427817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:37.499620 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3201/6360] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:37.500480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9ab8d-77c6-4501-91eb-b53b9acb4cfd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:37.513738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9ab8d-77c6-4501-91eb-b53b9acb4cfd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:37.541706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:37.566121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb6ed5-1943-4cdb-bfdc-51553bb000df admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:37.573626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:37.574527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.574806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.575073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.688480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:37.689405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.689778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.690145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.692761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:37.703672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:37.752571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:37.753539 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3202/6361] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:37 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:37.754516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:37.758428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:37.760239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:37.794674 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3160/6362] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:37.811419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:37.861013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6a7f6-834b-48eb-8510-eb7b47b0339b service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:37.889918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:37.891430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.892133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.892863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.894528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:37.899964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3eb4-d069-493e-8285-947e8285779b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:37.901810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:37.907742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:37.908553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:37.908964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:37.909325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:37.937805 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3161/6363] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:37.942003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:37.988946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d2723-f034-40f9-988a-3af3d4026096 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:38.015677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:38.016608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:38.016608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:38.017000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:38.024012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:38.025705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:38.070149 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3162/6364] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:38.070520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:38.072707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:38.081617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:38.113773 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3203/6365] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:38.126041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8252dd2-ac99-4b7c-b64f-3de1ebb4900b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:38.152292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644820d-f7ce-47e4-8173-7c2857b09698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:38.152292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644820d-f7ce-47e4-8173-7c2857b09698 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:38.152292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644820d-f7ce-47e4-8173-7c2857b09698 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:38.152609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644820d-f7ce-47e4-8173-7c2857b09698 admin admin] PATH_INFO: `/v3/projects/f3350348a2fe4bf0a0d8a3128c5e3d25/users/92414887760c4fef9efe19122087d1e9/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:38.228127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644820d-f7ce-47e4-8173-7c2857b09698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3350348a2fe4bf0a0d8a3128c5e3d25, user_id=92414887760c4fef9efe19122087d1e9, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:38.233849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:38.263963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644820d-f7ce-47e4-8173-7c2857b09698 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:38.279633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886b22a-6206-4031-abea-46ab91fe034c service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:38.304460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:38.305796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:38.306430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:38.307017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] PATH_INFO: `/v3/users/a2f9da3822fe4334baeb233659613c9d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:38.308895 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3163/6366] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:38 2022] PUT /identity/v3/projects/f3350348a2fe4bf0a0d8a3128c5e3d25/users/92414887760c4fef9efe19122087d1e9/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:38.309592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f9da3822fe4334baeb233659613c9d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:38.312899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:38.325275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:38.325791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:38.326322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:38.354316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:38.406041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] MFA Rules not processed for user `92414887760c4fef9efe19122087d1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:38.410730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:38.501142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f9da3822fe4334baeb233659613c9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:38.502366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f9da3822fe4334baeb233659613c9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:38.509053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:38.532757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f9da3822fe4334baeb233659613c9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:38.560826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f9da3822fe4334baeb233659613c9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:38.563883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-868d1a6f-00af-4e60-abd7-63e124b12b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f9da3822fe4334baeb233659613c9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:38.608713 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3204/6367] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:38 2022] DELETE /identity/v3/users/a2f9da3822fe4334baeb233659613c9d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:38.613525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:38.623826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:38.624767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:38.635382 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3164/6368] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:38.640132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:38.641596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:38.674896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:38.676497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:38.768564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698e35b-3563-47a2-9f44-85777607a4a4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:38.773342 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3205/6369] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:38.781844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:38.782909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:38.783488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:38.783950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:38.784819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:38.787659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:38.788976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:38.790682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:38.836037 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3165/6370] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:38 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:38.837702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:38.939233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562ea78-b32a-4e99-8622-792c76fc9b98 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:38.988580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b486dd7-ac8d-4b39-948b-4a2864fd1f1a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:38.996202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:38.997126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:38.997687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:38.998260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:38.998819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.001178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:39.068421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3166/6371] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:38 2022] POST /identity/v3/users => generated 294 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:39.078851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:39.109715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:39.110383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.110669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.110944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.126084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcf2aa-1e7b-4f3b-82b7-930970e5baa3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:39.133982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:39.135148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.135277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.135912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.136560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.136867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:39.137839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.141249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:39.145314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:39.164745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3206/6372] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:39.170504 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3167/6373] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:39.171530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:39.192812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b057f-8acd-45f9-a1e3-a8a4ebd43ad6 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:39.203881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:39.205003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.205285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.205925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] PATH_INFO: `/v3/projects/af198a3c22634ba292f2b0365bea36cf/users/ebbe3e02e3d24fbab90a169e72906c6b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.224769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=af198a3c22634ba292f2b0365bea36cf, user_id=ebbe3e02e3d24fbab90a169e72906c6b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.229746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:39.252343 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3207/6374] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:39 2022] PUT /identity/v3/projects/af198a3c22634ba292f2b0365bea36cf/users/ebbe3e02e3d24fbab90a169e72906c6b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:39.258065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:39.276544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24c687-99b6-42fd-b915-512e9e669fd3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:39.381366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fbc34-b5d8-4ac0-91c6-b0c907b447bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:39.382749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fbc34-b5d8-4ac0-91c6-b0c907b447bd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.383748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fbc34-b5d8-4ac0-91c6-b0c907b447bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.384537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fbc34-b5d8-4ac0-91c6-b0c907b447bd admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.385447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fbc34-b5d8-4ac0-91c6-b0c907b447bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.388391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fbc34-b5d8-4ac0-91c6-b0c907b447bd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:39.408277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:39.417150 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3168/6375] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:39.425435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:39.440934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af522051-26a5-4cf0-9998-0aa3e8dd1dad admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:39.453915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:39.455043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.455391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.455780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] PATH_INFO: `/v3/projects/af198a3c22634ba292f2b0365bea36cf/users/ebbe3e02e3d24fbab90a169e72906c6b/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.459786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=af198a3c22634ba292f2b0365bea36cf, user_id=ebbe3e02e3d24fbab90a169e72906c6b, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.462760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:39.478514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3208/6376] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:39 2022] PUT /identity/v3/projects/af198a3c22634ba292f2b0365bea36cf/users/ebbe3e02e3d24fbab90a169e72906c6b/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:39.490104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2563ff-507b-467a-bff1-0d68a247b94d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.490439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2563ff-507b-467a-bff1-0d68a247b94d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.490740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2563ff-507b-467a-bff1-0d68a247b94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.553460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8a2563ff-507b-467a-bff1-0d68a247b94d None None] MFA Rules not processed for user `ebbe3e02e3d24fbab90a169e72906c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:39.556853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a2563ff-507b-467a-bff1-0d68a247b94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:39.723390 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3169/6377] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:39.725927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:39.740543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17269ccf-63ca-4159-9633-7bc6e2d7a6ba admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:39.747367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:39.748084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.748483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.748907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.749649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.751359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:39.779231 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3209/6378] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:20:39 2022] POST /identity/v3/domains => generated 245 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:39.780645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2563ff-507b-467a-bff1-0d68a247b94d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:39.796401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2563ff-507b-467a-bff1-0d68a247b94d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:39.986922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:39.988047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:39.988203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:39.988425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:39.989322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:39.991400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:40.066213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:40.093278 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3170/6379] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:39 2022] POST /identity/v3/projects => generated 407 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:40.094443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:40.127853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d9f68-ea0d-4845-afbe-16e0129f980a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:40.246020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:40.246687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:40.246975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:40.248522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:40.251103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:40.257211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:40.331800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3210/6380] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:40 2022] POST /identity/v3/users => generated 348 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:40.336576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:40.356113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76dc1c-6810-4561-9cf3-c9304a806f1a tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:40.362056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda78997-deb5-4b7f-bc23-c410da4a8f09 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:40.362735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78997-deb5-4b7f-bc23-c410da4a8f09 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:40.363032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78997-deb5-4b7f-bc23-c410da4a8f09 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:40.363342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78997-deb5-4b7f-bc23-c410da4a8f09 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:40.364017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda78997-deb5-4b7f-bc23-c410da4a8f09 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:40.366371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda78997-deb5-4b7f-bc23-c410da4a8f09 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:40.439744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:40.440245 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3171/6381] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:40.461396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53933d10-df09-4534-b059-77a3ff0c9b87 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:40.474723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:40.476648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:40.477027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:40.477315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/projects/88981481bddd4140847ce8ac78548462/users/b306c2ca5b4e4d10914c093edec14b02/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:40.518193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:create_grant(project_id=88981481bddd4140847ce8ac78548462, user_id=b306c2ca5b4e4d10914c093edec14b02, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:40.528561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:40.556572 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3211/6382] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:40 2022] PUT /identity/v3/projects/88981481bddd4140847ce8ac78548462/users/b306c2ca5b4e4d10914c093edec14b02/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:40.560392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:40.560768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:40.560993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:40.627736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:40.629818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:40.816260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] MFA Rules not processed for user `b306c2ca5b4e4d10914c093edec14b02`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:40.817397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:40.835683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54335324-3bb6-4b9d-ab8e-d8b7af03494d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:40.856434 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3172/6383] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:40 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:40.856434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:40.856434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:40.864998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff760744-e484-4d8f-93e7-027a4b37ede3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:40.872957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:40.873916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:40.874384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:40.874843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] PATH_INFO: `/v3/users/377e47048a63456793dfc9c125f5a209` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:40.876246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377e47048a63456793dfc9c125f5a209)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:40.878458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:40.878779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:40.879538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:40.879960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:40.880304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] PATH_INFO: `/v3/projects/02a11eb5a0054a6d9e9ab49b2a46c3cd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:40.881929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a11eb5a0054a6d9e9ab49b2a46c3cd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:40.884192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:40.979671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a11eb5a0054a6d9e9ab49b2a46c3cd', 'request_id': 'req-655563d7-4056-4431-a6a7-5254f3304156'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:40.985276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:40.989590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377e47048a63456793dfc9c125f5a209'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:40.992057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377e47048a63456793dfc9c125f5a209'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:40.998430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a11eb5a0054a6d9e9ab49b2a46c3cd', 'request_id': 'req-655563d7-4056-4431-a6a7-5254f3304156'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.009184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377e47048a63456793dfc9c125f5a209'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.022263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377e47048a63456793dfc9c125f5a209'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.023475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377e47048a63456793dfc9c125f5a209'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.025913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a11eb5a0054a6d9e9ab49b2a46c3cd', 'request_id': 'req-655563d7-4056-4431-a6a7-5254f3304156'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.045254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] The token cache is being invalidate because project 02a11eb5a0054a6d9e9ab49b2a46c3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:41.045551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.048040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655563d7-4056-4431-a6a7-5254f3304156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.049742 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3173/6384] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:40 2022] DELETE /identity/v3/projects/02a11eb5a0054a6d9e9ab49b2a46c3cd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:41.054869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b131b8-066d-4424-9fae-b0c6e3bb2eff None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:41.055082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b131b8-066d-4424-9fae-b0c6e3bb2eff None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:41.055279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b131b8-066d-4424-9fae-b0c6e3bb2eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:41.075031 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3212/6385] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:40 2022] DELETE /identity/v3/users/377e47048a63456793dfc9c125f5a209 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:41.075031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:41.075031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:41.144226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:41.178883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-99b131b8-066d-4424-9fae-b0c6e3bb2eff None None] MFA Rules not processed for user `b306c2ca5b4e4d10914c093edec14b02`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:41.181628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99b131b8-066d-4424-9fae-b0c6e3bb2eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:41.195235 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3174/6386] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:41 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:41.196568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b131b8-066d-4424-9fae-b0c6e3bb2eff None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:41.210299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25acfe1-dc62-4b9e-98f4-f2ac40ea9bf1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:41.210556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b131b8-066d-4424-9fae-b0c6e3bb2eff None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:41.217666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b306c2ca5b4e4d10914c093edec14b02', 'user_domain_id': 'c74169a2bdb9446ab7cf7750df918ade', 'system_scope': None, 'project_id': '88981481bddd4140847ce8ac78548462', 'project_domain_id': 'c74169a2bdb9446ab7cf7750df918ade', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:41.218541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:41.218914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:41.219106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:41.219370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:41.219815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:41.220124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:41.220366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] PATH_INFO: `/v3/users/dd424ff515b04f9f826a358f5fb4f59e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:41.221574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd424ff515b04f9f826a358f5fb4f59e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:41.223528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:41.235184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:41.238634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:41.259209 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3175/6387] 10.222.0.26 () {56 vars in 1284 bytes} [Tue Sep 27 10:20:41 2022] GET /identity/v3/auth/tokens => generated 3360 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:41.266137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:41.273288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:41.329206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd424ff515b04f9f826a358f5fb4f59e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.332690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd424ff515b04f9f826a358f5fb4f59e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.359064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd424ff515b04f9f826a358f5fb4f59e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.374934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd424ff515b04f9f826a358f5fb4f59e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.378201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd424ff515b04f9f826a358f5fb4f59e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.422672 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3213/6388] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:41 2022] DELETE /identity/v3/users/dd424ff515b04f9f826a358f5fb4f59e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:41.440828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e461a2a-0911-4ee1-8d0f-d8f952b72020 tempest-test_project-642684536 tempest-user-1013100320] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:41.540022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:41.541205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:41.544106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:41.544950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:41.545232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:41.545504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/users/b306c2ca5b4e4d10914c093edec14b02` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:41.546665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b306c2ca5b4e4d10914c093edec14b02)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:41.548597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:41.664561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:41.675640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b306c2ca5b4e4d10914c093edec14b02'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.678206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b306c2ca5b4e4d10914c093edec14b02'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.711926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b306c2ca5b4e4d10914c093edec14b02'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.712919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa5a13-c639-45cd-b46f-563c2a68bf0c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:41.722434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:41.723104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:41.723485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:41.723788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] PATH_INFO: `/v3/projects/cb035a91b1f341ce8d8b71d39ff10a0b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:41.725672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb035a91b1f341ce8d8b71d39ff10a0b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:41.728236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b306c2ca5b4e4d10914c093edec14b02'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.728434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:41.748362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:41.768830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b306c2ca5b4e4d10914c093edec14b02'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.785521 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3176/6389] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:41 2022] DELETE /identity/v3/users/b306c2ca5b4e4d10914c093edec14b02 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:41.789358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:41.791430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:41.828855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'request_id': 'req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.849006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'request_id': 'req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.860911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'request_id': 'req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.876943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] The token cache is being invalidate because project cb035a91b1f341ce8d8b71d39ff10a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:41.877485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.878946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a9b6cd7-96c7-4861-888a-6dfb512a3d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:41.881999 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3214/6390] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:41 2022] DELETE /identity/v3/projects/cb035a91b1f341ce8d8b71d39ff10a0b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:41.889787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9504acb-0d6f-4156-87b8-55ff9464fa31 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:41.891262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9504acb-0d6f-4156-87b8-55ff9464fa31 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:41.891952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9504acb-0d6f-4156-87b8-55ff9464fa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:41.932880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818790d-16ab-4bf4-8bdd-7746f848377b tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:41.948346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a9504acb-0d6f-4156-87b8-55ff9464fa31 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:41.952553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a9504acb-0d6f-4156-87b8-55ff9464fa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.063470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.064184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.064469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.064765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/projects/88981481bddd4140847ce8ac78548462` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.068474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:delete_project(project_id=88981481bddd4140847ce8ac78548462)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.073025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.109035 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3215/6391] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:42.120562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69752a56-bed9-4d3d-9553-39b4416d6347 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.120875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69752a56-bed9-4d3d-9553-39b4416d6347 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.121686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69752a56-bed9-4d3d-9553-39b4416d6347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.165939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-69752a56-bed9-4d3d-9553-39b4416d6347 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:42.171261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69752a56-bed9-4d3d-9553-39b4416d6347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.187596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88981481bddd4140847ce8ac78548462', 'request_id': 'req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.196406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88981481bddd4140847ce8ac78548462', 'request_id': 'req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.227386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88981481bddd4140847ce8ac78548462', 'request_id': 'req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.251340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] The token cache is being invalidate because project 88981481bddd4140847ce8ac78548462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:42.251641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.252699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.258634 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3177/6392] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:41 2022] DELETE /identity/v3/projects/88981481bddd4140847ce8ac78548462 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:42.263546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.268201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.332499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:42.340170 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3216/6393] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:42.345855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69752a56-bed9-4d3d-9553-39b4416d6347 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.369011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69752a56-bed9-4d3d-9553-39b4416d6347 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:42.375862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.377042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.377593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.377875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.378765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:42.379059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.381076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.389379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46faa9ae-1eca-4dca-86ba-7fbbe3e8383d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:42.392008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:42.399622 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3217/6394] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:42.399622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.401558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.404696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.405597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.405892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.406175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/domains/c74169a2bdb9446ab7cf7750df918ade` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.408248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c74169a2bdb9446ab7cf7750df918ade)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.409889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.420320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f893f34-86dc-4c4a-8e3b-8ed9adc3031f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:42.426535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.427596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.427970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.428475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] PATH_INFO: `/v3/users/2e6e274484624e6bb6cb7d556b43840b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.429763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6e274484624e6bb6cb7d556b43840b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.432275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.435091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.436440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.476743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.477528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.480984 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3178/6395] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:20:42 2022] PATCH /identity/v3/domains/c74169a2bdb9446ab7cf7750df918ade => generated 246 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:42.481877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.483311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.494679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6e274484624e6bb6cb7d556b43840b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.495898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6e274484624e6bb6cb7d556b43840b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.509705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6e274484624e6bb6cb7d556b43840b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.522958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6e274484624e6bb6cb7d556b43840b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.524795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6e274484624e6bb6cb7d556b43840b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.554207 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3218/6396] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:42 2022] DELETE /identity/v3/users/2e6e274484624e6bb6cb7d556b43840b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:42.555637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.557432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.596624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5739ce-82e4-4f8b-88f2-aea210453831 tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:42.664955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61aa884-0464-4866-87f8-ccf572880183 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:42.671905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebbe3e02e3d24fbab90a169e72906c6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af198a3c22634ba292f2b0365bea36cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.672574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.672958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.673153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] PATH_INFO: `/v3/domains/c74169a2bdb9446ab7cf7750df918ade` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.690785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c74169a2bdb9446ab7cf7750df918ade)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.692641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.710451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75932f-ca4b-4956-ac67-8405cdd3e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.711387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75932f-ca4b-4956-ac67-8405cdd3e65d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.711832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75932f-ca4b-4956-ac67-8405cdd3e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.712411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75932f-ca4b-4956-ac67-8405cdd3e65d admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.713280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75932f-ca4b-4956-ac67-8405cdd3e65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.715652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75932f-ca4b-4956-ac67-8405cdd3e65d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.725587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.743299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.764448 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3179/6397] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:42 2022] POST /identity/v3/projects => generated 379 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:42.767623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.768115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.768847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.789572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:42.795298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:42.796784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.799145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade', 'request_id': 'req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.810522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade', 'request_id': 'req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.818839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade', 'request_id': 'req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.821454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:42.836862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] The token cache is being invalidate because project c74169a2bdb9446ab7cf7750df918ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:42.837151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.837951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.847339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade', 'request_id': 'req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.848150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade', 'request_id': 'req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.853662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c74169a2bdb9446ab7cf7750df918ade', 'request_id': 'req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:42.859698 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3219/6398] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:20:42 2022] DELETE /identity/v3/domains/c74169a2bdb9446ab7cf7750df918ade => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:42.860482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.864557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.872875 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3180/6399] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:42.877027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.878524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:42.889546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c7ddf-aae2-4b8a-8bad-433f9ce95a9d tempest-TestDefaultProjectId-168527496 tempest-TestDefaultProjectId-168527496-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:42.894342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.895113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.895399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.895708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.896348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.897748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd101323-adc9-4c15-92ac-608f59531388 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:42.898608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.903337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:42.904068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:42.904382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:42.904766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] PATH_INFO: `/v3/users/92414887760c4fef9efe19122087d1e9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:42.905951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92414887760c4fef9efe19122087d1e9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:42.914933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:42.929920 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3220/6400] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:42 2022] POST /identity/v3/users => generated 290 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:42.931058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:42.933191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.000280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92414887760c4fef9efe19122087d1e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.001280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92414887760c4fef9efe19122087d1e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.010235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92414887760c4fef9efe19122087d1e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.021046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ba7fa-1af6-4d37-9232-1658252f9941 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:43.021348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92414887760c4fef9efe19122087d1e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.025234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:43.025838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.026102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.026384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.028102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.040775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92414887760c4fef9efe19122087d1e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.057393 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3181/6401] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:42 2022] DELETE /identity/v3/users/92414887760c4fef9efe19122087d1e9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:43.058949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:43.060874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.097214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:43.098346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:43.133854 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3221/6402] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:43.137942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:43.144003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.160493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d37fd2-16f2-4809-bdc6-b9f0c0991ac5 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:43.176337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e542-b5c8-4a81-b0fa-228307183e09 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:43.190797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad826-7892-4a54-824d-9175c5549328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:43.191551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad826-7892-4a54-824d-9175c5549328 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.192044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad826-7892-4a54-824d-9175c5549328 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.192469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad826-7892-4a54-824d-9175c5549328 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.193768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ad826-7892-4a54-824d-9175c5549328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:43.196336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ad826-7892-4a54-824d-9175c5549328 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:43.215430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3182/6403] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:43.217089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55de9-cb10-4e82-9e4d-5422911006e3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.217477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55de9-cb10-4e82-9e4d-5422911006e3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.217935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55de9-cb10-4e82-9e4d-5422911006e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.268310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:43.272018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-edb55de9-cb10-4e82-9e4d-5422911006e3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:43.273432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-edb55de9-cb10-4e82-9e4d-5422911006e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.308148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:43.308769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.308959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:43.309177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.309528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] PATH_INFO: `/v3/projects/9e2f89c7118a44789b369ccd1e08e2e1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.311009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e2f89c7118a44789b369ccd1e08e2e1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:43.313326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:43.316765 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3183/6404] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:43.316765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55de9-cb10-4e82-9e4d-5422911006e3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:43.341320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55de9-cb10-4e82-9e4d-5422911006e3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:43.366504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:43.368619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.368935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.369680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.371314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:43.377500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:43.398802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e2f89c7118a44789b369ccd1e08e2e1', 'request_id': 'req-a4ed846f-cddf-4e21-b211-717c21c6d8ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.404931 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3184/6405] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:43.405817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:43.412045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.429134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e2f89c7118a44789b369ccd1e08e2e1', 'request_id': 'req-a4ed846f-cddf-4e21-b211-717c21c6d8ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.439204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e2f89c7118a44789b369ccd1e08e2e1', 'request_id': 'req-a4ed846f-cddf-4e21-b211-717c21c6d8ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.471991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] The token cache is being invalidate because project 9e2f89c7118a44789b369ccd1e08e2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:43.472274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.474628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:43.477788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3222/6406] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:43 2022] DELETE /identity/v3/projects/9e2f89c7118a44789b369ccd1e08e2e1 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:43.484558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:43.491970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.529665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230018c-e10a-47ac-be0e-93fbd3aecbdb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:43.618248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:43.619088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.619953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.619953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.622234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.652167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed846f-cddf-4e21-b211-717c21c6d8ce admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:43.667317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95868e61-4eff-4839-898b-0b5ee4002f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:43.668859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868e61-4eff-4839-898b-0b5ee4002f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.669338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868e61-4eff-4839-898b-0b5ee4002f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.669774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868e61-4eff-4839-898b-0b5ee4002f1c admin admin] PATH_INFO: `/v3/projects/10375bd836f84383b39f2df73f53800e/users/97c769dbba794531b0a9db8e764ff097/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.701802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95868e61-4eff-4839-898b-0b5ee4002f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10375bd836f84383b39f2df73f53800e, user_id=97c769dbba794531b0a9db8e764ff097, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:43.705724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95868e61-4eff-4839-898b-0b5ee4002f1c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:43.713072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:43.714910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:43.732854 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3223/6407] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:43 2022] PUT /identity/v3/projects/10375bd836f84383b39f2df73f53800e/users/97c769dbba794531b0a9db8e764ff097/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:43.734430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eff1c-b851-4ebd-8360-ef558706738f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.735062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eff1c-b851-4ebd-8360-ef558706738f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.735496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eff1c-b851-4ebd-8360-ef558706738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.741559 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3185/6408] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:43.742436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:43.746232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.769295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1c2eff1c-b851-4ebd-8360-ef558706738f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:43.770612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1c2eff1c-b851-4ebd-8360-ef558706738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.787031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db56c9-168f-43fe-bfe4-261f2acdf940 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:43.877399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:43.886452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:43.903146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:43.904676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.904980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.905323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] PATH_INFO: `/v3/projects/53602f1965754e138539682fefb56233` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.906636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53602f1965754e138539682fefb56233)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:43.907424 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3224/6409] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:43.908742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:43.909251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:43.909710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:43.912856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:43.957028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:43.969167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:43.998448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53602f1965754e138539682fefb56233', 'request_id': 'req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.007542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53602f1965754e138539682fefb56233', 'request_id': 'req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.035196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53602f1965754e138539682fefb56233', 'request_id': 'req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.074292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] The token cache is being invalidate because project 53602f1965754e138539682fefb56233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:44.074585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.076401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe49572f-02a2-4ec5-b19e-84fd1b0cae45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.084921 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3186/6410] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:43 2022] DELETE /identity/v3/projects/53602f1965754e138539682fefb56233 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:44.085904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.087008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.087924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.140582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:44.148000 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3225/6411] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:44.148000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.149304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.154094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.171838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3187/6412] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:44.175795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.175795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.181016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49cddb-6b71-42b2-ba28-b9e942f8307f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.187220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.188626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.189321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.190043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] PATH_INFO: `/v3/users/d74331a37a0f4f64872e47639631c69a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.192057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74331a37a0f4f64872e47639631c69a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.194834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.287318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74331a37a0f4f64872e47639631c69a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.288577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74331a37a0f4f64872e47639631c69a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.302723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74331a37a0f4f64872e47639631c69a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.314352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74331a37a0f4f64872e47639631c69a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.315219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74331a37a0f4f64872e47639631c69a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.326180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc34165-8902-489b-8d55-9c3c554aab89 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.337442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.338082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.338359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.338630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.340428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.341266 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3226/6413] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:44 2022] DELETE /identity/v3/users/d74331a37a0f4f64872e47639631c69a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:44.342628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.344237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.374397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.376055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.403375 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3188/6414] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:44.453952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b459ed-6083-45b5-8c5b-205083383bba admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.462811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.465073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.465786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.466341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.467296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:44.467971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.470888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.489927 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3227/6415] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:44.491995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.492753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.495642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.504425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:44.515695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcfe2e-5dbe-44bc-a7e9-94e08a79af50 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.517923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73016c8e-56b7-4324-bbd3-ce315366c928 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.528588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.529243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.529515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.529777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.530411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.532713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.533660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.534328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.534593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.534874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] PATH_INFO: `/v3/projects/f3350348a2fe4bf0a0d8a3128c5e3d25` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.543395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3350348a2fe4bf0a0d8a3128c5e3d25)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.545462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.638659 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3228/6416] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:44 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:44.639771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.652484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3350348a2fe4bf0a0d8a3128c5e3d25', 'request_id': 'req-7505cffb-7fe0-4de3-9271-62f5437112f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.678375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3350348a2fe4bf0a0d8a3128c5e3d25', 'request_id': 'req-7505cffb-7fe0-4de3-9271-62f5437112f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.700822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3350348a2fe4bf0a0d8a3128c5e3d25', 'request_id': 'req-7505cffb-7fe0-4de3-9271-62f5437112f1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.722070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] The token cache is being invalidate because project f3350348a2fe4bf0a0d8a3128c5e3d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:44.722389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.725309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:44.727710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3189/6417] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:44 2022] DELETE /identity/v3/projects/f3350348a2fe4bf0a0d8a3128c5e3d25 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:44.731918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.735927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.754182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:44.779568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe4086-93bb-4c61-86ed-03f96361378b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.790800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.792546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.793074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.793358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.794008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.796036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.856100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3229/6418] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:44 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:44.858580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.861748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:44.907739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9184522-58fa-4a05-90f2-594411322c2a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.915456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.916362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.916847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.917292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.919805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.922105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.940349 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3230/6419] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:44.941067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505cffb-7fe0-4de3-9271-62f5437112f1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:44.944905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:44.959061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:44.959836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:44.960181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:44.960581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] PATH_INFO: `/v3/users/ebbe3e02e3d24fbab90a169e72906c6b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:44.963143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbe3e02e3d24fbab90a169e72906c6b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:44.970156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:44.995548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe2cea-1847-4bd6-8647-c832a2116cfe admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:45.018145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:45.019945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.020863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.023074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.023916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:45.027537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:45.064072 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3231/6420] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:45.065565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:45.113640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03492e-3145-4b5f-bd5d-5ab08d09c893 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:45.132765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:45.133840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.134716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.135381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbe3e02e3d24fbab90a169e72906c6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.135974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] PATH_INFO: `/v3/projects/de7e33ac0a8c4e029cf13103283c86d4/users/443371b3885344b0a0e90c355ae1f0e0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.136406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbe3e02e3d24fbab90a169e72906c6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.170387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbe3e02e3d24fbab90a169e72906c6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.186741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7e33ac0a8c4e029cf13103283c86d4, user_id=443371b3885344b0a0e90c355ae1f0e0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:45.196390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbe3e02e3d24fbab90a169e72906c6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.198608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:45.238808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4935797a-4df0-43f4-831d-a12bbe6ec551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbe3e02e3d24fbab90a169e72906c6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.240706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3232/6421] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:45 2022] PUT /identity/v3/projects/de7e33ac0a8c4e029cf13103283c86d4/users/443371b3885344b0a0e90c355ae1f0e0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:45.243257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:45.246796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:45.253812 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3190/6422] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:44 2022] DELETE /identity/v3/users/ebbe3e02e3d24fbab90a169e72906c6b => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:45.258673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088060b2-143b-4d39-8151-d322002733ad None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.259207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088060b2-143b-4d39-8151-d322002733ad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.259497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088060b2-143b-4d39-8151-d322002733ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.313224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-088060b2-143b-4d39-8151-d322002733ad None None] MFA Rules not processed for user `443371b3885344b0a0e90c355ae1f0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:45.314939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-088060b2-143b-4d39-8151-d322002733ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:45.449086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659b3a-e8f9-46ee-b0da-c931be03e00a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:45.457945 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3191/6423] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:45.477702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088060b2-143b-4d39-8151-d322002733ad None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:45.484245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-088060b2-143b-4d39-8151-d322002733ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:45.497805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:45.515106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:45.597146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:45.598084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.598489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.598881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] PATH_INFO: `/v3/projects/8e9be0604dda4dfb9fb981fecab8676b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.601489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9be0604dda4dfb9fb981fecab8676b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:45.605122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:45.609156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088060b2-143b-4d39-8151-d322002733ad None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:45.615150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:45.616335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.616697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.617022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.617899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:45.620038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:45.716894 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3192/6424] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:45 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:45.720884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:45.736820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9be0604dda4dfb9fb981fecab8676b', 'request_id': 'req-af33bdb1-15bf-4a75-ad0a-d5770d04373d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.755519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9be0604dda4dfb9fb981fecab8676b', 'request_id': 'req-af33bdb1-15bf-4a75-ad0a-d5770d04373d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.763119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9be0604dda4dfb9fb981fecab8676b', 'request_id': 'req-af33bdb1-15bf-4a75-ad0a-d5770d04373d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.785037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] The token cache is being invalidate because project 8e9be0604dda4dfb9fb981fecab8676b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:45.787106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.788800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-af33bdb1-15bf-4a75-ad0a-d5770d04373d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:45.792389 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3233/6425] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:45 2022] DELETE /identity/v3/projects/8e9be0604dda4dfb9fb981fecab8676b => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:45.797497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f78491-da20-4fe0-9a26-55372e2f1167 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.801422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f78491-da20-4fe0-9a26-55372e2f1167 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.802243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f78491-da20-4fe0-9a26-55372e2f1167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.815400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874b690-377c-4842-aaa6-d31c6e4a4cda admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:45.828436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:45.829177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.829528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.830339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.830998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:45.837570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:45.883885 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3193/6426] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:45 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:45.888209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:45.893522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:45.924615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049e898-8bee-464c-949b-090c7cf8de3b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:45.932598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:45.933253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.933527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.933853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.934288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e4f78491-da20-4fe0-9a26-55372e2f1167 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:45.934446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:45.935783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e4f78491-da20-4fe0-9a26-55372e2f1167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:45.937628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:45.954548 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3234/6427] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:45.965571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89996f34-1f9d-4117-9924-755499abb494 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:45.965966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89996f34-1f9d-4117-9924-755499abb494 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:45.966334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89996f34-1f9d-4117-9924-755499abb494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:45.977425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3194/6428] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:45.977425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:46.002825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f42f51-f6cd-405a-8c26-a938913fc849 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:46.023403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:46.024358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.026127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.026945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] PATH_INFO: `/v3/projects/4865e0f487654ee7bf0e37d2a619004c/users/e29781b9c9814085bdb9a01adfea4244/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.031369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-89996f34-1f9d-4117-9924-755499abb494 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:46.038922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-89996f34-1f9d-4117-9924-755499abb494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:46.069705 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3235/6429] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:46.071051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89996f34-1f9d-4117-9924-755499abb494 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:46.081539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4865e0f487654ee7bf0e37d2a619004c, user_id=e29781b9c9814085bdb9a01adfea4244, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:46.085327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:46.089614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89996f34-1f9d-4117-9924-755499abb494 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:46.103680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:46.104534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.104854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.105161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.108348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:46.108725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:46.117853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:46.118880 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3195/6430] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:45 2022] PUT /identity/v3/projects/4865e0f487654ee7bf0e37d2a619004c/users/e29781b9c9814085bdb9a01adfea4244/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:46.124421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:46.162962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3236/6431] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:46.165265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:46.190236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd5d5-b4d7-4916-acac-8e19c9e086f6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:46.260025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:46.307399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:46.312815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc14617-9c1d-447e-82ec-3846e48341c2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:46.317433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:46.318391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.318838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.319314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.320597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:46.322873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:46.325399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.325858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.326347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.326772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:46.327477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:46.330947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:46.364906 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3196/6432] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:46.366121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:46.375851 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3237/6433] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:46 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:46.380873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:46.400171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9bc8f-c55f-4c89-92cf-1d466bce8f1c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:46.556106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:46.556106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.556106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.557237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] PATH_INFO: `/v3/projects/4865e0f487654ee7bf0e37d2a619004c/users/e29781b9c9814085bdb9a01adfea4244/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.557237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4865e0f487654ee7bf0e37d2a619004c, user_id=e29781b9c9814085bdb9a01adfea4244, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:46.566020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:46.589070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3197/6434] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:46 2022] PUT /identity/v3/projects/4865e0f487654ee7bf0e37d2a619004c/users/e29781b9c9814085bdb9a01adfea4244/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:46.590250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:46.592393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:46.602650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d6531-99a7-4e00-aaaa-52c76ea5b416 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:46.784965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8d20f-96d1-4d51-a88c-447bcbf3a7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:46.785891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8d20f-96d1-4d51-a88c-447bcbf3a7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.786483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8d20f-96d1-4d51-a88c-447bcbf3a7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.788052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8d20f-96d1-4d51-a88c-447bcbf3a7e9 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.791835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8d20f-96d1-4d51-a88c-447bcbf3a7e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:46.794136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8d20f-96d1-4d51-a88c-447bcbf3a7e9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:46.805974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0336b-b15d-4ac7-a112-6e1ab9e135cc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:46.813379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:46.814172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.815297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.819254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] PATH_INFO: `/v3/projects/af198a3c22634ba292f2b0365bea36cf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.821797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af198a3c22634ba292f2b0365bea36cf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:46.824313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:46.870538 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3238/6435] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:46 2022] POST /identity/v3/users => generated 286 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:46.875744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b92ed9-1c30-465c-8141-a47553fbdc04 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:46.876335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b92ed9-1c30-465c-8141-a47553fbdc04 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:46.878379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b92ed9-1c30-465c-8141-a47553fbdc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:46.930121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-62b92ed9-1c30-465c-8141-a47553fbdc04 None None] MFA Rules not processed for user `e29781b9c9814085bdb9a01adfea4244`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:46.932214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-62b92ed9-1c30-465c-8141-a47553fbdc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:46.952274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af198a3c22634ba292f2b0365bea36cf', 'request_id': 'req-72060348-f2f6-4174-8f86-6f0e7559ccb0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:46.967090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af198a3c22634ba292f2b0365bea36cf', 'request_id': 'req-72060348-f2f6-4174-8f86-6f0e7559ccb0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:46.991350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af198a3c22634ba292f2b0365bea36cf', 'request_id': 'req-72060348-f2f6-4174-8f86-6f0e7559ccb0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:47.020141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] The token cache is being invalidate because project af198a3c22634ba292f2b0365bea36cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:47.020504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:47.021411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:47.029699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3198/6436] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:46 2022] DELETE /identity/v3/projects/af198a3c22634ba292f2b0365bea36cf => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:47.032997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.034477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:47.068684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:47.140514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3239/6437] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:47.142945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.151409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.151788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.168911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:47.198058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:47.199826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:47.210153 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3240/6438] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:47.210389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060348-f2f6-4174-8f86-6f0e7559ccb0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:47.211503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.215977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:47.230711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb031b-0510-40fc-a97b-88f5d0ef7bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:47.246613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb031b-0510-40fc-a97b-88f5d0ef7bdc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.246977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb031b-0510-40fc-a97b-88f5d0ef7bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.247187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb031b-0510-40fc-a97b-88f5d0ef7bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.247920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb031b-0510-40fc-a97b-88f5d0ef7bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:47.258785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb031b-0510-40fc-a97b-88f5d0ef7bdc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:47.312551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43dbb6-0984-455f-999c-1a60c4479d5f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:47.319759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:47.321322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.321740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.322189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.323296 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3199/6439] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:47.326645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699fd3-148e-422d-91bf-f886d331804f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.329426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699fd3-148e-422d-91bf-f886d331804f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.334476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699fd3-148e-422d-91bf-f886d331804f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.337024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:47.339031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:47.360595 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3241/6440] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:47.361549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.387396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c5699fd3-148e-422d-91bf-f886d331804f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:47.388945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c5699fd3-148e-422d-91bf-f886d331804f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:47.393176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eba8e-63dd-4222-8bb5-e7c29919f2c7 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:47.408218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:47.409548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.410364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.412852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.413838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:47.417931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:47.439410 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3200/6441] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:47.445109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699fd3-148e-422d-91bf-f886d331804f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.452733 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3242/6442] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:47.458284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.476404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699fd3-148e-422d-91bf-f886d331804f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:47.479089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d8a16-651d-46f1-ade6-f38b5017d588 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:47.495693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:47.496553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.497064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.497400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] PATH_INFO: `/v3/projects/28da452e9e064d948584b9f54a14d430/users/41df6767ffca45ddac9242a0d942b0cc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.498350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:47.499493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.501142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.503824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.504628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:47.505497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:47.508121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:47.526357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28da452e9e064d948584b9f54a14d430, user_id=41df6767ffca45ddac9242a0d942b0cc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:47.532813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:47.541302 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3201/6443] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:47.543069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.569494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3243/6444] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:47 2022] PUT /identity/v3/projects/28da452e9e064d948584b9f54a14d430/users/41df6767ffca45ddac9242a0d942b0cc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:47.570481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.571753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa155-6d7a-4be1-9050-27bf7e0e7583 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:47.720730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4871d-c227-473a-9561-2c5f938fa6e8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:47.728528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:47.729312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.729707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.730050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.733633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:47.737837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399751d9-5ef3-47b5-899a-1bb722ee6bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:47.738582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399751d9-5ef3-47b5-899a-1bb722ee6bce admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.739624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399751d9-5ef3-47b5-899a-1bb722ee6bce admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.740107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399751d9-5ef3-47b5-899a-1bb722ee6bce admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.742225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399751d9-5ef3-47b5-899a-1bb722ee6bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:47.745653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399751d9-5ef3-47b5-899a-1bb722ee6bce admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:47.807439 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3244/6445] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:47 2022] POST /identity/v3/projects => generated 371 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:47.810726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed318a-7860-48be-90a1-2d88937c39f5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:47.811019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed318a-7860-48be-90a1-2d88937c39f5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:47.812629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed318a-7860-48be-90a1-2d88937c39f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:47.873532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:47.882859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:47.906436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ceed318a-7860-48be-90a1-2d88937c39f5 None None] MFA Rules not processed for user `41df6767ffca45ddac9242a0d942b0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:47.908103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ceed318a-7860-48be-90a1-2d88937c39f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:47.913781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3202/6446] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:47.915335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:47.946902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4794b2-6c38-45ab-be06-45c70c7e86ed service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.040855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:48.084874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:48.112934 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3245/6447] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:48.118220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.118871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.119196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.119406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.119552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed318a-7860-48be-90a1-2d88937c39f5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:48.121402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.123329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.155826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed318a-7860-48be-90a1-2d88937c39f5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.165218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.166499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.167019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.167559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.168490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.171152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.203671 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3203/6448] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:47 2022] POST /identity/v3/users => generated 286 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:48.206641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:48.224184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d29b71-2037-4b9e-8f32-76d7c1e53953 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.231691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.232629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.233116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.233381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.234083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.237151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.241477 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3246/6449] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:48 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:48.245023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:48.266815 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3204/6450] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:48.271077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:48.300575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079439-a7fe-4630-b9e3-223b7c5e5d28 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.423676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923d612-b3b8-4990-a50a-1e69d0375d26 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.445662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.446563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.446992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.447355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.449883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.451451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ab211-e596-469e-86c0-521df245834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.452317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ab211-e596-469e-86c0-521df245834c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.453064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ab211-e596-469e-86c0-521df245834c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.454139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ab211-e596-469e-86c0-521df245834c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.455322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ab211-e596-469e-86c0-521df245834c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.456005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.457641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ab211-e596-469e-86c0-521df245834c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.479441 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3205/6451] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:48.484811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ab211-e596-469e-86c0-521df245834c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:48.519648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ab211-e596-469e-86c0-521df245834c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.527923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.528481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.528804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.529082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] PATH_INFO: `/v3/projects/7768163646a1434c98c2b97cefa3e314/users/8946bf0664764117851dc8fc2a1dacf9/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.561805 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3247/6452] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:48 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:48.563136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:48.566880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:48.569753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7768163646a1434c98c2b97cefa3e314, user_id=8946bf0664764117851dc8fc2a1dacf9, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.573059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.614095 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3206/6453] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:48 2022] PUT /identity/v3/projects/7768163646a1434c98c2b97cefa3e314/users/8946bf0664764117851dc8fc2a1dacf9/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:48.615822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:48.620831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d39234-4ac6-4f93-a8a4-2e80edd355fa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.778951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231b56a-1723-4282-a29a-742438362935 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:48.796816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff99257-e5dc-4ed0-8671-37cea2535cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.799667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff99257-e5dc-4ed0-8671-37cea2535cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.799962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff99257-e5dc-4ed0-8671-37cea2535cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.800243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff99257-e5dc-4ed0-8671-37cea2535cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.801943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff99257-e5dc-4ed0-8671-37cea2535cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.802390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:48.803747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.806316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.807007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] PATH_INFO: `/v3/users/443371b3885344b0a0e90c355ae1f0e0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.807506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff99257-e5dc-4ed0-8671-37cea2535cf6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.810808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] RBAC: Authorizing `identity:delete_user(user_id=443371b3885344b0a0e90c355ae1f0e0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:48.814056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:48.845543 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3207/6454] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:48.846975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e2ec0-2457-42a6-9f0d-0cbeb23290a2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:48.847353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e2ec0-2457-42a6-9f0d-0cbeb23290a2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:48.847576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e2ec0-2457-42a6-9f0d-0cbeb23290a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:48.890411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f48e2ec0-2457-42a6-9f0d-0cbeb23290a2 None None] MFA Rules not processed for user `8946bf0664764117851dc8fc2a1dacf9`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:48.893590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f48e2ec0-2457-42a6-9f0d-0cbeb23290a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:48.949545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:48.962191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443371b3885344b0a0e90c355ae1f0e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:48.963591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443371b3885344b0a0e90c355ae1f0e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:48.965784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:48.986132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443371b3885344b0a0e90c355ae1f0e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:49.003940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443371b3885344b0a0e90c355ae1f0e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:49.005207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443371b3885344b0a0e90c355ae1f0e0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:49.055269 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3248/6455] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:48 2022] DELETE /identity/v3/users/443371b3885344b0a0e90c355ae1f0e0 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:49.056771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.060842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:49.061662 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3208/6456] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:49.063591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.064079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.065911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.100188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:49.102453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:49.229646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf95f-1583-47ae-a50b-730b2d30318a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.229951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3209/6457] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:49.230970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.235788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:49.244646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.245644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.246078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.246518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] PATH_INFO: `/v3/projects/499a5d95ab7e4f088a23ccfabccb61a0/users/10dd231166ca47948063da19afbdf84c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.260090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f39eb5-f5ea-4c83-87ce-dab8d814ab03 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.271284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.272016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.272391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.272813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.273553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.276141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:49.295168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] RBAC: Authorizing `identity:create_grant(project_id=499a5d95ab7e4f088a23ccfabccb61a0, user_id=10dd231166ca47948063da19afbdf84c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.300695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:49.317853 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3210/6458] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:49 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:49.319167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.324147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:49.325820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3249/6459] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:49 2022] PUT /identity/v3/projects/499a5d95ab7e4f088a23ccfabccb61a0/users/10dd231166ca47948063da19afbdf84c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:49.328938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.449443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25984346-5066-4a43-bd2e-f9f0fd7df42d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.483732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1c5f4-62b2-4dc6-8543-fe9a57145aee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.487943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7698c-2427-469a-97f7-f4caf2dd9848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.488712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7698c-2427-469a-97f7-f4caf2dd9848 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.489085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7698c-2427-469a-97f7-f4caf2dd9848 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.489654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7698c-2427-469a-97f7-f4caf2dd9848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.530371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:49.540204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.541871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.542906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.543148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.543841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.547149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:49.587706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:49.590851 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3250/6460] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:49 2022] POST /identity/v3/users => generated 291 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:49.596443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.606114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7698c-2427-469a-97f7-f4caf2dd9848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.610938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7698c-2427-469a-97f7-f4caf2dd9848 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:49.623210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda8482-cf08-4a93-87e7-5e078dc0a554 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.632752 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3211/6461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:49.635094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e343f80-7f70-4589-ad79-8fd828714d7b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.635377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e343f80-7f70-4589-ad79-8fd828714d7b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.635700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e343f80-7f70-4589-ad79-8fd828714d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.635812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.636325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.636604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.636900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.637696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.639320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:49.660050 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3251/6462] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:49.662131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.673726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3e343f80-7f70-4589-ad79-8fd828714d7b None None] MFA Rules not processed for user `1d92f7d2ee2d47beaa4f5d43ac40cd82`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:49.675603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3e343f80-7f70-4589-ad79-8fd828714d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:49.716975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad719e1-fb38-49b2-85d5-aa5ff0bffd4c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.723754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.725215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.725513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.725799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.726434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.728439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:49.752526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3252/6463] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:49.753993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.774645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835c3e4-f768-4552-bc95-2cfcf11bd3d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.780774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c64115-c0c3-4f2d-a79e-82d05a93016f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.781876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64115-c0c3-4f2d-a79e-82d05a93016f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.782412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64115-c0c3-4f2d-a79e-82d05a93016f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.782826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64115-c0c3-4f2d-a79e-82d05a93016f admin admin] PATH_INFO: `/v3/projects/499a5d95ab7e4f088a23ccfabccb61a0/users/10dd231166ca47948063da19afbdf84c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.785808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c64115-c0c3-4f2d-a79e-82d05a93016f admin admin] RBAC: Authorizing `identity:create_grant(project_id=499a5d95ab7e4f088a23ccfabccb61a0, user_id=10dd231166ca47948063da19afbdf84c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.787743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3212/6464] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:49.788864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e343f80-7f70-4589-ad79-8fd828714d7b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:49.795976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c64115-c0c3-4f2d-a79e-82d05a93016f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:49.812055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3253/6465] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:49 2022] PUT /identity/v3/projects/499a5d95ab7e4f088a23ccfabccb61a0/users/10dd231166ca47948063da19afbdf84c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:49.829264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae4ce1-ebba-4356-b095-347be2f2bdb3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.829550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae4ce1-ebba-4356-b095-347be2f2bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.830460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae4ce1-ebba-4356-b095-347be2f2bdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.834713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e343f80-7f70-4589-ad79-8fd828714d7b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:49.879939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-39ae4ce1-ebba-4356-b095-347be2f2bdb3 None None] MFA Rules not processed for user `10dd231166ca47948063da19afbdf84c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:49.881746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-39ae4ce1-ebba-4356-b095-347be2f2bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:49.953602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:49.955394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:49.956135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:49.956432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] PATH_INFO: `/v3/projects/a14606f1df0e430eac371e73dc956fda/users/bd080356b1d742159770a7d094181e0d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:49.978565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14606f1df0e430eac371e73dc956fda, user_id=bd080356b1d742159770a7d094181e0d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:49.981577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.002629 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3254/6466] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:50.005748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae4ce1-ebba-4356-b095-347be2f2bdb3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.019135 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3213/6467] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:49 2022] PUT /identity/v3/projects/a14606f1df0e430eac371e73dc956fda/users/bd080356b1d742159770a7d094181e0d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:50.029049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.036791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae4ce1-ebba-4356-b095-347be2f2bdb3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.166555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd512d9-431e-433a-813f-15b1d5b2dc3d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.172902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.173541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.173823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.174101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.174878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.176906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.197562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10dd231166ca47948063da19afbdf84c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '499a5d95ab7e4f088a23ccfabccb61a0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.198556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.198885 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3214/6468] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:50.199402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.199742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.200391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:50.200777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.202546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.203436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.219035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5ce8d-1433-4315-8faa-c20437660575 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.220848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3255/6469] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:50.224679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.233101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29819ed0-251f-4ea8-8923-5c8757744774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.234139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29819ed0-251f-4ea8-8923-5c8757744774 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.234502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29819ed0-251f-4ea8-8923-5c8757744774 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.234851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29819ed0-251f-4ea8-8923-5c8757744774 admin admin] PATH_INFO: `/v3/projects/a14606f1df0e430eac371e73dc956fda/users/bd080356b1d742159770a7d094181e0d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.236327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54110-f02c-47ba-b859-dfdd5067f036 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.237267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29819ed0-251f-4ea8-8923-5c8757744774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14606f1df0e430eac371e73dc956fda, user_id=bd080356b1d742159770a7d094181e0d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.243290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29819ed0-251f-4ea8-8923-5c8757744774 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.245261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10dd231166ca47948063da19afbdf84c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '499a5d95ab7e4f088a23ccfabccb61a0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.246003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.246350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.246879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.247608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.250363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.252246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:50.268725 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3215/6470] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:50 2022] PUT /identity/v3/projects/a14606f1df0e430eac371e73dc956fda/users/bd080356b1d742159770a7d094181e0d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:50.292156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4feac0-93ab-4629-87a1-a8eb6dbbcf5a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.292986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4feac0-93ab-4629-87a1-a8eb6dbbcf5a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.293347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4feac0-93ab-4629-87a1-a8eb6dbbcf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.307575 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3256/6471] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:50 2022] POST /identity/v3/projects => generated 350 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:50.309146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.331677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:50.338163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-be4feac0-93ab-4629-87a1-a8eb6dbbcf5a None None] MFA Rules not processed for user `bd080356b1d742159770a7d094181e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:50.339784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-be4feac0-93ab-4629-87a1-a8eb6dbbcf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:50.443554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d694b4b-e6a4-4c55-89f9-caa6b790cea8 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.457115 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3216/6472] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:50.463440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10dd231166ca47948063da19afbdf84c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '499a5d95ab7e4f088a23ccfabccb61a0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.464151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.464580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.464880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.465156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4feac0-93ab-4629-87a1-a8eb6dbbcf5a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.465544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:50.465851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.467472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.481694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4feac0-93ab-4629-87a1-a8eb6dbbcf5a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.487348 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3257/6473] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:50.493511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.495123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.496116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.496943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.497968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.500895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.503398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.543900 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3217/6474] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:50 2022] POST /identity/v3/services => generated 289 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:50.549737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.622856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dccb3-2019-4ec0-b64d-63af6a8b3fb2 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.625862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d443ea-1e03-4ff0-bb3e-53796b56684e tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.629900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.631373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.631664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.631944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.632775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.633933 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:50.633933 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:20:50.633933 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:20:50.634900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.655808 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3218/6475] 10.222.0.26 () {60 vars in 1213 bytes} [Tue Sep 27 10:20:50 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:50.660048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.685466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.685926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657e9dc-ffc5-4540-a36a-d837dba3cdc8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.686279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.686604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.686829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.700569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.701556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.702102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.703062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.703980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.706412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.708471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.709960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.736404 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3219/6476] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:50 2022] POST /identity/v3/services => generated 287 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:50.740762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.757370 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3258/6477] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:50 2022] GET /identity/v3/auth/tokens => generated 3583 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:50.781570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a1da1-457f-4932-a5e7-4b3ba28e9e39 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.790979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.793722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.796902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.798332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.801541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.808022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.832773 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3220/6478] 10.222.0.26 () {60 vars in 1213 bytes} [Tue Sep 27 10:20:50 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:20:50.837192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.848694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19a349-cb13-457b-9480-ffb6da7ea379 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.855157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.855533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.855811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.856085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.856746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.857769 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:50.857769 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:20:50.857769 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:20:50.858913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.870676 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3259/6479] 10.222.0.26 () {58 vars in 1191 bytes} [Tue Sep 27 10:20:50 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 764 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:50.871585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.888553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff895-c603-43a1-be30-c68c84567ff8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.897345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.898851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.899326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.900254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.901399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=1fba272c6447490da9001dd38de3b18a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.902635 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:50.902635 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:20:50.902635 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:20:50.904141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.917211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.924977 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3221/6480] 10.222.0.26 () {58 vars in 1290 bytes} [Tue Sep 27 10:20:50 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a => generated 333 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:50.930529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805492a-dcce-4791-ac1f-b134d639a4b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.937512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.938330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.938674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.939031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.939793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=1fba272c6447490da9001dd38de3b18a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.941908 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:50.941908 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:20:50.941908 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:20:50.942725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.955760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:50.962383 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3260/6481] 10.222.0.26 () {58 vars in 1291 bytes} [Tue Sep 27 10:20:50 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a => generated 0 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:50.968496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e688212-b362-4d19-8984-ee094f23625f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:50.974380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:50.975258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:50.975547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:50.975828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:50.976745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=1fba272c6447490da9001dd38de3b18a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:50.977960 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:50.977960 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:20:50.977960 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:20:50.978894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:50.999836 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3222/6482] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:20:50 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:51.001882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.038855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8593c-60c8-4874-b14c-c8a62a2e8f4e tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.049801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.051431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.051723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.052092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.052958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.071482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.082917 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3261/6483] 10.222.0.26 () {58 vars in 1191 bytes} [Tue Sep 27 10:20:51 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:51.083571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.100254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b5c59-6a13-43be-ad2c-f389ad833fff tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.112073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.112904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.113257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.113606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.114378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=1fba272c6447490da9001dd38de3b18a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.120141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Could not find Endpoint Group: 1fba272c6447490da9001dd38de3b18a.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 1fba272c6447490da9001dd38de3b18a. Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Sep 27 10:20:51.147621 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Sep 27 10:20:51.149134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Sep 27 10:20:51.149134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Sep 27 10:20:51.149134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Sep 27 10:20:51.149134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Sep 27 10:20:51.149134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 1fba272c6447490da9001dd38de3b18a. Sep 27 10:20:51.149134 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:20:51.150450 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3223/6484] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:20:51 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/1fba272c6447490da9001dd38de3b18a => generated 120 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:20:51.155862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.177879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6581c-5a07-4845-afa0-260d37dafc6f tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.220086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.220804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.221087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.221361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services/99b5888a25ca4674aa9908babb2e38bc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.221955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_service(service_id=99b5888a25ca4674aa9908babb2e38bc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.223584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.283508 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3262/6485] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:51 2022] DELETE /identity/v3/services/99b5888a25ca4674aa9908babb2e38bc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:51.289643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.303685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b08ead-6939-406c-a4e6-b72df7935d45 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.341751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.342940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.344895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.345428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.347150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.349188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.380738 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3224/6486] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:51 2022] POST /identity/v3/services => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:51.383011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.400054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7a0be-bd64-403b-bd88-09e1ee519a01 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.423178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.425050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:51.459351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.460266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.461384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.462099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.463049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.465030 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:51.465030 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:20:51.465030 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:20:51.466334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.491639 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3263/6487] 10.222.0.26 () {60 vars in 1213 bytes} [Tue Sep 27 10:20:51 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 332 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:51.494896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.502952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:51.516101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f2a0b-7ebf-42e5-918a-2c2ef667c828 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.523800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.524706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.528714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.529043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.529736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.531530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.568722 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3264/6488] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:51 2022] POST /identity/v3/services => generated 289 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:51.573064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.579473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91224ef9-59ee-4dc2-b2dd-d22c966ee645 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.585441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:51.604080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.604828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.605438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.605715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] PATH_INFO: `/v3/projects/de7e33ac0a8c4e029cf13103283c86d4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.611026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7e33ac0a8c4e029cf13103283c86d4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.613061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.622187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4429d-aff2-4ff5-9bb7-462c71519421 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.635850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.636619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.637019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.637311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/12cbddb6939648da9d0af0b48d7b3874` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.638787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=12cbddb6939648da9d0af0b48d7b3874)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.641772 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:51.641772 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:20:51.641772 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:20:51.643261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.675890 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3265/6489] 10.222.0.26 () {60 vars in 1313 bytes} [Tue Sep 27 10:20:51 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/12cbddb6939648da9d0af0b48d7b3874 => generated 335 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:20:51.681107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.694736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff9792-7277-4dbe-a4dd-5d96335669ad tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.712556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7e33ac0a8c4e029cf13103283c86d4', 'request_id': 'req-efa41b8c-57f1-412e-b7bc-ce877e0db159'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:51.732112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7e33ac0a8c4e029cf13103283c86d4', 'request_id': 'req-efa41b8c-57f1-412e-b7bc-ce877e0db159'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:51.760840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7e33ac0a8c4e029cf13103283c86d4', 'request_id': 'req-efa41b8c-57f1-412e-b7bc-ce877e0db159'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:51.785920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] The token cache is being invalidate because project de7e33ac0a8c4e029cf13103283c86d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:51.786315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:51.787303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:51.792843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3225/6490] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:51 2022] DELETE /identity/v3/projects/de7e33ac0a8c4e029cf13103283c86d4 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:51.794561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.797181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:51.853082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.855976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.856349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.856767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services/4cc3e05d37004505811ceece77bb8c53` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.858227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_service(service_id=4cc3e05d37004505811ceece77bb8c53)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.861285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:51.916364 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3266/6491] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:51 2022] DELETE /identity/v3/services/4cc3e05d37004505811ceece77bb8c53 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:51.918232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:51.925738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:51.976333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b1caf-3d83-45fd-8f61-a7377639c371 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:51.982762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:51.983606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:51.983911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:51.984182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/12cbddb6939648da9d0af0b48d7b3874` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:51.984857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=12cbddb6939648da9d0af0b48d7b3874)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:51.985955 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:20:51.985955 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:20:51.985955 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:20:51.986920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:52.003849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3267/6492] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:20:51 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/12cbddb6939648da9d0af0b48d7b3874 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:52.012503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:52.015241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41b8c-57f1-412e-b7bc-ce877e0db159 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:52.042039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6124d4-8536-4011-9ec5-d5fae6087da3 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:52.061531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:52.064317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:52.064651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:52.066063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services/4d27dd11a70940b28f30dc8462ab08ff` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:52.068145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_service(service_id=4d27dd11a70940b28f30dc8462ab08ff)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:52.072070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:52.075195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:52.078905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:52.079091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:52.079189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] PATH_INFO: `/v3/users/e29781b9c9814085bdb9a01adfea4244` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:52.082888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e29781b9c9814085bdb9a01adfea4244)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:52.090078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:52.140388 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3268/6493] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:52 2022] DELETE /identity/v3/services/4d27dd11a70940b28f30dc8462ab08ff => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:52.146155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:52.195785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f70c2-6bf1-476b-8f31-3309b13bff62 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:52.208701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:52.221383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:52.222456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:52.225784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:52.226453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/2870d8d8085c431cbcb22d2adb1f7079` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:52.229107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=2870d8d8085c431cbcb22d2adb1f7079)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:52.233743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:52.234374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e29781b9c9814085bdb9a01adfea4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.235679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e29781b9c9814085bdb9a01adfea4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.259714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e29781b9c9814085bdb9a01adfea4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.268330 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3269/6494] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:20:52 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/2870d8d8085c431cbcb22d2adb1f7079 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:52.274607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:52.278711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:52.285458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e29781b9c9814085bdb9a01adfea4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.324645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958c640-5be0-46d6-b2f1-136984e119b8 tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:52.330344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e29781b9c9814085bdb9a01adfea4244'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.338753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7163cc4-31ec-4e68-955c-59d67258481c tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd080356b1d742159770a7d094181e0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a14606f1df0e430eac371e73dc956fda', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:52.340071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163cc4-31ec-4e68-955c-59d67258481c tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:52.340361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163cc4-31ec-4e68-955c-59d67258481c tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:52.340646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163cc4-31ec-4e68-955c-59d67258481c tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] PATH_INFO: `/v3/services/ea3b5bab27ac4c33a76043c52350d57c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:52.341273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7163cc4-31ec-4e68-955c-59d67258481c tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorizing `identity:delete_service(service_id=ea3b5bab27ac4c33a76043c52350d57c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:52.345258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7163cc4-31ec-4e68-955c-59d67258481c tempest-EndPointGroupsTest-70020262 tempest-EndPointGroupsTest-70020262-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:52.348207 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3226/6495] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:51 2022] DELETE /identity/v3/users/e29781b9c9814085bdb9a01adfea4244 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:52.392469 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3270/6496] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:52 2022] DELETE /identity/v3/services/ea3b5bab27ac4c33a76043c52350d57c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:52.397178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:52.399175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:52.509392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f240ad-3f52-4a97-a102-99f6a529d8ef admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:52.542855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:52.543884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:52.544246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:52.544549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] PATH_INFO: `/v3/users/8946bf0664764117851dc8fc2a1dacf9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:52.545974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8946bf0664764117851dc8fc2a1dacf9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:52.548452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:52.685233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8946bf0664764117851dc8fc2a1dacf9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.687904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8946bf0664764117851dc8fc2a1dacf9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.700985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8946bf0664764117851dc8fc2a1dacf9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.723848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8946bf0664764117851dc8fc2a1dacf9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.753932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8946bf0664764117851dc8fc2a1dacf9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:52.765183 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3227/6497] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:52 2022] DELETE /identity/v3/users/8946bf0664764117851dc8fc2a1dacf9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:52.770941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:52.771230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:52.771536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:52.811023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:52.812868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:52.923328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:UPDATE=1,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:52.979653 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3271/6498] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:53.144232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:53.147120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:53.226794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:53.375171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad3f20-1622-426f-8632-a4bb20fba457 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:53.382428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:53.383449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:53.383947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:53.385140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:53.400750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:53.403156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:53.419015 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3228/6499] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:54.275226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:54.276847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:54.310100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7f51d-ab69-4d74-a8bb-2dd360652b66 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:54.315108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:54.315954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:54.316241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:54.316519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] PATH_INFO: `/v3/projects/7768163646a1434c98c2b97cefa3e314` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:54.317808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7768163646a1434c98c2b97cefa3e314)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:54.319749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:54.430194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7768163646a1434c98c2b97cefa3e314', 'request_id': 'req-b3feb239-ba1c-418d-831a-1fa07365dff5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.447332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7768163646a1434c98c2b97cefa3e314', 'request_id': 'req-b3feb239-ba1c-418d-831a-1fa07365dff5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.478245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7768163646a1434c98c2b97cefa3e314', 'request_id': 'req-b3feb239-ba1c-418d-831a-1fa07365dff5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.495828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] The token cache is being invalidate because project 7768163646a1434c98c2b97cefa3e314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:54.496436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.497372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.499197 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3272/6500] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:54 2022] DELETE /identity/v3/projects/7768163646a1434c98c2b97cefa3e314 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:54.503267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:54.514554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bbd20-cf51-4601-b668-b6e8cbe2d29d service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:54.628880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:54.629535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:54.629700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:54.630044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] PATH_INFO: `/v3/users/bd080356b1d742159770a7d094181e0d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:54.631307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd080356b1d742159770a7d094181e0d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:54.633567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:54.702365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:54.705285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:54.732468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3feb239-ba1c-418d-831a-1fa07365dff5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:54.742372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:54.743405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:54.743803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:54.744387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] PATH_INFO: `/v3/projects/4865e0f487654ee7bf0e37d2a619004c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:54.746809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4865e0f487654ee7bf0e37d2a619004c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:54.749373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:54.800485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd080356b1d742159770a7d094181e0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.802064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd080356b1d742159770a7d094181e0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.811733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd080356b1d742159770a7d094181e0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.824146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd080356b1d742159770a7d094181e0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.853672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b4968d6e-554d-459c-833e-6261450aecee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd080356b1d742159770a7d094181e0d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.864183 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3229/6501] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:54 2022] DELETE /identity/v3/users/bd080356b1d742159770a7d094181e0d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:54.874442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4865e0f487654ee7bf0e37d2a619004c', 'request_id': 'req-2d473241-4d55-409f-b35f-2ff5455ce80f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.890216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4865e0f487654ee7bf0e37d2a619004c', 'request_id': 'req-2d473241-4d55-409f-b35f-2ff5455ce80f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.901493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4865e0f487654ee7bf0e37d2a619004c', 'request_id': 'req-2d473241-4d55-409f-b35f-2ff5455ce80f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.959314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] The token cache is being invalidate because project 4865e0f487654ee7bf0e37d2a619004c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:54.959657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.960399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d473241-4d55-409f-b35f-2ff5455ce80f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:54.967685 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3273/6502] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:54 2022] DELETE /identity/v3/projects/4865e0f487654ee7bf0e37d2a619004c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:54.968212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfa9be-12b9-4e65-923c-dfeed0a21246 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:54.968510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfa9be-12b9-4e65-923c-dfeed0a21246 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:54.968817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfa9be-12b9-4e65-923c-dfeed0a21246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.005818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ffbfa9be-12b9-4e65-923c-dfeed0a21246 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:55.007110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ffbfa9be-12b9-4e65-923c-dfeed0a21246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:55.023993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:55.164383 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3230/6503] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:55.167315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdcc33-5d44-48dc-be15-3c928752efb4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.167603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdcc33-5d44-48dc-be15-3c928752efb4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.167876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdcc33-5d44-48dc-be15-3c928752efb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.213223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5ccdcc33-5d44-48dc-be15-3c928752efb4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:55.214508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5ccdcc33-5d44-48dc-be15-3c928752efb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:55.238117 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3274/6504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:55.240019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfa9be-12b9-4e65-923c-dfeed0a21246 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:55.270117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfa9be-12b9-4e65-923c-dfeed0a21246 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:55.307035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:55.308185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.308185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.308447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.309122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:55.309622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:55.312102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:55.335316 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3231/6505] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:55.336891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdcc33-5d44-48dc-be15-3c928752efb4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:55.366756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdcc33-5d44-48dc-be15-3c928752efb4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:55.379363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:55.383385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.383791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.384073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.384930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:55.387329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:55.430497 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3275/6506] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:55 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:55.434785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:55.446926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc9d93-d013-48d0-9d97-29a6c3ce5007 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:55.517684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:55.518709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.518803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.519073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.520093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:55.522039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:55.561325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:55.573909 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3232/6507] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:55 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:55.578299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b252db-6f85-4406-a4f9-ce5a008ade93 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:55.624379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:55.628523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.628944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.629315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.631865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:55.644256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:55.670570 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3276/6508] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:55.675164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:55.691080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716958e1-9fb6-4e2a-995e-ae8be614bda6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:55.697560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b579a6-501c-4fcb-ad13-22761f6f28d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:55.697805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b579a6-501c-4fcb-ad13-22761f6f28d5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.698311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b579a6-501c-4fcb-ad13-22761f6f28d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.698842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b579a6-501c-4fcb-ad13-22761f6f28d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.699714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b579a6-501c-4fcb-ad13-22761f6f28d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:55.701536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b579a6-501c-4fcb-ad13-22761f6f28d5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:55.724266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:55.724730 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3233/6509] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:55.740458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:55.752466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b70ef-5c23-42e8-b728-402a63f5067b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:55.761686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:55.764914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.765468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.766710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] PATH_INFO: `/v3/projects/140e3033364a498c9752cae22f3108a6/users/1be0da58001e4b71976eaf46bd246b2d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.805300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=140e3033364a498c9752cae22f3108a6, user_id=1be0da58001e4b71976eaf46bd246b2d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:55.810327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:55.848803 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3277/6510] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:55 2022] PUT /identity/v3/projects/140e3033364a498c9752cae22f3108a6/users/1be0da58001e4b71976eaf46bd246b2d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:55.856852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f5960-4b0a-451c-b4d5-1d7142b5367c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:55.857440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f5960-4b0a-451c-b4d5-1d7142b5367c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:55.857943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f5960-4b0a-451c-b4d5-1d7142b5367c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:55.916763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8e9f5960-4b0a-451c-b4d5-1d7142b5367c None None] MFA Rules not processed for user `1be0da58001e4b71976eaf46bd246b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:55.918264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8e9f5960-4b0a-451c-b4d5-1d7142b5367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:56.022287 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3234/6511] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:56.022535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:56.201921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde78d0-20ec-42d4-bc5d-cf74e8b4f18f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:56.214088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:56.215345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:56.216090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:56.216941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:56.217801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:56.220099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:56.259477 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3278/6512] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:56 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:56.264095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f5960-4b0a-451c-b4d5-1d7142b5367c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:56.277568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f5960-4b0a-451c-b4d5-1d7142b5367c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:56.281692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:56.372932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:56.375297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:56.393527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:56.393946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:56.400498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:56.400498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:56.401242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:56.415962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:56.494941 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3235/6513] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:56 2022] POST /identity/v3/users => generated 296 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:56.496261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:56.520062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c918-35ff-47eb-a670-de859471eb18 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:56.530774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:56.532825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:56.533205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:56.533572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:56.534913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:56.538641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:56.569067 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3236/6514] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:56.571895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69063925-0871-4ee2-b2b3-10acc3491031 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:56.578064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:56.592747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:56.593769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:56.594359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:56.595058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] PATH_INFO: `/v3/projects/a14606f1df0e430eac371e73dc956fda` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:56.596489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14606f1df0e430eac371e73dc956fda)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:56.601315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:56.607635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7680175-ed0a-4fcf-a85e-3baa99a50cbc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:56.620880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:56.621578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:56.621856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:56.622130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] PATH_INFO: `/v3/projects/ad7ae413ffc64977ab4ea87bc0362d76/users/7229e538689e42ed9b1f22e6464979ed/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:56.666401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7ae413ffc64977ab4ea87bc0362d76, user_id=7229e538689e42ed9b1f22e6464979ed, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:56.668481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:56.700435 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3237/6515] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:56 2022] PUT /identity/v3/projects/ad7ae413ffc64977ab4ea87bc0362d76/users/7229e538689e42ed9b1f22e6464979ed/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:56.702411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:56.746190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14606f1df0e430eac371e73dc956fda', 'request_id': 'req-2daab805-529f-4ab9-a3f7-2436c914e1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:56.762667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14606f1df0e430eac371e73dc956fda', 'request_id': 'req-2daab805-529f-4ab9-a3f7-2436c914e1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:56.774692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14606f1df0e430eac371e73dc956fda', 'request_id': 'req-2daab805-529f-4ab9-a3f7-2436c914e1f6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:56.776161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:56.801079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] The token cache is being invalidate because project a14606f1df0e430eac371e73dc956fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:56.801663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:56.802588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2daab805-529f-4ab9-a3f7-2436c914e1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:56.804462 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3279/6516] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:56 2022] DELETE /identity/v3/projects/a14606f1df0e430eac371e73dc956fda => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:56.809977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1ee2-93e9-4541-97c8-b67c9be690ab None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:56.810266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1ee2-93e9-4541-97c8-b67c9be690ab None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:56.810547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1ee2-93e9-4541-97c8-b67c9be690ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:56.860488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-806a1ee2-93e9-4541-97c8-b67c9be690ab None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:56.861853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-806a1ee2-93e9-4541-97c8-b67c9be690ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:56.872326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721893-6ea4-4dbc-9394-47c7c9529341 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.011053 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3280/6517] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:57.022182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b29eba-54d0-4ba2-8189-fd550655730b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.023801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b29eba-54d0-4ba2-8189-fd550655730b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.024567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b29eba-54d0-4ba2-8189-fd550655730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.048991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.049686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.049971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.050276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.050898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.054709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.091330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c9b29eba-54d0-4ba2-8189-fd550655730b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:57.095558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c9b29eba-54d0-4ba2-8189-fd550655730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:57.100242 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3238/6518] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:57.101248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.109782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:57.124065 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3281/6519] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:57.128958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b29eba-54d0-4ba2-8189-fd550655730b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.132787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:57.136449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1abe0c-a917-4380-b35b-f0620a1b8bd8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.142500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b29eba-54d0-4ba2-8189-fd550655730b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.147509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1973ee0-11dd-4403-aae7-7847b200b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.148202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1973ee0-11dd-4403-aae7-7847b200b3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.148479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1973ee0-11dd-4403-aae7-7847b200b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.148792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1973ee0-11dd-4403-aae7-7847b200b3f3 admin admin] PATH_INFO: `/v3/projects/ad7ae413ffc64977ab4ea87bc0362d76/users/7229e538689e42ed9b1f22e6464979ed/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.149742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.151025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.151394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.152047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.153766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:20:57.154149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.154864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1973ee0-11dd-4403-aae7-7847b200b3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7ae413ffc64977ab4ea87bc0362d76, user_id=7229e538689e42ed9b1f22e6464979ed, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.155954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.159725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1973ee0-11dd-4403-aae7-7847b200b3f3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.177991 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3239/6520] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:57 2022] PUT /identity/v3/projects/ad7ae413ffc64977ab4ea87bc0362d76/users/7229e538689e42ed9b1f22e6464979ed/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:57.178413 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3282/6521] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:20:57.185379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.200442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b308fa-e618-4cc2-989e-d46eefff0b29 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.302356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.302613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.302937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.335243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] MFA Rules not processed for user `7229e538689e42ed9b1f22e6464979ed`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:57.336498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:57.337600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.338816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.340761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.341284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.342141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.348114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.418025 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3283/6522] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:57 2022] POST /identity/v3/projects => generated 373 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:57.421538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.505333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3240/6523] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:57.512357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.513833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:57.557554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24d88f-08a0-4295-9c26-123dbdad1292 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.564411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.565798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.566999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.566999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.567264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.570264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.581809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:57.609472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3284/6524] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:57 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:57.610700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.625354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab517886-e23b-4f7b-8ba9-082b2dd174d2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.634294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.634930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.635363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.635644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.636297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.638013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.652568 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3285/6525] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:57.656817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.671512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd02ad-1c7b-419a-a4b0-7a1e1f7a5966 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.673673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cc0bc-458e-4b6e-9951-1cea8dc08bec admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.679630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.680256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.681032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.681032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.681628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.684098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.700164 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3286/6526] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:20:57.705287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.718288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8890cd-d05e-4dc9-8e4c-5db1616d27c7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:57.727769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1dba0-39e0-42be-9f90-36b7721821a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.728926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1dba0-39e0-42be-9f90-36b7721821a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.729293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1dba0-39e0-42be-9f90-36b7721821a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.729628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1dba0-39e0-42be-9f90-36b7721821a7 admin admin] PATH_INFO: `/v3/projects/25330fd966d24b0d8d79ce82837f7c48/users/63690bb54cc54cc2a02ab8002e437939/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.748077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1dba0-39e0-42be-9f90-36b7721821a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25330fd966d24b0d8d79ce82837f7c48, user_id=63690bb54cc54cc2a02ab8002e437939, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.750639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:57.751374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.751708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1dba0-39e0-42be-9f90-36b7721821a7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.751870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.752212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.754438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:57.772546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3287/6527] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:57 2022] PUT /identity/v3/projects/25330fd966d24b0d8d79ce82837f7c48/users/63690bb54cc54cc2a02ab8002e437939/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:20:57.783280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1345-1d69-4124-9ae6-79164853f228 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.783280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1345-1d69-4124-9ae6-79164853f228 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.783656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1345-1d69-4124-9ae6-79164853f228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.822831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5b7a1345-1d69-4124-9ae6-79164853f228 None None] MFA Rules not processed for user `63690bb54cc54cc2a02ab8002e437939`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:57.824537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5b7a1345-1d69-4124-9ae6-79164853f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:57.877856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:57.930271 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3288/6528] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:20:57.939072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:57.939661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:57.940946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:57.941326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:57.945296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:57.964180 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3241/6529] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:57.967643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:57.998829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e6147-880b-4fc7-b2f0-3a16ab49a666 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:58.017667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] MFA Rules not processed for user `97c769dbba794531b0a9db8e764ff097`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:58.022686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:58.214510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:58.216013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:58.217118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:58.218155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:58.220160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:58.223496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:58.266088 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3289/6530] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:58.267455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:58.269543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:58.307522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3242/6531] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:20:57 2022] POST /identity/v3/projects => generated 373 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:58.313795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:58.328390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:58.369704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:58.524453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e657-5e2f-4263-960b-95d30c881745 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:58.638829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10dd231166ca47948063da19afbdf84c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '499a5d95ab7e4f088a23ccfabccb61a0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:58.640183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:58.640616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:58.641280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] PATH_INFO: `/v3/projects/0aa9fc0d0dfb4fa3821969c10135c532` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:58.643597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0aa9fc0d0dfb4fa3821969c10135c532)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:58.646525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:58.680762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386f99-374c-429d-9896-8daaf04c18f5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:58.700118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:58.705283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:58.705751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:58.706102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:58.749936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa9fc0d0dfb4fa3821969c10135c532', 'request_id': 'req-18a80456-1822-4085-bb76-308dc617e072'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:58.765816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa9fc0d0dfb4fa3821969c10135c532', 'request_id': 'req-18a80456-1822-4085-bb76-308dc617e072'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:58.782900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa9fc0d0dfb4fa3821969c10135c532', 'request_id': 'req-18a80456-1822-4085-bb76-308dc617e072'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:58.800139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:58.813817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] The token cache is being invalidate because project 0aa9fc0d0dfb4fa3821969c10135c532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:20:58.814124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:58.816643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:58.819951 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3290/6532] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:20:58 2022] DELETE /identity/v3/projects/0aa9fc0d0dfb4fa3821969c10135c532 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:58.826799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:58.827851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:58.829520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:58.830671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:58.855159 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3243/6533] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:20:58.857975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:58.861283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:58.991358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae63d3d-3969-410f-bd5e-88007e398e7c service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:59.009347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:59.038569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a80456-1822-4085-bb76-308dc617e072 tempest-QuotasNegativeTest-1669060430 tempest-QuotasNegativeTest-1669060430-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:59.048310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:59.048976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.049256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.049528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.050171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:59.060241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:59.112000 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3291/6534] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:20:58 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:20:59.112316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:59.113052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:59.113285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.130770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.132257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] PATH_INFO: `/v3/users/41df6767ffca45ddac9242a0d942b0cc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.136043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e327a9d-621c-412b-9e70-94035621eb40 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:59.136970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41df6767ffca45ddac9242a0d942b0cc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:59.143322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:59.145417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:59.146813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.147104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.147372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.148031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:59.149674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:59.202580 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3292/6535] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:59.208118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:59.226597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da615e45-d3df-4159-88ba-34998e0ae388 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:59.228156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41df6767ffca45ddac9242a0d942b0cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:59.229626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41df6767ffca45ddac9242a0d942b0cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:59.232349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc269b-16a6-432d-9d64-e3055847fd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:59.233521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc269b-16a6-432d-9d64-e3055847fd26 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.233904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc269b-16a6-432d-9d64-e3055847fd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.234285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc269b-16a6-432d-9d64-e3055847fd26 admin admin] PATH_INFO: `/v3/projects/7f31abedc78e4a908772334262499342/users/fb9865d046e146a38946218d1a85b579/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.242372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41df6767ffca45ddac9242a0d942b0cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:59.256388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc269b-16a6-432d-9d64-e3055847fd26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f31abedc78e4a908772334262499342, user_id=fb9865d046e146a38946218d1a85b579, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:59.257519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41df6767ffca45ddac9242a0d942b0cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:59.262040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc269b-16a6-432d-9d64-e3055847fd26 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:59.277231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41df6767ffca45ddac9242a0d942b0cc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:20:59.285918 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3244/6536] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:20:58 2022] DELETE /identity/v3/users/41df6767ffca45ddac9242a0d942b0cc => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:59.288901 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3293/6537] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:59 2022] PUT /identity/v3/projects/7f31abedc78e4a908772334262499342/users/fb9865d046e146a38946218d1a85b579/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:59.290780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.290988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.291224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.292432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:59.294594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:59.318490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:59.319405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:59.334249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:59.402671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7389f-b19e-4608-80e2-ef3fee67c35b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:59.417361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:59.423777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.424331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.424621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.425485 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3294/6538] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:20:59.425791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:59.427965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:59.438663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:59.440490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:59.459170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3245/6539] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:20:59.463345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:20:59.491735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edceafc-c898-47cd-8c2d-872420d41494 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:59.503994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18721f1d-605e-4ee9-8dff-bce56c9e5dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:59.506272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18721f1d-605e-4ee9-8dff-bce56c9e5dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.506444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18721f1d-605e-4ee9-8dff-bce56c9e5dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.508769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18721f1d-605e-4ee9-8dff-bce56c9e5dd4 admin admin] PATH_INFO: `/v3/projects/7f31abedc78e4a908772334262499342/users/fb9865d046e146a38946218d1a85b579/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.513732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18721f1d-605e-4ee9-8dff-bce56c9e5dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f31abedc78e4a908772334262499342, user_id=fb9865d046e146a38946218d1a85b579, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:59.520511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18721f1d-605e-4ee9-8dff-bce56c9e5dd4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:59.554760 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3246/6540] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:20:59 2022] PUT /identity/v3/projects/7f31abedc78e4a908772334262499342/users/fb9865d046e146a38946218d1a85b579/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:20:59.567190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3372b-a316-44f5-ae51-9a333f66f62e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:20:59.573226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538921-fa76-4853-83c6-c818c27ac689 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.576577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538921-fa76-4853-83c6-c818c27ac689 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.578010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538921-fa76-4853-83c6-c818c27ac689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.654159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:20:59.656371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:20:59.656779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:20:59.657283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:20:59.722488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-15538921-fa76-4853-83c6-c818c27ac689 None None] MFA Rules not processed for user `fb9865d046e146a38946218d1a85b579`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:20:59.726414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-15538921-fa76-4853-83c6-c818c27ac689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:20:59.729674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:59.963815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:20:59.965999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:20:59.987086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3295/6541] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:00.050194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3247/6542] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:20:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:21:00.059325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:00.091627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d034567-5b38-40a5-b07f-53e441171a53 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:00.107195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:00.109750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:00.111069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:00.111455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:00.112356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:00.116021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:00.149605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3296/6543] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:21:00 2022] POST /identity/v3/domains => generated 276 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:00.151178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15538921-fa76-4853-83c6-c818c27ac689 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:00.165477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15538921-fa76-4853-83c6-c818c27ac689 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:00.346315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:00.347980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:00.348281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:00.348561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:00.349472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:00.351447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:00.458973 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3248/6544] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:00 2022] POST /identity/v3/projects => generated 402 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:00.464056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:00.489171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b7f0-fbb2-4857-b9aa-cdadc47c3599 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:00.670658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:00.672425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:00.673090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:00.673595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:00.674446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:00.676983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:00.715973 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3297/6545] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:21:00 2022] POST /identity/v3/groups => generated 302 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:00.717884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:00.736708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:00.758348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827bb0-cdfb-49d1-8112-dcf90e6b02ee tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:00.780631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:00.781679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:00.782405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:00.783570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:00.789195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:00.797178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:00.870226 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3249/6546] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:00 2022] POST /identity/v3/users => generated 442 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:00.878374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:00.896637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18df7-d86a-44df-9c1c-9873642625df tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:00.908754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:00.909402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:00.909682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:00.909955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:00.910676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:00.913595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:00.964560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:00.975978 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3298/6547] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:00 2022] POST /identity/v3/roles => generated 240 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:00.984357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784e50c-4a53-4474-9c29-e2a6c70ace0c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:00.999524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:01.002479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:01.002775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:01.003042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/groups/8d90839c1e63419f8a49cc563a256577/roles/8ae9292a31934c33b1f5f66cd80fde88/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:01.022590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_grant(project_id=11a6f354c0724a9cabcadb5b24295246, group_id=8d90839c1e63419f8a49cc563a256577, role_id=8ae9292a31934c33b1f5f66cd80fde88)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:01.027645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:01.050684 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3250/6548] 10.222.0.26 () {60 vars in 1585 bytes} [Tue Sep 27 10:21:00 2022] PUT /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/groups/8d90839c1e63419f8a49cc563a256577/roles/8ae9292a31934c33b1f5f66cd80fde88/inherited_to_projects => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:01.056085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:01.068933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d771e-3865-4adf-ab7c-4b2bdf8aea99 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:01.237083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:01.237828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:01.238109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:01.238387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/groups/8d90839c1e63419f8a49cc563a256577/roles/8ae9292a31934c33b1f5f66cd80fde88/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:01.247512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:check_grant(project_id=11a6f354c0724a9cabcadb5b24295246, group_id=8d90839c1e63419f8a49cc563a256577, role_id=8ae9292a31934c33b1f5f66cd80fde88)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:01.250243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:01.276195 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3299/6549] 10.222.0.26 () {58 vars in 1567 bytes} [Tue Sep 27 10:21:01 2022] HEAD /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/groups/8d90839c1e63419f8a49cc563a256577/roles/8ae9292a31934c33b1f5f66cd80fde88/inherited_to_projects => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:01.289536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:01.307797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef371e-ed97-4d32-b68d-5910e51c9092 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:01.325827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:01.326781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:01.327069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:01.327349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/groups/8d90839c1e63419f8a49cc563a256577/roles/8ae9292a31934c33b1f5f66cd80fde88/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:01.341136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=11a6f354c0724a9cabcadb5b24295246, group_id=8d90839c1e63419f8a49cc563a256577, role_id=8ae9292a31934c33b1f5f66cd80fde88)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:01.343808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:01.382369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 8ae9292a31934c33b1f5f66cd80fde88 was removed from group 8d90839c1e63419f8a49cc563a256577 on project 11a6f354c0724a9cabcadb5b24295246. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:01.382559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:01.386586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0d26f523-4054-4ac4-b2ae-743051abd4e8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:01.432957 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3251/6550] 10.222.0.26 () {58 vars in 1569 bytes} [Tue Sep 27 10:21:01 2022] DELETE /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/groups/8d90839c1e63419f8a49cc563a256577/roles/8ae9292a31934c33b1f5f66cd80fde88/inherited_to_projects => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:01.437373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:01.439235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:01.467228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8826e-4ac2-4686-8dbc-ec0ba40874fb tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:01.471401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:01.471929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:01.472361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:01.557213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:01.576982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:01.583352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:01.724605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:01.725269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:01.725484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:01.725818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles/8ae9292a31934c33b1f5f66cd80fde88` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:01.728746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8ae9292a31934c33b1f5f66cd80fde88)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:01.735919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:01.808884 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3252/6551] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:21:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:21:01.852712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:01.854915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:01.908916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 8ae9292a31934c33b1f5f66cd80fde88 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:01.909225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:01.912587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:01.919174 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3300/6552] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:01 2022] DELETE /identity/v3/roles/8ae9292a31934c33b1f5f66cd80fde88 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:01.924395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:01.933917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:02.144303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d034cf7-7bf5-425a-86ee-fb6d91fc9bce tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:02.152261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:02.153223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:02.153599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:02.154531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:02.155397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:02.159055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:02.192291 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3301/6553] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:01 2022] POST /identity/v3/roles => generated 239 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:02.194225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:02.223119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc3700-480e-46bf-9819-bc52fc188c3a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:02.240472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:02.241504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:02.241960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:02.242415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/8674cde0298a43d59f5d0be90e4a369f/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:02.275345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_grant(project_id=11a6f354c0724a9cabcadb5b24295246, user_id=9868988125ee4227958e29ec0b41d493, role_id=8674cde0298a43d59f5d0be90e4a369f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:02.279017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:02.289267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a0fec-b875-4c32-81af-d11fffc8245b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:02.314437 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3302/6554] 10.222.0.26 () {60 vars in 1582 bytes} [Tue Sep 27 10:21:02 2022] PUT /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/8674cde0298a43d59f5d0be90e4a369f/inherited_to_projects => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:02.321093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:02.425464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:02.496198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:02.498359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:02.499767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:02.502560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:02.511106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:02.517623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:02.541361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b093a-e2a5-453a-841f-6a910335a42e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:02.560462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:02.563900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:02.565970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:02.566503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/8674cde0298a43d59f5d0be90e4a369f/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:02.574305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:check_grant(project_id=11a6f354c0724a9cabcadb5b24295246, user_id=9868988125ee4227958e29ec0b41d493, role_id=8674cde0298a43d59f5d0be90e4a369f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:02.583103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:02.596564 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3303/6555] 10.222.0.26 () {58 vars in 1564 bytes} [Tue Sep 27 10:21:02 2022] HEAD /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/8674cde0298a43d59f5d0be90e4a369f/inherited_to_projects => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:02.601623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:02.641849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e12653-f20a-42ff-968c-a9004c4284ef tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:02.662764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:02.685893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:02.686256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:02.686489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/8674cde0298a43d59f5d0be90e4a369f/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:02.701801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=11a6f354c0724a9cabcadb5b24295246, user_id=9868988125ee4227958e29ec0b41d493, role_id=8674cde0298a43d59f5d0be90e4a369f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:02.708820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:02.758652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 8674cde0298a43d59f5d0be90e4a369f was removed from user 9868988125ee4227958e29ec0b41d493 on project 11a6f354c0724a9cabcadb5b24295246. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:02.759340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:02.759635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:02.779299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:02.780962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:02.803831 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3304/6556] 10.222.0.26 () {58 vars in 1566 bytes} [Tue Sep 27 10:21:02 2022] DELETE /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/8674cde0298a43d59f5d0be90e4a369f/inherited_to_projects => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:02.815049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:02.819674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:02.824249 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3253/6557] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:21:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 972 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:21:02.831378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:02.882315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45722399-ec08-4e68-8052-2e87e230f2d4 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:03.032714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a46c1c-aad4-439d-a4ed-c34da6096303 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:03.038579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:03.039510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:03.039967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:03.040400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] PATH_INFO: `/v3/projects/28da452e9e064d948584b9f54a14d430` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:03.042362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=28da452e9e064d948584b9f54a14d430)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:03.044482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:03.097057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:03.098035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:03.098503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:03.099157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles/8674cde0298a43d59f5d0be90e4a369f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:03.100983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8674cde0298a43d59f5d0be90e4a369f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:03.103238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:03.170286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28da452e9e064d948584b9f54a14d430', 'request_id': 'req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.196385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28da452e9e064d948584b9f54a14d430', 'request_id': 'req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.204619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28da452e9e064d948584b9f54a14d430', 'request_id': 'req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.232563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] The token cache is being invalidate because project 28da452e9e064d948584b9f54a14d430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:03.232977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.233735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 8674cde0298a43d59f5d0be90e4a369f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:03.233907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.234051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.234666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.235796 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3305/6558] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:02 2022] DELETE /identity/v3/projects/28da452e9e064d948584b9f54a14d430 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:03.236710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3254/6559] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:02 2022] DELETE /identity/v3/roles/8674cde0298a43d59f5d0be90e4a369f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:03.243360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:03.244566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:03.246557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:03.249003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:03.379428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:03.433743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd14eb-4689-46c5-a331-3e6c663c1226 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:03.440300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc19c72-875f-4604-9ce5-4a02b45e7bcf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:03.450242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:03.451596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:03.451902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:03.452160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:03.453693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:03.456417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:03.476236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:03.526895 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3255/6560] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:03 2022] POST /identity/v3/roles => generated 240 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:03.531111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:03.565196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2af994-47da-4420-9b93-461d85e6a4dc tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:03.590162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:03.591098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:03.591464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:03.593037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/fc408a8803ca44bb9db2b69e9eadc0b1/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:03.607555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, group_id=8d90839c1e63419f8a49cc563a256577, role_id=fc408a8803ca44bb9db2b69e9eadc0b1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:03.612154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:03.655914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3256/6561] 10.222.0.26 () {60 vars in 1582 bytes} [Tue Sep 27 10:21:03 2022] PUT /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/fc408a8803ca44bb9db2b69e9eadc0b1/inherited_to_projects => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:03.659542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:03.696192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:03.698851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:03.699412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:03.699803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] PATH_INFO: `/v3/users/10dd231166ca47948063da19afbdf84c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:03.703564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=10dd231166ca47948063da19afbdf84c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:03.712495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:03.862692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23d72-f817-4fdb-bc4f-58f83f09a2d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:03.876242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:03.877340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:03.877785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:03.878233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:03.881925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=a793186f67b1439f8f62fcac6c3d6471, group_id=8d90839c1e63419f8a49cc563a256577)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:03.883273 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:21:03.883273 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:21:03.883273 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:21:03.892127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10dd231166ca47948063da19afbdf84c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.894331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:03.895947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10dd231166ca47948063da19afbdf84c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.928152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10dd231166ca47948063da19afbdf84c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.937498 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3257/6562] 10.222.0.26 () {58 vars in 1464 bytes} [Tue Sep 27 10:21:03 2022] GET /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/inherited_to_projects => generated 448 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:03.939106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10dd231166ca47948063da19afbdf84c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.940794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10dd231166ca47948063da19afbdf84c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:03.941994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:03.945150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:03.976323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57c62-c23a-4278-afa2-37c582f6047c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:03.988394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:03.989374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:03.989918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:03.990309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/fc408a8803ca44bb9db2b69e9eadc0b1/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:03.995565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3306/6563] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:03 2022] DELETE /identity/v3/users/10dd231166ca47948063da19afbdf84c => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:03.996620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, group_id=8d90839c1e63419f8a49cc563a256577, role_id=fc408a8803ca44bb9db2b69e9eadc0b1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:03.999586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:04.014401 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3258/6564] 10.222.0.26 () {58 vars in 1564 bytes} [Tue Sep 27 10:21:03 2022] HEAD /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/fc408a8803ca44bb9db2b69e9eadc0b1/inherited_to_projects => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:04.018747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:04.208957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778dbf8-5468-43a9-ad56-9eabfc9efe5e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:04.221663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:04.222520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:04.224952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:04.225280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/fc408a8803ca44bb9db2b69e9eadc0b1/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:04.227849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, group_id=8d90839c1e63419f8a49cc563a256577, role_id=fc408a8803ca44bb9db2b69e9eadc0b1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:04.232638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:04.247063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role fc408a8803ca44bb9db2b69e9eadc0b1 was removed from group 8d90839c1e63419f8a49cc563a256577 on domain a793186f67b1439f8f62fcac6c3d6471. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:04.247514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:04.248882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:04.304125 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3307/6565] 10.222.0.26 () {58 vars in 1566 bytes} [Tue Sep 27 10:21:04 2022] DELETE /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/groups/8d90839c1e63419f8a49cc563a256577/roles/fc408a8803ca44bb9db2b69e9eadc0b1/inherited_to_projects => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:04.308050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:04.309921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:04.343590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab557f-51cb-4ff6-afc1-ab6cc36a210a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:04.463722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:04.464532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:04.464935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:04.465314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles/fc408a8803ca44bb9db2b69e9eadc0b1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:04.468199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_role(role_id=fc408a8803ca44bb9db2b69e9eadc0b1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:04.469989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:04.546675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role fc408a8803ca44bb9db2b69e9eadc0b1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:04.547125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:04.549501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:04.554675 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3259/6566] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:04 2022] DELETE /identity/v3/roles/fc408a8803ca44bb9db2b69e9eadc0b1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:04.561633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:04.584728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703f7c5-dcb2-421d-9f48-a30b64e6cb12 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:04.621305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:04.753552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:04.754199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:04.754485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:04.754762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:04.755532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:04.757214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:04.809176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3308/6567] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:04 2022] POST /identity/v3/roles => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:04.809338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:04.811402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:04.839649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c80a06-67a0-4614-8c7e-e69ba58223c5 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:04.846977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:04.847686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:04.848029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:04.849160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/85463bdd779f4856b8b4f8831960f17e/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:04.851539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, user_id=9868988125ee4227958e29ec0b41d493, role_id=85463bdd779f4856b8b4f8831960f17e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:04.856239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:04.885830 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3260/6568] 10.222.0.26 () {60 vars in 1579 bytes} [Tue Sep 27 10:21:04 2022] PUT /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/85463bdd779f4856b8b4f8831960f17e/inherited_to_projects => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:04.889049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:04.909561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c577e90-3e4d-4c72-9a6d-27d1f2ed0bb1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:05.072412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:05.073317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:05.073711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:05.074077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:05.080456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=a793186f67b1439f8f62fcac6c3d6471, user_id=9868988125ee4227958e29ec0b41d493)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:05.090282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:05.131660 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3309/6569] 10.222.0.26 () {58 vars in 1461 bytes} [Tue Sep 27 10:21:04 2022] GET /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/inherited_to_projects => generated 447 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:05.136816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:05.148542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:05.156687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ca9df-29d5-4acf-8738-7617408d3714 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:05.184802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:05.185514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:05.185891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:05.186235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/85463bdd779f4856b8b4f8831960f17e/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:05.189902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, user_id=9868988125ee4227958e29ec0b41d493, role_id=85463bdd779f4856b8b4f8831960f17e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:05.195625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:05.213598 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3261/6570] 10.222.0.26 () {58 vars in 1561 bytes} [Tue Sep 27 10:21:05 2022] HEAD /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/85463bdd779f4856b8b4f8831960f17e/inherited_to_projects => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:05.218460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:05.232351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936da8-a11b-4801-9e61-2fd0e640fc55 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:05.241644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:05.242564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:05.243520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:05.243917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/85463bdd779f4856b8b4f8831960f17e/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:05.247184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, user_id=9868988125ee4227958e29ec0b41d493, role_id=85463bdd779f4856b8b4f8831960f17e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:05.252030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:05.261939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 85463bdd779f4856b8b4f8831960f17e was removed from user 9868988125ee4227958e29ec0b41d493 on domain a793186f67b1439f8f62fcac6c3d6471. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:05.262386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:05.263762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:05.298080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3310/6571] 10.222.0.26 () {58 vars in 1563 bytes} [Tue Sep 27 10:21:05 2022] DELETE /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/85463bdd779f4856b8b4f8831960f17e/inherited_to_projects => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:05.304256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:05.306590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:05.359888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebce16-b627-4bea-a019-f82431674664 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:05.530162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:05.532087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:05.535996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:05.536574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles/85463bdd779f4856b8b4f8831960f17e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:05.543758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_role(role_id=85463bdd779f4856b8b4f8831960f17e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:05.546799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:05.697545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 85463bdd779f4856b8b4f8831960f17e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:05.697909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:05.698846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:05.716079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:05.716639 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3262/6572] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:05 2022] DELETE /identity/v3/roles/85463bdd779f4856b8b4f8831960f17e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:05.734947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaf2f6-ccba-4afb-934a-9063d66cc671 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:05.866946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:05.867962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:05.868308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:05.868772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:05.869830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:05.871943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:05.914402 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3311/6573] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:05 2022] POST /identity/v3/roles => generated 240 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:05.919259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:05.920956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:05.946732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b87f77-83b5-4022-bc41-c1d7555bb9b6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:05.960202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:05.961240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:05.961240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:05.961549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:05.969489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:05.975240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:06.079174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:06.079610 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3263/6574] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:05 2022] POST /identity/v3/projects => generated 410 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:06.094904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a65e-0652-4352-aeb6-bf7fd3277c5e tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:06.178374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:06.293205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:06.294437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:06.295034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:06.295581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/7f053360b9ec439cb1cb923c53b633a0/inherited_to_projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:06.299680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, user_id=9868988125ee4227958e29ec0b41d493, role_id=7f053360b9ec439cb1cb923c53b633a0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:06.305432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:06.348641 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3312/6575] 10.222.0.26 () {60 vars in 1579 bytes} [Tue Sep 27 10:21:06 2022] PUT /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/7f053360b9ec439cb1cb923c53b633a0/inherited_to_projects => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:06.353289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:06.367189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2e131-a370-42f4-963f-4b6f98149083 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:06.620900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:06.621601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:06.621818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:06.622091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:06.622836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Adding query filter params (scope.project.id=11a6f354c0724a9cabcadb5b24295246, user.id=9868988125ee4227958e29ec0b41d493) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:06.623140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:06.624169 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:21:06.624169 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:21:06.624169 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:21:06.635242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:06.775253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:06.778328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:06.790447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:06.885492 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3264/6576] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:21:06 2022] GET /identity/v3/role_assignments?scope.project.id=11a6f354c0724a9cabcadb5b24295246&user.id=9868988125ee4227958e29ec0b41d493&effective => generated 648 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:06.886661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:06.941125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69eb5b-ddc1-48c0-b600-f8610b879fe8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:06.974925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:06.975574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:06.976520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:06.976835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:06.977672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Adding query filter params (scope.project.id=293041d9fd0d44f2903deb33e5d49102, user.id=9868988125ee4227958e29ec0b41d493) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:06.982643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:06.984612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:07.025520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912d723-8543-4966-acf5-513072ba782d tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:07.043425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:07.044224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.044903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:07.045289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] PATH_INFO: `/v3/projects/499a5d95ab7e4f088a23ccfabccb61a0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:07.047836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=499a5d95ab7e4f088a23ccfabccb61a0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:07.052835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:07.196775 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3265/6577] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:21:06 2022] GET /identity/v3/role_assignments?scope.project.id=293041d9fd0d44f2903deb33e5d49102&user.id=9868988125ee4227958e29ec0b41d493&effective => generated 648 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:07.197968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:07.208986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499a5d95ab7e4f088a23ccfabccb61a0', 'request_id': 'req-5eddeeaa-53c5-4ae7-960f-26739c33e041'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:07.210527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc6581-591d-41ae-ab17-7da1680f91ac tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:07.226860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499a5d95ab7e4f088a23ccfabccb61a0', 'request_id': 'req-5eddeeaa-53c5-4ae7-960f-26739c33e041'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:07.250755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499a5d95ab7e4f088a23ccfabccb61a0', 'request_id': 'req-5eddeeaa-53c5-4ae7-960f-26739c33e041'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:07.275861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] The token cache is being invalidate because project 499a5d95ab7e4f088a23ccfabccb61a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:07.276324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:07.277135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5eddeeaa-53c5-4ae7-960f-26739c33e041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:07.280277 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3313/6578] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:06 2022] DELETE /identity/v3/projects/499a5d95ab7e4f088a23ccfabccb61a0 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:07.295217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6ece-e380-4226-b62a-f0538f06bb3d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.296914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6ece-e380-4226-b62a-f0538f06bb3d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:07.297343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6ece-e380-4226-b62a-f0538f06bb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:07.323167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:07.324136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.326045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:07.326527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/7f053360b9ec439cb1cb923c53b633a0/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:07.333764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a793186f67b1439f8f62fcac6c3d6471, user_id=9868988125ee4227958e29ec0b41d493, role_id=7f053360b9ec439cb1cb923c53b633a0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:07.335941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0dfb6ece-e380-4226-b62a-f0538f06bb3d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:07.337979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0dfb6ece-e380-4226-b62a-f0538f06bb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:07.342108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:07.357222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 7f053360b9ec439cb1cb923c53b633a0 was removed from user 9868988125ee4227958e29ec0b41d493 on domain a793186f67b1439f8f62fcac6c3d6471. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:07.358356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:07.362315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:07.402091 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3266/6579] 10.222.0.26 () {58 vars in 1563 bytes} [Tue Sep 27 10:21:07 2022] DELETE /identity/v3/OS-INHERIT/domains/a793186f67b1439f8f62fcac6c3d6471/users/9868988125ee4227958e29ec0b41d493/roles/7f053360b9ec439cb1cb923c53b633a0/inherited_to_projects => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:07.406560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:07.409500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:07.456150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:07.550026 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3314/6580] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:07.552197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d221-2109-4fa4-b1da-eae3f006fde5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.553226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d221-2109-4fa4-b1da-eae3f006fde5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:07.553755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d221-2109-4fa4-b1da-eae3f006fde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:07.583438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-dee9d221-2109-4fa4-b1da-eae3f006fde5 None None] MFA Rules not processed for user `97c769dbba794531b0a9db8e764ff097`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:07.585516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dee9d221-2109-4fa4-b1da-eae3f006fde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:07.666743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831618-a5ad-44ee-a5fb-79e790d7d265 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:07.686814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:07.687452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.687735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:07.688009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:07.697116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Adding query filter params (scope.project.id=11a6f354c0724a9cabcadb5b24295246, user.id=9868988125ee4227958e29ec0b41d493) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:07.697393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:07.707499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:07.764301 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3315/6581] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:07.765807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5368e-b7cc-4d3f-bfbc-cca04bc0c426 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.766205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5368e-b7cc-4d3f-bfbc-cca04bc0c426 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:07.766608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5368e-b7cc-4d3f-bfbc-cca04bc0c426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:07.798391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-33f5368e-b7cc-4d3f-bfbc-cca04bc0c426 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:07.800475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-33f5368e-b7cc-4d3f-bfbc-cca04bc0c426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:07.808573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:07.873864 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3267/6582] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:21:07 2022] GET /identity/v3/role_assignments?scope.project.id=11a6f354c0724a9cabcadb5b24295246&user.id=9868988125ee4227958e29ec0b41d493&effective => generated 228 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:21:07.880356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:07.923162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf167e28-980a-403c-ac6c-67b6c688064f tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:07.944898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:07.945603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.945908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:07.946102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:07.946821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Adding query filter params (scope.project.id=293041d9fd0d44f2903deb33e5d49102, user.id=9868988125ee4227958e29ec0b41d493) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:07.947193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:07.949051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:07.957753 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3316/6583] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:07.964110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5368e-b7cc-4d3f-bfbc-cca04bc0c426 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:07.982859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5368e-b7cc-4d3f-bfbc-cca04bc0c426 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:07.996437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:07.998286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:07.998812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:08.001703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:08.001913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:08.002694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:08.059604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:08.059604 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3317/6584] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:08.066181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:08.114092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111001-1e0a-48af-b363-781755d66c54 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:08.122913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3268/6585] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:21:07 2022] GET /identity/v3/role_assignments?scope.project.id=293041d9fd0d44f2903deb33e5d49102&user.id=9868988125ee4227958e29ec0b41d493&effective => generated 228 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:08.123863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:08.137455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:08.140064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:08.140635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:08.144224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:08.146187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:08.157344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248baf16-3c60-4d40-aba5-d2098219b8d6 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:08.158314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:08.187343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:08.188031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:08.188322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:08.188603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/projects/293041d9fd0d44f2903deb33e5d49102` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:08.206680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_project(project_id=293041d9fd0d44f2903deb33e5d49102)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:08.208793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:08.278848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3318/6586] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:08 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:08.280162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:08.378885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293041d9fd0d44f2903deb33e5d49102', 'request_id': 'req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:08.398207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293041d9fd0d44f2903deb33e5d49102', 'request_id': 'req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:08.421235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293041d9fd0d44f2903deb33e5d49102', 'request_id': 'req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:08.452107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:08.463457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a18d8-dcc9-45b5-9a25-6c2f09cc9608 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:08.464556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] The token cache is being invalidate because project 293041d9fd0d44f2903deb33e5d49102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:08.465133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:08.466018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:08.471230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3269/6587] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:08 2022] DELETE /identity/v3/projects/293041d9fd0d44f2903deb33e5d49102 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:08.472320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:08.474900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:08.629983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:08.630797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:08.631258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:08.631631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:08.632385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:08.634468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:08.650563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a887b3-f301-4d71-bf9c-c4a9c5faecf8 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:08.663014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:08.663955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:08.664406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:08.664890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles/7f053360b9ec439cb1cb923c53b633a0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:08.668637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7f053360b9ec439cb1cb923c53b633a0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:08.674405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:08.683915 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3319/6588] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:08 2022] POST /identity/v3/users => generated 289 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:08.688097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:08.692755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:08.714026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:08.719890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ae66f-f102-41bf-9578-37a085e31ce5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:08.745973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:08.749520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role 7f053360b9ec439cb1cb923c53b633a0 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:08.749805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:08.753546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:08.753954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:08.754333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:08.754514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:08.756149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:08.757078 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3270/6589] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:08 2022] DELETE /identity/v3/roles/7f053360b9ec439cb1cb923c53b633a0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:08.759721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:08.765720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:08.767178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:08.813127 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3320/6590] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:08.814428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:08.825244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:08.855298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dc2ef-30dc-4218-a486-01ea80b4edd6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:08.949949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596742-1e74-4858-81fa-bfdc5523f6a1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:08.969811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:08.970816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:08.971345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:08.971834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:08.972745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:08.975015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.014205 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3271/6591] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:08 2022] POST /identity/v3/roles => generated 240 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:09.014463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:09.014463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.014463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.014463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.014820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:09.016144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:09.017337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.038277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61a07b-33fd-4b1c-912e-95b8f5d35a29 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:09.042834 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3321/6592] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:08 2022] GET /identity/v3/roles => generated 1651 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:09.047220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:09.048476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:09.049384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.050184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.050902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.051822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:09.054007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.074595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d315ace-d651-40b6-af40-97df7ee6d4df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:09.086314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2a43e-7431-4d68-b84a-82272526b727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:09.086833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2a43e-7431-4d68-b84a-82272526b727 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.087175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2a43e-7431-4d68-b84a-82272526b727 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.088165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2a43e-7431-4d68-b84a-82272526b727 admin admin] PATH_INFO: `/v3/projects/05ee5f7ac3e0456ea3f3c84fa5c33266/users/404378dde74649c29604372480c0f97b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.133022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2a43e-7431-4d68-b84a-82272526b727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ee5f7ac3e0456ea3f3c84fa5c33266, user_id=404378dde74649c29604372480c0f97b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:09.137200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2a43e-7431-4d68-b84a-82272526b727 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.186146 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3322/6593] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:09 2022] PUT /identity/v3/projects/05ee5f7ac3e0456ea3f3c84fa5c33266/users/404378dde74649c29604372480c0f97b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:09.188969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaff5d-08b9-46a6-a03f-035dd2673e7d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.191845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaff5d-08b9-46a6-a03f-035dd2673e7d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.192411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaff5d-08b9-46a6-a03f-035dd2673e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.201867 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3272/6594] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:09 2022] POST /identity/v3/projects => generated 409 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:09.208395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:09.260593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aacaff5d-08b9-46a6-a03f-035dd2673e7d None None] MFA Rules not processed for user `404378dde74649c29604372480c0f97b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:09.262853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aacaff5d-08b9-46a6-a03f-035dd2673e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:09.411336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79c9a0-7b2b-437b-ac35-d91a00a8aaa3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:09.418155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:09.429539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:09.430549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.433428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.433921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/fd3dc635cb614d19ad6535131e644773/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.442828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:create_grant(project_id=11a6f354c0724a9cabcadb5b24295246, user_id=9868988125ee4227958e29ec0b41d493, role_id=fd3dc635cb614d19ad6535131e644773)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:09.446380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.482948 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3273/6595] 10.222.0.26 () {60 vars in 1582 bytes} [Tue Sep 27 10:21:09 2022] PUT /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/fd3dc635cb614d19ad6535131e644773/inherited_to_projects => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:09.487037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:09.495784 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3323/6596] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:09.500398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaff5d-08b9-46a6-a03f-035dd2673e7d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:09.652605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:09.736374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec8f0-1f50-4cf7-8514-fe0ac962272c tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:09.741815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:09.742597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.742880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.743150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.743879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Adding query filter params (scope.project.id=feba93f493fc48658322a3845bac4bc3, user.id=9868988125ee4227958e29ec0b41d493) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:09.744178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:09.746264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.747555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaff5d-08b9-46a6-a03f-035dd2673e7d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:09.845932 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3274/6597] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:21:09 2022] GET /identity/v3/role_assignments?scope.project.id=feba93f493fc48658322a3845bac4bc3&user.id=9868988125ee4227958e29ec0b41d493&effective => generated 649 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:21:09.848006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:09.877541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4dc5-0f2d-4ecb-8c18-abb00c42d749 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:09.894178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:09.897168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.898225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.898691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/fd3dc635cb614d19ad6535131e644773/inherited_to_projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.910215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=11a6f354c0724a9cabcadb5b24295246, user_id=9868988125ee4227958e29ec0b41d493, role_id=fd3dc635cb614d19ad6535131e644773)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:09.927100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:09.929614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.931364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:09.934229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:09.934229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:09.940488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:09.948775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role fd3dc635cb614d19ad6535131e644773 was removed from user 9868988125ee4227958e29ec0b41d493 on project 11a6f354c0724a9cabcadb5b24295246. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:09.949545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:09.951847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:09.953635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:10.017257 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3275/6598] 10.222.0.26 () {58 vars in 1566 bytes} [Tue Sep 27 10:21:09 2022] DELETE /identity/v3/OS-INHERIT/projects/11a6f354c0724a9cabcadb5b24295246/users/9868988125ee4227958e29ec0b41d493/roles/fd3dc635cb614d19ad6535131e644773/inherited_to_projects => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:10.024618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:10.026283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:10.070593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3324/6599] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:09 2022] POST /identity/v3/projects => generated 375 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:10.072222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:10.074927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:10.205562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc6087-6159-434e-9547-c44dff5a5505 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:10.218594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:10.219400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:10.221007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:10.223211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:10.224485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:10.230746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:10.248295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565f0f-f357-43ef-aa25-bb5c88f6c6b3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:10.319499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3325/6600] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:10 2022] POST /identity/v3/users => generated 294 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:10.323584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:10.347275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b0468-1dbd-4868-bf18-b160c0416dc2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:10.366537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:10.367526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:10.368554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:10.370388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:10.371218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:10.376066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:10.380149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:10.453586 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3326/6601] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:10 2022] GET /identity/v3/roles => generated 1651 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:10.454461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:10.455740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:10.459784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:10.460219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:10.460610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:10.461923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Adding query filter params (scope.project.id=feba93f493fc48658322a3845bac4bc3, user.id=9868988125ee4227958e29ec0b41d493) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:10.462353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:10.466962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c04e9-93af-47e3-b995-21db543db5d3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:10.468227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:10.473064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:10.474333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:10.479179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:10.479629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] PATH_INFO: `/v3/projects/526c4d6a63004031baef7e7b321a31ac/users/51d9a02f69ed4d5cb3368407284dc1cd/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:10.515307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=526c4d6a63004031baef7e7b321a31ac, user_id=51d9a02f69ed4d5cb3368407284dc1cd, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:10.521691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:10.541479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:10.561079 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3327/6602] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:10 2022] PUT /identity/v3/projects/526c4d6a63004031baef7e7b321a31ac/users/51d9a02f69ed4d5cb3368407284dc1cd/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:10.563552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:10.616213 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3276/6603] 10.222.0.26 () {58 vars in 1356 bytes} [Tue Sep 27 10:21:10 2022] GET /identity/v3/role_assignments?scope.project.id=feba93f493fc48658322a3845bac4bc3&user.id=9868988125ee4227958e29ec0b41d493&effective => generated 228 bytes in 592 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:10.618050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:10.636823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8bc93-c37b-447f-b861-c0f23ad7f57a tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:10.803330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:10.804231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:10.804753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:10.806643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/projects/feba93f493fc48658322a3845bac4bc3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:10.809038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_project(project_id=feba93f493fc48658322a3845bac4bc3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:10.813059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:10.819754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d4806-60a5-44aa-95fa-68a4e9126c46 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:10.836060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:10.841353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:10.842003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:10.847094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:10.848039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:10.853180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:10.887036 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3328/6604] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:10 2022] GET /identity/v3/roles => generated 1651 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:10.887960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:10.901721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850153f-9750-4afa-8bd5-42dbdc2fc6f8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:10.928189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5df98-0524-4588-a67a-ef6906e503b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:10.929095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5df98-0524-4588-a67a-ef6906e503b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:10.929432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5df98-0524-4588-a67a-ef6906e503b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:10.931299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5df98-0524-4588-a67a-ef6906e503b6 admin admin] PATH_INFO: `/v3/projects/526c4d6a63004031baef7e7b321a31ac/users/51d9a02f69ed4d5cb3368407284dc1cd/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:10.934729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5df98-0524-4588-a67a-ef6906e503b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=526c4d6a63004031baef7e7b321a31ac, user_id=51d9a02f69ed4d5cb3368407284dc1cd, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:10.939924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5df98-0524-4588-a67a-ef6906e503b6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:11.008302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feba93f493fc48658322a3845bac4bc3', 'request_id': 'req-e0db6b7e-82c8-4150-8b7c-a303855970e4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:11.023484 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3329/6605] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:10 2022] PUT /identity/v3/projects/526c4d6a63004031baef7e7b321a31ac/users/51d9a02f69ed4d5cb3368407284dc1cd/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:11.031319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feba93f493fc48658322a3845bac4bc3', 'request_id': 'req-e0db6b7e-82c8-4150-8b7c-a303855970e4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:11.040443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:11.040831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:11.041220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:11.071647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feba93f493fc48658322a3845bac4bc3', 'request_id': 'req-e0db6b7e-82c8-4150-8b7c-a303855970e4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:11.127971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] The token cache is being invalidate because project feba93f493fc48658322a3845bac4bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:11.128246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:11.134362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] MFA Rules not processed for user `51d9a02f69ed4d5cb3368407284dc1cd`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:11.134876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:11.136615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:11.137372 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3277/6606] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:10 2022] DELETE /identity/v3/projects/feba93f493fc48658322a3845bac4bc3 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:11.141679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:11.146713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:11.377165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3330/6607] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:21:11.388905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db6b7e-82c8-4150-8b7c-a303855970e4 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:11.411701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:11.412490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:11.412812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:11.413094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/roles/fd3dc635cb614d19ad6535131e644773` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:11.421601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_role(role_id=fd3dc635cb614d19ad6535131e644773)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:11.423347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:11.424731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:11.426159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:11.509865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:11.601911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invalidating the token cache because role fd3dc635cb614d19ad6535131e644773 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:11.602478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:11.605030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:11.615443 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3278/6608] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:11 2022] DELETE /identity/v3/roles/fd3dc635cb614d19ad6535131e644773 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:11.622078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:11.622948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:11.625972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:11.672384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1082b-dfd3-4a31-93ce-56aaa411d7ae None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:11.849595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa84fb1-63ec-4658-895f-d8052cd152e1 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:11.886521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:11.887998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:11.888455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:11.889079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/users/9868988125ee4227958e29ec0b41d493` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:11.891336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_user(user_id=9868988125ee4227958e29ec0b41d493)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:11.893874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:12.009842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:12.010551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:12.011300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:12.011596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:12.015022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:12.098810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9868988125ee4227958e29ec0b41d493'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:12.102630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9868988125ee4227958e29ec0b41d493'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:12.144481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9868988125ee4227958e29ec0b41d493'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:12.174760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9868988125ee4227958e29ec0b41d493'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:12.217258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9868988125ee4227958e29ec0b41d493'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:12.243294 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3279/6609] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:11 2022] DELETE /identity/v3/users/9868988125ee4227958e29ec0b41d493 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:12.246975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:12.251961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:12.388810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:12.399968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:12.454828 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3331/6610] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 1030 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:21:12.480000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:12.499660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd784-3935-4d17-8a2d-24ac73368574 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:12.529100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:12.532248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:12.532543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:12.550942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/groups/8d90839c1e63419f8a49cc563a256577` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:12.552887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_group(group_id=8d90839c1e63419f8a49cc563a256577)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:12.553874 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:21:12.553874 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:21:12.553874 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:21:12.560970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:12.844091 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3280/6611] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:21:12 2022] DELETE /identity/v3/groups/8d90839c1e63419f8a49cc563a256577 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:12.848034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:12.914636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703c215-3a42-4ac2-ab1e-bf5d3ac52e0b service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:13.163690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:13.164362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:13.164648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:13.168475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/projects/11a6f354c0724a9cabcadb5b24295246` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:13.169989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_project(project_id=11a6f354c0724a9cabcadb5b24295246)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:13.172779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:13.258075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:13.261209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a6f354c0724a9cabcadb5b24295246', 'request_id': 'req-78df0224-588b-472e-b49a-5da78f47e7c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.292087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a6f354c0724a9cabcadb5b24295246', 'request_id': 'req-78df0224-588b-472e-b49a-5da78f47e7c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.299891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a6f354c0724a9cabcadb5b24295246', 'request_id': 'req-78df0224-588b-472e-b49a-5da78f47e7c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.322742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] The token cache is being invalidate because project 11a6f354c0724a9cabcadb5b24295246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:13.325187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.328740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.334908 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3332/6612] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:12 2022] DELETE /identity/v3/projects/11a6f354c0724a9cabcadb5b24295246 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:13.335753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:13.337802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:13.379090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748a74b-0d7c-4a0a-9ac5-3828b7c8e291 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:13.571810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:13.572600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:13.573020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:13.578672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/domains/a793186f67b1439f8f62fcac6c3d6471` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:13.580805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a793186f67b1439f8f62fcac6c3d6471)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:13.582723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:13.642803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.643704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.721832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.724880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:13.734538 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3281/6613] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:13 2022] PATCH /identity/v3/domains/a793186f67b1439f8f62fcac6c3d6471 => generated 277 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:13.736981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:13.755583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df0224-588b-472e-b49a-5da78f47e7c3 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:13.894177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9865d046e146a38946218d1a85b579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f31abedc78e4a908772334262499342', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:13.894955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:13.895133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:13.895413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] PATH_INFO: `/v3/domains/a793186f67b1439f8f62fcac6c3d6471` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:13.925870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a793186f67b1439f8f62fcac6c3d6471)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:13.927801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:13.993105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.017869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.105989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471', 'request_id': 'req-ccebe86f-431e-4110-9cb0-70906afdf6ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.125402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471', 'request_id': 'req-ccebe86f-431e-4110-9cb0-70906afdf6ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.132778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471', 'request_id': 'req-ccebe86f-431e-4110-9cb0-70906afdf6ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.160550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] The token cache is being invalidate because project a793186f67b1439f8f62fcac6c3d6471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:14.160858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.169237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.189774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471', 'request_id': 'req-ccebe86f-431e-4110-9cb0-70906afdf6ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.190582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471', 'request_id': 'req-ccebe86f-431e-4110-9cb0-70906afdf6ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.199579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ccebe86f-431e-4110-9cb0-70906afdf6ad tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a793186f67b1439f8f62fcac6c3d6471', 'request_id': 'req-ccebe86f-431e-4110-9cb0-70906afdf6ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.216877 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3333/6614] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:21:13 2022] DELETE /identity/v3/domains/a793186f67b1439f8f62fcac6c3d6471 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:14.222602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:14.224156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:14.358454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9df8c-161c-4631-ab50-9814cbe62802 tempest-InheritsV3TestJSON-1323156724 tempest-InheritsV3TestJSON-1323156724-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:14.403044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:14.457707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:14.459077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:14.460177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:14.460754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] PATH_INFO: `/v3/users/63690bb54cc54cc2a02ab8002e437939` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:14.462871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63690bb54cc54cc2a02ab8002e437939)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:14.466687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:14.640671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63690bb54cc54cc2a02ab8002e437939'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.643962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63690bb54cc54cc2a02ab8002e437939'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.669161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63690bb54cc54cc2a02ab8002e437939'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.753732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63690bb54cc54cc2a02ab8002e437939'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.773261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63690bb54cc54cc2a02ab8002e437939'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:14.784121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3282/6615] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:14 2022] DELETE /identity/v3/users/63690bb54cc54cc2a02ab8002e437939 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:14.786442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60efdb-705c-4d41-b5fb-c68ad5289064 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:14.787404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60efdb-705c-4d41-b5fb-c68ad5289064 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:14.787916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60efdb-705c-4d41-b5fb-c68ad5289064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:14.820939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cc60efdb-705c-4d41-b5fb-c68ad5289064 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:14.823132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cc60efdb-705c-4d41-b5fb-c68ad5289064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:14.836873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:14.916599 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3334/6616] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:14.943460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:14.944986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:15.104445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c793e-acf3-438b-81c2-24aaa14daf73 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:15.116435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:15.118702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:15.119249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:15.119375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:15.141518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:15.143150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:15.171247 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3283/6617] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:21:16.800139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23172acf-0c49-4348-8418-86e60271918f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:16.800139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23172acf-0c49-4348-8418-86e60271918f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:16.800634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23172acf-0c49-4348-8418-86e60271918f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:16.843623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:16.845101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:16.885291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-23172acf-0c49-4348-8418-86e60271918f None None] MFA Rules not processed for user `64c9f44e57164b858421b55114acf4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:16.887395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23172acf-0c49-4348-8418-86e60271918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:16.891921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17c518-14ed-4e94-bd77-2b808cf460b3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:16.920237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:16.929511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:16.939359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:16.940881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] PATH_INFO: `/v3/projects/25330fd966d24b0d8d79ce82837f7c48` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:16.943776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=25330fd966d24b0d8d79ce82837f7c48)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:16.961881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:17.084740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:17.088613 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3335/6618] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:17.089411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25330fd966d24b0d8d79ce82837f7c48', 'request_id': 'req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.113324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25330fd966d24b0d8d79ce82837f7c48', 'request_id': 'req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.135768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25330fd966d24b0d8d79ce82837f7c48', 'request_id': 'req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.166079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] The token cache is being invalidate because project 25330fd966d24b0d8d79ce82837f7c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:17.166696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.168207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.178273 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3284/6619] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:16 2022] DELETE /identity/v3/projects/25330fd966d24b0d8d79ce82837f7c48 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:17.182218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23172acf-0c49-4348-8418-86e60271918f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:17.196056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23172acf-0c49-4348-8418-86e60271918f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:17.370921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:17.371665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:17.371872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:17.372147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] PATH_INFO: `/v3/users/fb9865d046e146a38946218d1a85b579` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:17.373713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb9865d046e146a38946218d1a85b579)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:17.378374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:17.498937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb9865d046e146a38946218d1a85b579'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.499968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb9865d046e146a38946218d1a85b579'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.507765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:17.522073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb9865d046e146a38946218d1a85b579'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.535588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb9865d046e146a38946218d1a85b579'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.539293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb9865d046e146a38946218d1a85b579'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:17.592506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3336/6620] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:17 2022] DELETE /identity/v3/users/fb9865d046e146a38946218d1a85b579 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:18.513457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:18.515476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:18.704680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52d4e9-35d8-4137-a9ae-2cb86c84a19a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:18.836074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be9dba-8353-46a7-b03a-6bba131ab5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:18.837044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be9dba-8353-46a7-b03a-6bba131ab5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:18.837044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be9dba-8353-46a7-b03a-6bba131ab5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:18.837289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be9dba-8353-46a7-b03a-6bba131ab5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:18.840379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-42be9dba-8353-46a7-b03a-6bba131ab5c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:19.006216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be9dba-8353-46a7-b03a-6bba131ab5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:19.007953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be9dba-8353-46a7-b03a-6bba131ab5c1 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:19.033312 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3285/6621] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:20.385672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:20.387134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:20.494937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3c56d-2946-487a-a067-24ae5a24b5f9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:20.509436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:20.510175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:20.510474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:20.511861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] PATH_INFO: `/v3/projects/7f31abedc78e4a908772334262499342` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:20.517933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f31abedc78e4a908772334262499342)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:20.520949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:20.636718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f31abedc78e4a908772334262499342', 'request_id': 'req-526dbb38-b91f-4892-95d7-2238000adf79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:20.654585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f31abedc78e4a908772334262499342', 'request_id': 'req-526dbb38-b91f-4892-95d7-2238000adf79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:20.690150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f31abedc78e4a908772334262499342', 'request_id': 'req-526dbb38-b91f-4892-95d7-2238000adf79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:20.721529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] The token cache is being invalidate because project 7f31abedc78e4a908772334262499342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:20.721825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:20.722913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-526dbb38-b91f-4892-95d7-2238000adf79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:20.724512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3337/6622] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:20 2022] DELETE /identity/v3/projects/7f31abedc78e4a908772334262499342 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:20.730138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66075b03-fea4-4380-93c7-df3a9ff390de None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:20.730738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66075b03-fea4-4380-93c7-df3a9ff390de None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:20.730866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66075b03-fea4-4380-93c7-df3a9ff390de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:20.777707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-66075b03-fea4-4380-93c7-df3a9ff390de None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:20.779499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66075b03-fea4-4380-93c7-df3a9ff390de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:20.857554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:20.921587 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3286/6623] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:20.944588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ee7d-4ada-4f48-895b-0052d58ccb1e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:20.948683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ee7d-4ada-4f48-895b-0052d58ccb1e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:20.949094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ee7d-4ada-4f48-895b-0052d58ccb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:21.087908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1b99ee7d-4ada-4f48-895b-0052d58ccb1e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:21.091768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1b99ee7d-4ada-4f48-895b-0052d58ccb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:21.139276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66075b03-fea4-4380-93c7-df3a9ff390de None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:21.152242 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3338/6624] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:21.154008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66075b03-fea4-4380-93c7-df3a9ff390de None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:21.159655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:21.160759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:21.161022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:21.161381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:21.162493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:21.162699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:21.164979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:21.203695 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3287/6625] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:21.206873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99ee7d-4ada-4f48-895b-0052d58ccb1e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:21.239321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99ee7d-4ada-4f48-895b-0052d58ccb1e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:21.257398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:21.258329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:21.258802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:21.259228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:21.260041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:21.268467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:21.361709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3339/6626] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:21 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:21.364121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:21.390115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ac7ae-8342-485f-a199-bc36dad0345a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:21.567665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:21.569936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:21.571756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:21.572898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:21.574221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:21.580253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:21.685218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:21.711583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3288/6627] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:21 2022] POST /identity/v3/users => generated 285 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:21.716527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c76e05-8cc7-420e-ba09-669f1d1607c9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:21.754099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:21.756227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:21.756950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:21.757563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:21.759697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:21.762691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:21.796126 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3340/6628] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:21:21.797454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:21.810617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c063a41-3a61-4a35-a144-180b48f84ce4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:21.836195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ef6c1-6e45-4c65-8638-f1f82239f701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:21.837057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ef6c1-6e45-4c65-8638-f1f82239f701 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:21.837227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ef6c1-6e45-4c65-8638-f1f82239f701 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:21.837389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ef6c1-6e45-4c65-8638-f1f82239f701 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:21.838086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ef6c1-6e45-4c65-8638-f1f82239f701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:21.856564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ef6c1-6e45-4c65-8638-f1f82239f701 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:21.915549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:21.924705 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3289/6629] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:21.957522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab530c-c800-4a23-9567-abf93085e8c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:21.980965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:21.981842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:21.982183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:21.982527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] PATH_INFO: `/v3/projects/ef5ecf11300a46d28205d24d59fadbc0/users/27bfd1734cd9400586d3da90d5cb9fed/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:22.036273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef5ecf11300a46d28205d24d59fadbc0, user_id=27bfd1734cd9400586d3da90d5cb9fed, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:22.041400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:22.045574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:22.080821 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3341/6630] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:21 2022] PUT /identity/v3/projects/ef5ecf11300a46d28205d24d59fadbc0/users/27bfd1734cd9400586d3da90d5cb9fed/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:22.090178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499fddb-36e7-440a-ab93-74d8128230d1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:22.090464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499fddb-36e7-440a-ab93-74d8128230d1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:22.090713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499fddb-36e7-440a-ab93-74d8128230d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:22.181940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a499fddb-36e7-440a-ab93-74d8128230d1 None None] MFA Rules not processed for user `27bfd1734cd9400586d3da90d5cb9fed`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:22.183217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a499fddb-36e7-440a-ab93-74d8128230d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:22.372588 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3290/6631] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:22.373915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:22.537749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1974-0f84-4212-b7e7-33c4115735da admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:22.546032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:22.547028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:22.547407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:22.562037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:22.579465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:22.588300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:22.728784 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3342/6632] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:22 2022] POST /identity/v3/projects => generated 365 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:22.729755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499fddb-36e7-440a-ab93-74d8128230d1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:22.747978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499fddb-36e7-440a-ab93-74d8128230d1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:22.881014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:22.962694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:22.963347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:22.963725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:22.963915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:22.964535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:22.966523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:23.030772 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3291/6633] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:22 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:23.036118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:23.060733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e02820-f611-4f08-8941-34f135f65e14 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:23.069160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:23.070421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:23.070892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:23.071250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:23.073093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:23.074951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:23.097526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3343/6634] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:23.103442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:23.136931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd3ad8-df9b-4bb3-b10a-23cf870cefa5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:23.163801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:23.164838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:23.165262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:23.165717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] PATH_INFO: `/v3/projects/a7acb7054d8945d28e96ed6aca92f2ee/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:23.195105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7acb7054d8945d28e96ed6aca92f2ee, user_id=2a982e7f9eaf4636a491718641ee0294, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:23.198295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:23.238426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3292/6635] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:23 2022] PUT /identity/v3/projects/a7acb7054d8945d28e96ed6aca92f2ee/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:23.242566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:23.269369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de287b5f-e28c-4987-9712-344ac6d17d3a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:23.446047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d73d0f-e385-4c01-877f-4667551b0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:23.449922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d73d0f-e385-4c01-877f-4667551b0eac admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:23.450357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d73d0f-e385-4c01-877f-4667551b0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:23.452123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d73d0f-e385-4c01-877f-4667551b0eac admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:23.452971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d73d0f-e385-4c01-877f-4667551b0eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:23.454871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d73d0f-e385-4c01-877f-4667551b0eac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:23.482936 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3344/6636] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:23.484215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:23.503410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf5e5b-e86b-496b-a46e-31b515bed11e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:23.510481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:23.511358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:23.511882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:23.512248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] PATH_INFO: `/v3/projects/a7acb7054d8945d28e96ed6aca92f2ee/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:23.514359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7acb7054d8945d28e96ed6aca92f2ee, user_id=2a982e7f9eaf4636a491718641ee0294, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:23.517169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:23.542881 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3293/6637] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:23 2022] PUT /identity/v3/projects/a7acb7054d8945d28e96ed6aca92f2ee/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:23.545496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7f028-1510-49e9-a7e3-8283ccd3a9f4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:23.546182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7f028-1510-49e9-a7e3-8283ccd3a9f4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:23.546551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7f028-1510-49e9-a7e3-8283ccd3a9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:23.606032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-80f7f028-1510-49e9-a7e3-8283ccd3a9f4 None None] MFA Rules not processed for user `2a982e7f9eaf4636a491718641ee0294`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:23.609033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-80f7f028-1510-49e9-a7e3-8283ccd3a9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:23.743070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:23.794037 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3345/6638] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:23.798403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:24.057765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485371-e70f-4031-91b1-afce9a82efc5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:24.066441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:24.067096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:24.067503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:24.067926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:24.069270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:24.071797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:24.125990 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3294/6639] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:23 2022] POST /identity/v3/projects => generated 367 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:24.126905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7f028-1510-49e9-a7e3-8283ccd3a9f4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:24.170207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7f028-1510-49e9-a7e3-8283ccd3a9f4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:24.345867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:24.346927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:24.347056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:24.347410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:24.352413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:24.354597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:24.418458 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3346/6640] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:24 2022] POST /identity/v3/users => generated 284 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:24.425978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:24.453207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562b249-02cd-4aa9-ad60-93aa2e63f6e0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:24.473104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:24.473814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:24.478342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:24.478653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:24.479354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:24.483555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:24.520637 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3295/6641] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:24.522684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:24.539511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6fdb8-dbd5-4ac3-99ed-ab3ffa2f1c8e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:24.553844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da0008-3475-43e6-b9a3-bb28812653c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:24.554488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da0008-3475-43e6-b9a3-bb28812653c6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:24.554767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da0008-3475-43e6-b9a3-bb28812653c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:24.555041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da0008-3475-43e6-b9a3-bb28812653c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:24.555701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da0008-3475-43e6-b9a3-bb28812653c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:24.562256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da0008-3475-43e6-b9a3-bb28812653c6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:24.598769 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3347/6642] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:24.602762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:24.618243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73a7a-4ee7-4bf5-b8a2-7947d8345f16 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:24.632891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:24.633602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:24.633884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:24.636753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] PATH_INFO: `/v3/projects/d2553392831841a88631ca974179bc68/users/770ce92d416741eb83285aca9d262adc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:24.684150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2553392831841a88631ca974179bc68, user_id=770ce92d416741eb83285aca9d262adc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:24.687032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:24.746352 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3296/6643] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:24 2022] PUT /identity/v3/projects/d2553392831841a88631ca974179bc68/users/770ce92d416741eb83285aca9d262adc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:24.757264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408444a-0bdf-4f4b-b13c-7b4e79fa090f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:24.761965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408444a-0bdf-4f4b-b13c-7b4e79fa090f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:24.762302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408444a-0bdf-4f4b-b13c-7b4e79fa090f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:24.877769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c408444a-0bdf-4f4b-b13c-7b4e79fa090f None None] MFA Rules not processed for user `770ce92d416741eb83285aca9d262adc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:24.879190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c408444a-0bdf-4f4b-b13c-7b4e79fa090f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:25.039281 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3348/6644] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:25.040715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:25.155474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:25.223219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010f09b-8d63-4732-a127-298d3b1eeff4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:25.238397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:25.239974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:25.241608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:25.242001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:25.242805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:25.244571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:25.291052 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3297/6645] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:21:25 2022] POST /identity/v3/domains => generated 276 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:25.298802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408444a-0bdf-4f4b-b13c-7b4e79fa090f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:25.319980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408444a-0bdf-4f4b-b13c-7b4e79fa090f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:25.523917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:25.524914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:25.525424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:25.525852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:25.526823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:25.532360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:25.542389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:25.572170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:25.585956 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3349/6646] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:25 2022] POST /identity/v3/roles => generated 239 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:25.615827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63b9aa-a6f6-485a-b180-e2801cc30e26 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:25.627507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:25.628253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:25.628537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:25.628863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1dd998911e474be4b6a4fd49423c9115/users/770ce92d416741eb83285aca9d262adc/roles/1ad2a0e9e249467f952358b56ffd518a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:25.646163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=1dd998911e474be4b6a4fd49423c9115, user_id=770ce92d416741eb83285aca9d262adc, role_id=1ad2a0e9e249467f952358b56ffd518a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:25.649671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:25.678256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:25.679427 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3298/6647] 10.222.0.26 () {60 vars in 1480 bytes} [Tue Sep 27 10:21:25 2022] PUT /identity/v3/domains/1dd998911e474be4b6a4fd49423c9115/users/770ce92d416741eb83285aca9d262adc/roles/1ad2a0e9e249467f952358b56ffd518a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:25.692623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e881c-9c91-4820-969b-f1c7de24f81c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:25.857399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:25.858094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:25.858323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:25.858601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:25.865062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:25.869283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:25.916492 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3350/6648] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:21:25 2022] POST /identity/v3/domains => generated 276 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:25.918502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:25.946174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5660945-5ae2-4dce-b4fb-8ef95eb88f89 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:26.159479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:26.160539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:26.160920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:26.161330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:26.162561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:26.164422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:26.243185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:26.244037 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3299/6649] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:25 2022] POST /identity/v3/roles => generated 239 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:26.264630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713854c0-6d2c-47a3-b1a5-bd1218f60f9b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:26.270350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:26.273800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:26.274476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:26.275036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:26.275999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:26.278747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:26.342451 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3351/6650] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:21:26 2022] POST /identity/v3/groups => generated 327 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:26.347717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:26.367827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6fc76-b17d-4e66-9b1a-5a6e46566f2e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:26.452395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:26.457956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:26.460068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:26.460538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups/0d005cc006354d57b0bf20839bcd21f2/users/770ce92d416741eb83285aca9d262adc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:26.488811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=0d005cc006354d57b0bf20839bcd21f2, user_id=770ce92d416741eb83285aca9d262adc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:26.491735 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:21:26.491735 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:21:26.491735 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:21:26.493616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:26.569600 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3300/6651] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:21:26 2022] PUT /identity/v3/groups/0d005cc006354d57b0bf20839bcd21f2/users/770ce92d416741eb83285aca9d262adc => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:26.574048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:26.614072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc85dfb-eeff-4578-9381-70f89f33fb92 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:26.740570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:26.741329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:26.741619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:26.741999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1d04045207d343c3ba8b3a3e06799e73/groups/0d005cc006354d57b0bf20839bcd21f2/roles/11f7152da45a43038d78a2b0c3306cd6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:26.744603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=1d04045207d343c3ba8b3a3e06799e73, group_id=0d005cc006354d57b0bf20839bcd21f2, role_id=11f7152da45a43038d78a2b0c3306cd6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:26.747673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:26.791868 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3352/6652] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:26 2022] PUT /identity/v3/domains/1d04045207d343c3ba8b3a3e06799e73/groups/0d005cc006354d57b0bf20839bcd21f2/roles/11f7152da45a43038d78a2b0c3306cd6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:26.796052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:26.811788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c8412-4d1a-4b78-9004-705a630c148a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:27.023113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:27.027510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:27.029362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:27.029824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:27.032844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:27.034800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:27.101627 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3301/6653] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:21:26 2022] POST /identity/v3/domains => generated 277 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:27.101867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:27.118158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ebcc6-20e5-46f7-9db7-936d5073f8ab tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:27.263870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:27.264764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:27.265054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:27.265336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:27.266022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:27.268065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:27.272001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:27.305856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3353/6654] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:27 2022] POST /identity/v3/roles => generated 240 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:27.309944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:27.335018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59692d53-f98c-4863-aa95-30d437cc2063 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:27.341219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:27.342124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:27.342459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:27.342783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/95271a3806a543bfa206fbba960101c7/users/770ce92d416741eb83285aca9d262adc/roles/bdcf93dcf9d748b68e213bd2cacd3f89` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:27.351583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:27.358637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=95271a3806a543bfa206fbba960101c7, user_id=770ce92d416741eb83285aca9d262adc, role_id=bdcf93dcf9d748b68e213bd2cacd3f89)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:27.361417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:27.391710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3302/6655] 10.222.0.26 () {60 vars in 1480 bytes} [Tue Sep 27 10:21:27 2022] PUT /identity/v3/domains/95271a3806a543bfa206fbba960101c7/users/770ce92d416741eb83285aca9d262adc/roles/bdcf93dcf9d748b68e213bd2cacd3f89 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:27.395962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:27.409453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368e447-5b11-44f7-b679-5713b5a2e4f3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:27.530178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:27.531141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:27.531720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:27.533308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:27.535485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:27.539747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:27.580369 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3354/6656] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:21:27 2022] POST /identity/v3/domains => generated 275 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:27.582349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:27.598465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdd0a3-7839-4631-9529-74bc4d961732 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:27.738090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:27.739021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:27.739425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:27.741226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:27.742560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:27.743868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:27.776879 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3303/6657] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:27 2022] POST /identity/v3/roles => generated 240 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:27.780884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:27.807397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdd93a-9e38-4be8-a827-71630282ca1c tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:27.818761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:27.820193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:27.820481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:27.820805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:27.821458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:27.824035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:27.878146 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3355/6658] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:21:27 2022] POST /identity/v3/groups => generated 327 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:27.878381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:27.890372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ccafe-3f27-4764-b700-51a46d7a7913 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:27.903326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:27.904186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:27.904752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:27.905141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups/1b3e4121ebd94bf78d789e79a6793686/users/770ce92d416741eb83285aca9d262adc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:27.925545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=1b3e4121ebd94bf78d789e79a6793686, user_id=770ce92d416741eb83285aca9d262adc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:27.932835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:28.011604 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3304/6659] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:21:27 2022] PUT /identity/v3/groups/1b3e4121ebd94bf78d789e79a6793686/users/770ce92d416741eb83285aca9d262adc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:28.016480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:28.040584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686b181-2315-48c5-adf7-40bf0c27cf02 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:28.194655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:28.196130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:28.196599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:28.198533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/6934953d209d4a10ba7952d956a4a39c/groups/1b3e4121ebd94bf78d789e79a6793686/roles/d9e5a33118684720a59444840b1e60e2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:28.201694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=6934953d209d4a10ba7952d956a4a39c, group_id=1b3e4121ebd94bf78d789e79a6793686, role_id=d9e5a33118684720a59444840b1e60e2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:28.204694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:28.234971 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3356/6660] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:28 2022] PUT /identity/v3/domains/6934953d209d4a10ba7952d956a4a39c/groups/1b3e4121ebd94bf78d789e79a6793686/roles/d9e5a33118684720a59444840b1e60e2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:28.239262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:28.538412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ebc6-c1c1-48df-ad33-4bcc55b70f7e tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:28.673167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:28.790181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '770ce92d416741eb83285aca9d262adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2553392831841a88631ca974179bc68', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:28.791014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:28.791300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:28.791660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] PATH_INFO: `/v3/auth/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:28.792296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:28.793482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:28.964044 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3305/6661] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:21:28 2022] GET /identity/v3/auth/domains => generated 1172 bytes in 725 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:28.969150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:29.066779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e433c61-bfcb-44c5-8dad-824638182120 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:29.072002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:29.073261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:29.073657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:29.074013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/6934953d209d4a10ba7952d956a4a39c/groups/1b3e4121ebd94bf78d789e79a6793686/roles/d9e5a33118684720a59444840b1e60e2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:29.076543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=6934953d209d4a10ba7952d956a4a39c, group_id=1b3e4121ebd94bf78d789e79a6793686, role_id=d9e5a33118684720a59444840b1e60e2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:29.079720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:29.083590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:29.087871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role d9e5a33118684720a59444840b1e60e2 was removed from group 1b3e4121ebd94bf78d789e79a6793686 on domain 6934953d209d4a10ba7952d956a4a39c. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:29.088177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:29.089391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:29.116341 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3357/6662] 10.222.0.26 () {58 vars in 1467 bytes} [Tue Sep 27 10:21:28 2022] DELETE /identity/v3/domains/6934953d209d4a10ba7952d956a4a39c/groups/1b3e4121ebd94bf78d789e79a6793686/roles/d9e5a33118684720a59444840b1e60e2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:29.121718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:29.124578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:29.156547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc212da-5eb0-477f-89c5-8603faae691c tempest-TokensV3TestJSON-593646340 tempest-TokensV3TestJSON-593646340-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:29.235996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:29.237318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:29.237820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:29.238646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups/1b3e4121ebd94bf78d789e79a6793686/users/770ce92d416741eb83285aca9d262adc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:29.246213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=1b3e4121ebd94bf78d789e79a6793686, user_id=770ce92d416741eb83285aca9d262adc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:29.247925 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:21:29.247925 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:21:29.247925 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:21:29.250135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:29.272484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '770ce92d416741eb83285aca9d262adc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:29.303129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3306/6663] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:21:29 2022] DELETE /identity/v3/groups/1b3e4121ebd94bf78d789e79a6793686/users/770ce92d416741eb83285aca9d262adc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:29.307669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:29.329860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d560ec4-7146-4656-9a93-b9f310643477 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:29.451869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:29.452559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:29.452865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:29.453147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups/1b3e4121ebd94bf78d789e79a6793686` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:29.455214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_group(group_id=1b3e4121ebd94bf78d789e79a6793686)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:29.465557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:29.650072 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3358/6664] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:21:29 2022] DELETE /identity/v3/groups/1b3e4121ebd94bf78d789e79a6793686 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:29.652525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:29.675152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f6fef-06b1-45b5-9d0f-30dab26390e3 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:29.839702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:29.893507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:29.894569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:29.895033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:29.895597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles/d9e5a33118684720a59444840b1e60e2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:29.901409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d9e5a33118684720a59444840b1e60e2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:29.912340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:30.115931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role d9e5a33118684720a59444840b1e60e2 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:30.118006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:30.125206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:30.134666 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3307/6665] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:29 2022] DELETE /identity/v3/roles/d9e5a33118684720a59444840b1e60e2 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:30.139955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:30.142468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:30.187901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c02da-6fb9-422c-9524-c0adc022580d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:30.402145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:30.406721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:30.410354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:30.411325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/6934953d209d4a10ba7952d956a4a39c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:30.416297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6934953d209d4a10ba7952d956a4a39c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:30.420180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:30.513144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6934953d209d4a10ba7952d956a4a39c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:30.516490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6934953d209d4a10ba7952d956a4a39c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:30.608426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6934953d209d4a10ba7952d956a4a39c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:30.610580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6934953d209d4a10ba7952d956a4a39c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:30.639174 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3359/6666] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:30 2022] PATCH /identity/v3/domains/6934953d209d4a10ba7952d956a4a39c => generated 276 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:30.640552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:30.667804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847f7bf-01ca-4100-917f-f657040786f9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:30.817655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:30.818510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:30.818919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:30.819307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/6934953d209d4a10ba7952d956a4a39c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:30.850890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6934953d209d4a10ba7952d956a4a39c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:30.853079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:30.893372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6934953d209d4a10ba7952d956a4a39c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:30.900939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6934953d209d4a10ba7952d956a4a39c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.032157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6934953d209d4a10ba7952d956a4a39c', 'request_id': 'req-423d458e-9f26-4bcb-a377-50e8080f6531'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.069073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6934953d209d4a10ba7952d956a4a39c', 'request_id': 'req-423d458e-9f26-4bcb-a377-50e8080f6531'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.096380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6934953d209d4a10ba7952d956a4a39c', 'request_id': 'req-423d458e-9f26-4bcb-a377-50e8080f6531'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.127091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] The token cache is being invalidate because project 6934953d209d4a10ba7952d956a4a39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:31.131861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.133107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.139003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:31.156872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6934953d209d4a10ba7952d956a4a39c', 'request_id': 'req-423d458e-9f26-4bcb-a377-50e8080f6531'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.158021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6934953d209d4a10ba7952d956a4a39c', 'request_id': 'req-423d458e-9f26-4bcb-a377-50e8080f6531'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.170262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6934953d209d4a10ba7952d956a4a39c', 'request_id': 'req-423d458e-9f26-4bcb-a377-50e8080f6531'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.190588 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3308/6667] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:21:30 2022] DELETE /identity/v3/domains/6934953d209d4a10ba7952d956a4a39c => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:31.191337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:31.193428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:31.217224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd74a4-9398-46d6-9159-76b6a7a896d8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:31.240180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:31.352058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:31.352879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:31.353166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:31.353414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/95271a3806a543bfa206fbba960101c7/users/770ce92d416741eb83285aca9d262adc/roles/bdcf93dcf9d748b68e213bd2cacd3f89` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:31.356401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=95271a3806a543bfa206fbba960101c7, user_id=770ce92d416741eb83285aca9d262adc, role_id=bdcf93dcf9d748b68e213bd2cacd3f89)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:31.359390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:31.367196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role bdcf93dcf9d748b68e213bd2cacd3f89 was removed from user 770ce92d416741eb83285aca9d262adc on domain 95271a3806a543bfa206fbba960101c7. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:31.367485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.368204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.402526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3360/6668] 10.222.0.26 () {58 vars in 1464 bytes} [Tue Sep 27 10:21:31 2022] DELETE /identity/v3/domains/95271a3806a543bfa206fbba960101c7/users/770ce92d416741eb83285aca9d262adc/roles/bdcf93dcf9d748b68e213bd2cacd3f89 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:31.407949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:31.410327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:31.424321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d458e-9f26-4bcb-a377-50e8080f6531 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:31.530173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:31.532197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:31.532705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:31.533356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles/bdcf93dcf9d748b68e213bd2cacd3f89` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:31.535943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bdcf93dcf9d748b68e213bd2cacd3f89)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:31.539486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:31.597663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role bdcf93dcf9d748b68e213bd2cacd3f89 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:31.598219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.601134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.607544 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3309/6669] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:31 2022] DELETE /identity/v3/roles/bdcf93dcf9d748b68e213bd2cacd3f89 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:31.610859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:31.613004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:31.628694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32570e08-ff72-426e-9ec7-6ac9fa750224 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:31.803215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:31.804636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:31.805181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:31.805690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/95271a3806a543bfa206fbba960101c7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:31.807480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=95271a3806a543bfa206fbba960101c7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:31.811126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:31.877743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:31.879177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:31.897237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '95271a3806a543bfa206fbba960101c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.897237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '95271a3806a543bfa206fbba960101c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.930976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '95271a3806a543bfa206fbba960101c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.931721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '95271a3806a543bfa206fbba960101c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:31.935879 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3361/6670] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:31 2022] PATCH /identity/v3/domains/95271a3806a543bfa206fbba960101c7 => generated 278 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:31.940537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:31.942495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:31.979155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061928c1-1818-456a-ba2e-7cda70fc8824 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:32.075595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:32.076385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:32.076695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:32.077057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:32.085791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add18fc2-55e1-4174-922d-ad481c8879d9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:32.099168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:32.100842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:32.101597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:32.101890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:32.102158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/95271a3806a543bfa206fbba960101c7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:32.122511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=95271a3806a543bfa206fbba960101c7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:32.122511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:32.123970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:32.133926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:32.178530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '95271a3806a543bfa206fbba960101c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.211017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '95271a3806a543bfa206fbba960101c7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.300212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:32.302516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:32.347084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95271a3806a543bfa206fbba960101c7', 'request_id': 'req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.347952 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3310/6671] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:21:32.380471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95271a3806a543bfa206fbba960101c7', 'request_id': 'req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.388628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95271a3806a543bfa206fbba960101c7', 'request_id': 'req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.404473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] The token cache is being invalidate because project 95271a3806a543bfa206fbba960101c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:32.405516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.406456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.419480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '95271a3806a543bfa206fbba960101c7', 'request_id': 'req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.420571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '95271a3806a543bfa206fbba960101c7', 'request_id': 'req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.428565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '95271a3806a543bfa206fbba960101c7', 'request_id': 'req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.443813 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3362/6672] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:21:31 2022] DELETE /identity/v3/domains/95271a3806a543bfa206fbba960101c7 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:32.448640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:32.463144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a58c40-7963-4a49-8b3b-358a35bb5b85 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:32.591747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:32.592535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:32.592931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:32.593573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1d04045207d343c3ba8b3a3e06799e73/groups/0d005cc006354d57b0bf20839bcd21f2/roles/11f7152da45a43038d78a2b0c3306cd6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:32.597222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=1d04045207d343c3ba8b3a3e06799e73, group_id=0d005cc006354d57b0bf20839bcd21f2, role_id=11f7152da45a43038d78a2b0c3306cd6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:32.601657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:32.612701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role 11f7152da45a43038d78a2b0c3306cd6 was removed from group 0d005cc006354d57b0bf20839bcd21f2 on domain 1d04045207d343c3ba8b3a3e06799e73. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:32.613584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.615800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.652273 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3311/6673] 10.222.0.26 () {58 vars in 1467 bytes} [Tue Sep 27 10:21:32 2022] DELETE /identity/v3/domains/1d04045207d343c3ba8b3a3e06799e73/groups/0d005cc006354d57b0bf20839bcd21f2/roles/11f7152da45a43038d78a2b0c3306cd6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:32.656735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:32.658690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:32.677819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbd5f3-4d7f-4ac7-849d-b79b14b0b7a1 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:32.778541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:32.785021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:32.827206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:32.827892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:32.828167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:32.828439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups/0d005cc006354d57b0bf20839bcd21f2/users/770ce92d416741eb83285aca9d262adc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:32.835837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=0d005cc006354d57b0bf20839bcd21f2, user_id=770ce92d416741eb83285aca9d262adc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:32.837853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:32.861507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '770ce92d416741eb83285aca9d262adc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:32.891297 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3363/6674] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:21:32 2022] DELETE /identity/v3/groups/0d005cc006354d57b0bf20839bcd21f2/users/770ce92d416741eb83285aca9d262adc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:32.895514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:32.898248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:32.899204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7378732-6d44-48a7-8dfe-07e820bb82f7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:32.985330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:33.005400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:33.044486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:33.046737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:33.047138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:33.048840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] PATH_INFO: `/v3/users/0c5bbb6cec7042eda3dfc7a6839dfeda` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:33.054508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5bbb6cec7042eda3dfc7a6839dfeda)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:33.057190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:33.085232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cf9c9-900e-4eb2-ad30-21cb87962e55 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:33.111040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:33.112284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:33.112920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:33.113041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/groups/0d005cc006354d57b0bf20839bcd21f2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:33.117488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_group(group_id=0d005cc006354d57b0bf20839bcd21f2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:33.119446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:33.186672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5bbb6cec7042eda3dfc7a6839dfeda'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.189284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5bbb6cec7042eda3dfc7a6839dfeda'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.223664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5bbb6cec7042eda3dfc7a6839dfeda'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.245411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5bbb6cec7042eda3dfc7a6839dfeda'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.249326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3364/6675] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:21:32 2022] DELETE /identity/v3/groups/0d005cc006354d57b0bf20839bcd21f2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:33.250529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:33.252267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:33.275431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4f73fd4c-bad1-471f-958e-0ed500001125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5bbb6cec7042eda3dfc7a6839dfeda'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.291379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3312/6676] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:32 2022] DELETE /identity/v3/users/0c5bbb6cec7042eda3dfc7a6839dfeda => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:33.297949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730ae7-aaf7-4a2c-a749-72f9a480b8f9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:33.299152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730ae7-aaf7-4a2c-a749-72f9a480b8f9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:33.302514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730ae7-aaf7-4a2c-a749-72f9a480b8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:33.345506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70a285-a253-4fef-87cf-41db34327295 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:33.347210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-77730ae7-aaf7-4a2c-a749-72f9a480b8f9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:33.350037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-77730ae7-aaf7-4a2c-a749-72f9a480b8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:33.464685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:33.464685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:33.464685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:33.465519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles/11f7152da45a43038d78a2b0c3306cd6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:33.465519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=11f7152da45a43038d78a2b0c3306cd6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:33.465519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:33.511486 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3313/6677] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:33.531329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role 11f7152da45a43038d78a2b0c3306cd6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:33.531525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.532313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.538836 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3365/6678] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:33 2022] DELETE /identity/v3/roles/11f7152da45a43038d78a2b0c3306cd6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:33.540033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77730ae7-aaf7-4a2c-a749-72f9a480b8f9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:33.561096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77730ae7-aaf7-4a2c-a749-72f9a480b8f9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:33.561443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:33.565173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:33.586791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:33.725836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:33.727034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:33.728677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:33.729538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1d04045207d343c3ba8b3a3e06799e73` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:33.734389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=1d04045207d343c3ba8b3a3e06799e73)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:33.736029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:33.743613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:33.752212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589bd25-7f9f-41eb-81d5-779567a82bcb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:33.766915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:33.768389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:33.773224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:33.773754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:33.779197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.780277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.786221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:33.840244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.842239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:33.849136 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3314/6679] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:33 2022] PATCH /identity/v3/domains/1d04045207d343c3ba8b3a3e06799e73 => generated 277 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:21:33.853415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:33.854968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:33.954032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790a02f-b83c-48cb-b23c-3f60fb7d5ef8 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:33.961402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:33.962588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:33.963549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:33.963932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1d04045207d343c3ba8b3a3e06799e73` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:33.994557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=1d04045207d343c3ba8b3a3e06799e73)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:33.998125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:34.000718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:34.003015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:34.028744 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3366/6680] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:34.064642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.072367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.212071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73', 'request_id': 'req-dedd7c94-79a3-49d2-9947-a79f947a0b4d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.225351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73', 'request_id': 'req-dedd7c94-79a3-49d2-9947-a79f947a0b4d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.237643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73', 'request_id': 'req-dedd7c94-79a3-49d2-9947-a79f947a0b4d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.255143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] The token cache is being invalidate because project 1d04045207d343c3ba8b3a3e06799e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:34.256736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.257697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.269607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73', 'request_id': 'req-dedd7c94-79a3-49d2-9947-a79f947a0b4d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.271053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73', 'request_id': 'req-dedd7c94-79a3-49d2-9947-a79f947a0b4d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.277115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '1d04045207d343c3ba8b3a3e06799e73', 'request_id': 'req-dedd7c94-79a3-49d2-9947-a79f947a0b4d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.287221 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3315/6681] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:21:33 2022] DELETE /identity/v3/domains/1d04045207d343c3ba8b3a3e06799e73 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:34.291368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:34.293385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:34.313930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b9bbb-4621-4ce7-985d-d25c559f1342 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:34.407600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:34.408231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:34.408440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:34.408772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1dd998911e474be4b6a4fd49423c9115/users/770ce92d416741eb83285aca9d262adc/roles/1ad2a0e9e249467f952358b56ffd518a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:34.415141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=1dd998911e474be4b6a4fd49423c9115, user_id=770ce92d416741eb83285aca9d262adc, role_id=1ad2a0e9e249467f952358b56ffd518a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:34.415141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:34.421282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role 1ad2a0e9e249467f952358b56ffd518a was removed from user 770ce92d416741eb83285aca9d262adc on domain 1dd998911e474be4b6a4fd49423c9115. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:34.421749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.422632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.444779 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3367/6682] 10.222.0.26 () {58 vars in 1464 bytes} [Tue Sep 27 10:21:34 2022] DELETE /identity/v3/domains/1dd998911e474be4b6a4fd49423c9115/users/770ce92d416741eb83285aca9d262adc/roles/1ad2a0e9e249467f952358b56ffd518a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:34.450027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:34.453100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:34.467666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:34.472493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd7c94-79a3-49d2-9947-a79f947a0b4d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:34.562338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:34.563018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:34.563303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:34.563578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles/1ad2a0e9e249467f952358b56ffd518a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:34.565094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1ad2a0e9e249467f952358b56ffd518a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:34.566745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:34.642064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role 1ad2a0e9e249467f952358b56ffd518a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:34.642372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.643433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.646628 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3316/6683] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:34 2022] DELETE /identity/v3/roles/1ad2a0e9e249467f952358b56ffd518a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:34.650881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:34.652352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:34.674015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76e8d5-10d3-48e6-b18f-15210da70d15 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:34.785279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:34.805590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:34.807240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:34.877921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:34.879519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:34.883790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:34.884072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1dd998911e474be4b6a4fd49423c9115` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:34.885728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=1dd998911e474be4b6a4fd49423c9115)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:34.889297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:34.967671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '1dd998911e474be4b6a4fd49423c9115'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:34.970954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '1dd998911e474be4b6a4fd49423c9115'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.094708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27cb59-0abf-473d-b6ed-f613de3a9691 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:35.098036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '1dd998911e474be4b6a4fd49423c9115'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.099549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '1dd998911e474be4b6a4fd49423c9115'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.110601 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3368/6684] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:34 2022] PATCH /identity/v3/domains/1dd998911e474be4b6a4fd49423c9115 => generated 277 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:35.117648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:35.121134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:35.259372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:35.260191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:35.260477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:35.260831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] PATH_INFO: `/v3/projects/0e592e34dc61402f8919ad1b64fbe041` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:35.263495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e592e34dc61402f8919ad1b64fbe041)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:35.265543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:35.314827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:35.372211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e592e34dc61402f8919ad1b64fbe041', 'request_id': 'req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.385547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e592e34dc61402f8919ad1b64fbe041', 'request_id': 'req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.401555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec272d-c911-4ae2-98c3-fd2907a9c7cf tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:35.404408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e592e34dc61402f8919ad1b64fbe041', 'request_id': 'req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.421983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] The token cache is being invalidate because project 0e592e34dc61402f8919ad1b64fbe041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:35.434778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.435824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.438333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3317/6685] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:34 2022] DELETE /identity/v3/projects/0e592e34dc61402f8919ad1b64fbe041 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:35.439296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:35.443760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:35.566923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:35.567687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:35.567974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:35.569680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1dd998911e474be4b6a4fd49423c9115` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:35.591535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=1dd998911e474be4b6a4fd49423c9115)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:35.593279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:35.623300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf888d33-d11d-4b88-b3fa-5a62ebcc6ff1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:35.635001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fd9b0-d929-44e0-96de-59e3aeb0f9cd tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51d9a02f69ed4d5cb3368407284dc1cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526c4d6a63004031baef7e7b321a31ac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:35.637266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd9b0-d929-44e0-96de-59e3aeb0f9cd tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:35.637560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd9b0-d929-44e0-96de-59e3aeb0f9cd tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:35.637841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd9b0-d929-44e0-96de-59e3aeb0f9cd tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:35.638525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd9b0-d929-44e0-96de-59e3aeb0f9cd tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:35.638834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd9b0-d929-44e0-96de-59e3aeb0f9cd tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:35.643391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd9b0-d929-44e0-96de-59e3aeb0f9cd tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:35.670390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '1dd998911e474be4b6a4fd49423c9115'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.678385 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3318/6686] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:35.679755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4f0e-e241-462d-9734-f59ece45e718 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:35.680049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4f0e-e241-462d-9734-f59ece45e718 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:35.682512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4f0e-e241-462d-9734-f59ece45e718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:35.697433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '1dd998911e474be4b6a4fd49423c9115'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.733735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-298a4f0e-e241-462d-9734-f59ece45e718 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:35.735050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-298a4f0e-e241-462d-9734-f59ece45e718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:35.746719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:35.804564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd998911e474be4b6a4fd49423c9115', 'request_id': 'req-afd3b408-bea8-4789-8622-be6e85397ad9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.821817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd998911e474be4b6a4fd49423c9115', 'request_id': 'req-afd3b408-bea8-4789-8622-be6e85397ad9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.835325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd998911e474be4b6a4fd49423c9115', 'request_id': 'req-afd3b408-bea8-4789-8622-be6e85397ad9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.859581 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3319/6687] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:35.860452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4f0e-e241-462d-9734-f59ece45e718 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:35.869781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] The token cache is being invalidate because project 1dd998911e474be4b6a4fd49423c9115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:35.870088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.870875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.894660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '1dd998911e474be4b6a4fd49423c9115', 'request_id': 'req-afd3b408-bea8-4789-8622-be6e85397ad9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.895624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '1dd998911e474be4b6a4fd49423c9115', 'request_id': 'req-afd3b408-bea8-4789-8622-be6e85397ad9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.900078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4f0e-e241-462d-9734-f59ece45e718 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:35.903950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-afd3b408-bea8-4789-8622-be6e85397ad9 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '1dd998911e474be4b6a4fd49423c9115', 'request_id': 'req-afd3b408-bea8-4789-8622-be6e85397ad9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:35.930901 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3369/6688] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:21:35 2022] DELETE /identity/v3/domains/1dd998911e474be4b6a4fd49423c9115 => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:35.933395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:35.933669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:35.933946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:35.986606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:35.987840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:36.012291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:36.057905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51d9a02f69ed4d5cb3368407284dc1cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526c4d6a63004031baef7e7b321a31ac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:36.062659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:36.063481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:36.064905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:36.069938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:36.079665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:36.150259 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3370/6689] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:36.151247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.155934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:36.195800 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3320/6690] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:35 2022] POST /identity/v3/projects => generated 349 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:36.195800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.329139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553d8cf-2e9b-40b1-82a3-afb1f7991844 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:36.358843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d1d61-fe6f-4e34-b8e5-e4fd3f3c05d5 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:36.366639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:36.367576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:36.367932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:36.368300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:36.369212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:36.370285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:36.373067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:36.389829 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3321/6691] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:36.390918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.395157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:36.425585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:36.466968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:36.467615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:36.467977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:36.468214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:36.468899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:36.473545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:36.505910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29a1f6-7f4e-49a9-b49e-6d7e11ae0365 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:36.511011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:36.511956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:36.512544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:36.512743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:36.512874 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3371/6692] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:36.513987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.514968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:36.519790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:36.652694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:36.654669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:36.685583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3322/6693] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:36.687126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.711158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4ca5-7656-40ae-8c21-9b6a6f9b2add tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:36.715040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:36.721939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51d9a02f69ed4d5cb3368407284dc1cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526c4d6a63004031baef7e7b321a31ac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:36.723874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:36.725274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:36.725624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:36.727783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:36.727783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:36.729204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5d03f-d8ac-4656-ab92-b21cc6abf260 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:36.732519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:36.745528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:36.746426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:36.746760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:36.747032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:36.748029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:36.751250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:36.765674 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3372/6694] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:36.766700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.792574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafc3ef-ee06-4854-adce-aacc2beead09 tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:36.798476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:36.799275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:36.799620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:36.799963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects/ef5ecf11300a46d28205d24d59fadbc0/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:36.804681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ef5ecf11300a46d28205d24d59fadbc0, user_id=2a982e7f9eaf4636a491718641ee0294, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:36.808116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:36.837747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3323/6695] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:36 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:36.840783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.854059 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3373/6696] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:36 2022] PUT /identity/v3/projects/ef5ecf11300a46d28205d24d59fadbc0/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:36.855403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:36.923549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0b1eb-0b7b-42ec-b335-532386427b07 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.024919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8fce4-0329-458a-bb74-06a24f04a8e6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.030701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.031356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.031635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.031908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.032490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.045505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.057686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.058523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.058644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.058906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.059559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.063070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.103569 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3324/6697] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:36 2022] POST /identity/v3/users => generated 290 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:37.104650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.124287 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3374/6698] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:21:36 2022] GET /identity/v3/auth/projects => generated 824 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:37.125776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.131448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effae00-e7b2-457f-b4b9-88395a49222e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.141727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.147561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878484ca-6397-4cf2-9e57-0da7d79fe970 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.154302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.155166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.155494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.155776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects/ef5ecf11300a46d28205d24d59fadbc0/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.156098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.156887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.157390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.158316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=ef5ecf11300a46d28205d24d59fadbc0, user_id=2a982e7f9eaf4636a491718641ee0294, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.158480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.160352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.162880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.170572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role 53aff494b6b44d5da69d62199ff0d237 was removed from user 2a982e7f9eaf4636a491718641ee0294 on project ef5ecf11300a46d28205d24d59fadbc0. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:37.171100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:37.172301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:37.177938 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3325/6699] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:37.181725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.184338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:37.189462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:37.199439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95e3d4-e441-4885-a7e8-0f0519b613b3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.200440 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3375/6700] 10.222.0.26 () {58 vars in 1467 bytes} [Tue Sep 27 10:21:37 2022] DELETE /identity/v3/projects/ef5ecf11300a46d28205d24d59fadbc0/users/2a982e7f9eaf4636a491718641ee0294/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:37.202077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.205372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:37.301234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9afb9-1acd-4e79-b8cf-ea984abb496a tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.315040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.315682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.315969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.316246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.316924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.319198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.325811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.326881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.328005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.328701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.330421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.333682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.361715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3326/6701] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:21:37.362785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.372779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:37.377803 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3376/6702] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:37 2022] POST /identity/v3/users => generated 321 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:37.388826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.389152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124fc2e-76a3-4758-bf99-9da368fd87c6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.407635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.408623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.408965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.409232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] PATH_INFO: `/v3/projects/ff1e17cb29c54a1aabe7da7009c347c5/users/2e3ba57b36d34b40bcfa113cb71ef8bc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.436539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f3c84-5d2b-4e21-bf2b-3ffed3587ec6 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.444290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.445176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.445481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.446070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.447222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.450439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.451123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1e17cb29c54a1aabe7da7009c347c5, user_id=2e3ba57b36d34b40bcfa113cb71ef8bc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.454743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.516633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3327/6703] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:37 2022] PUT /identity/v3/projects/ff1e17cb29c54a1aabe7da7009c347c5/users/2e3ba57b36d34b40bcfa113cb71ef8bc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:37.518440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.527924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:37.541075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3377/6704] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:37 2022] POST /identity/v3/projects => generated 364 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:37.542534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.553407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:37.678491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74efd1-83da-4440-82cc-bedb029e466a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.702692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3392a3-c7f8-46c5-8780-7de12767ccfd tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:37.711013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51d9a02f69ed4d5cb3368407284dc1cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '526c4d6a63004031baef7e7b321a31ac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.712115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.713410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.713691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] PATH_INFO: `/v3/projects/5392e2a7a34e4409958d476ee477b075` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.714976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5392e2a7a34e4409958d476ee477b075)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.717196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.758449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:37.759664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.760116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.760647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.764251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:37.785492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5392e2a7a34e4409958d476ee477b075', 'request_id': 'req-9284624c-026d-4589-8075-ff536a47af9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:37.796858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5392e2a7a34e4409958d476ee477b075', 'request_id': 'req-9284624c-026d-4589-8075-ff536a47af9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:37.803259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5392e2a7a34e4409958d476ee477b075', 'request_id': 'req-9284624c-026d-4589-8075-ff536a47af9c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:37.812520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:37.817928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] The token cache is being invalidate because project 5392e2a7a34e4409958d476ee477b075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:37.818211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:37.819143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9284624c-026d-4589-8075-ff536a47af9c tempest-RoutersIpV6AdminTest-986746364 tempest-RoutersIpV6AdminTest-986746364-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:37.821708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3378/6705] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:37 2022] DELETE /identity/v3/projects/5392e2a7a34e4409958d476ee477b075 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:37.828505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:37.828828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:37.829102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:37.865187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] MFA Rules not processed for user `2e3ba57b36d34b40bcfa113cb71ef8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:37.866716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:37.866842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:37.868521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:37.883593 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3328/6706] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:37.884775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.930723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:37.987081 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3379/6707] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:37.987081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:37.987081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:37.997267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966adac6-7114-4853-b875-d48368df1ab0 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.082645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dca28-1b6c-4a5e-82e8-73d3beefe90c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.087989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.088899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.089552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.089829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.091283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.092232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.092627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.092780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:38.104915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.104915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.104915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.152426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3329/6708] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:37 2022] POST /identity/v3/projects => generated 364 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:38.153855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:38.191172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.193547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.209821 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3380/6709] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:38.211322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:38.216871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:38.240892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a33e77-e080-48a3-b6a9-3786d15c4347 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.245624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.246539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.246842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.247095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.249726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab3f95-a70c-4dfa-a0d9-b98c58c8dad2 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.257743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.258388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.258693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.260253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.263148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:38.277038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.278495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.292042 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3330/6710] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:38.293450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:38.295162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:38.315576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:38.335828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.337396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.350619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3381/6711] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:38.378878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cb757-31b4-4c77-9ed6-084a1b4adaea service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.384506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.385276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.385593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.386286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.386554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.388217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.393311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:38.411105 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3331/6712] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:21:38 2022] POST /identity/v3/roles => generated 239 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:38.412590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:38.427717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9fb89-6348-487e-b095-bd41bb7a2d55 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.436810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.437637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.438066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.438489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects/7f6c2ef9843d4f6396b05865c37d1e14/users/16afdc0fcd9f47e787bf61fdf2e78ba0/roles/98ba668f78114223a37696e2980325b5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.458963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7f6c2ef9843d4f6396b05865c37d1e14, user_id=16afdc0fcd9f47e787bf61fdf2e78ba0, role_id=98ba668f78114223a37696e2980325b5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.463141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.491415 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3382/6713] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:38 2022] PUT /identity/v3/projects/7f6c2ef9843d4f6396b05865c37d1e14/users/16afdc0fcd9f47e787bf61fdf2e78ba0/roles/98ba668f78114223a37696e2980325b5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:38.492952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:38.506395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0970c-3404-40e7-8a6e-f82caf27e375 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.529108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:38.610605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5d089-22f4-4be0-b896-839607ed5670 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.612365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d089-22f4-4be0-b896-839607ed5670 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.613567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d089-22f4-4be0-b896-839607ed5670 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.614348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d089-22f4-4be0-b896-839607ed5670 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects/c6d1b138e19c4fba961ecb54054cd127/users/16afdc0fcd9f47e787bf61fdf2e78ba0/roles/98ba668f78114223a37696e2980325b5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.614989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ec656-5564-411c-adad-3ff493ae159d tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:38.618107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5d089-22f4-4be0-b896-839607ed5670 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c6d1b138e19c4fba961ecb54054cd127, user_id=16afdc0fcd9f47e787bf61fdf2e78ba0, role_id=98ba668f78114223a37696e2980325b5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.622433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5d089-22f4-4be0-b896-839607ed5670 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.623259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:38.624185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.624721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.625256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.627318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:38.650653 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3332/6714] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:38 2022] PUT /identity/v3/projects/c6d1b138e19c4fba961ecb54054cd127/users/16afdc0fcd9f47e787bf61fdf2e78ba0/roles/98ba668f78114223a37696e2980325b5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:38.652176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc457c1-25dc-4301-bb2a-9d40e74ebf8c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.652464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc457c1-25dc-4301-bb2a-9d40e74ebf8c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.652762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc457c1-25dc-4301-bb2a-9d40e74ebf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.693962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ffc457c1-25dc-4301-bb2a-9d40e74ebf8c None None] MFA Rules not processed for user `16afdc0fcd9f47e787bf61fdf2e78ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:38.695639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ffc457c1-25dc-4301-bb2a-9d40e74ebf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:38.702456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3333/6715] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:38 2022] POST /identity/v3/auth/tokens => generated 334 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 10:21:38.707529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f116-e984-44e9-a007-2a0bc474a06f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:38.708492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f116-e984-44e9-a007-2a0bc474a06f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:38.710958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f116-e984-44e9-a007-2a0bc474a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:38.736021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-62b5f116-e984-44e9-a007-2a0bc474a06f None None] MFA Rules not processed for user `16afdc0fcd9f47e787bf61fdf2e78ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:38.737523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62b5f116-e984-44e9-a007-2a0bc474a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:38.763193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:38.765629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:38.786289 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3383/6716] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:38.890158 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3334/6717] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:38 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 27 10:21:38.892000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:38.909037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7386c-53d9-4453-9dff-287c570d66a0 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:39.071732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:39.085792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:39.086822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:39.087274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:39.087598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:39.111001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:39.113401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:39.145420 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3384/6718] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Sep 27 10:21:38 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:39.149936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06e187-00f5-40c1-88eb-8be791b10334 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:39.150532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06e187-00f5-40c1-88eb-8be791b10334 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:39.151078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06e187-00f5-40c1-88eb-8be791b10334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:39.171862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7d06e187-00f5-40c1-88eb-8be791b10334 None None] MFA Rules not processed for user `16afdc0fcd9f47e787bf61fdf2e78ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:39.173943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7d06e187-00f5-40c1-88eb-8be791b10334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:39.356183 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3335/6719] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 27 10:21:39.360634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:39.374075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf23cd8-30cc-453b-8ebf-49a7421c337f tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:39.383603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:39.384271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:39.384557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:39.384861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/roles/98ba668f78114223a37696e2980325b5` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:39.388940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=98ba668f78114223a37696e2980325b5)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:39.391315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:39.503849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invalidating the token cache because role 98ba668f78114223a37696e2980325b5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:39.504279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.505860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.512094 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3385/6720] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:39 2022] DELETE /identity/v3/roles/98ba668f78114223a37696e2980325b5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:39.516808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06e187-00f5-40c1-88eb-8be791b10334 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:39.534480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06e187-00f5-40c1-88eb-8be791b10334 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:39.572452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:39.638930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:39.639597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:39.639884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:39.640157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects/c6d1b138e19c4fba961ecb54054cd127` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:39.642017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c6d1b138e19c4fba961ecb54054cd127)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:39.643303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:39.716299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d1b138e19c4fba961ecb54054cd127', 'request_id': 'req-38a20ad2-a869-4c59-b57d-fa9086a0330b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.728912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d1b138e19c4fba961ecb54054cd127', 'request_id': 'req-38a20ad2-a869-4c59-b57d-fa9086a0330b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.742760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d1b138e19c4fba961ecb54054cd127', 'request_id': 'req-38a20ad2-a869-4c59-b57d-fa9086a0330b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.763585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] The token cache is being invalidate because project c6d1b138e19c4fba961ecb54054cd127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:39.764056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.764857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.771141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:39.771740 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3336/6721] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:39 2022] DELETE /identity/v3/projects/c6d1b138e19c4fba961ecb54054cd127 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:39.773217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:39.790403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc166d1-4c22-4ba5-b981-9a6cd28dbcfe tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:39.859425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:39.860074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:39.860355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:39.860625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects/c6d1b138e19c4fba961ecb54054cd127` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:39.868448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c6d1b138e19c4fba961ecb54054cd127)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:39.870388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Could not find project: c6d1b138e19c4fba961ecb54054cd127.: keystone.exception.ProjectNotFound: Could not find project: c6d1b138e19c4fba961ecb54054cd127. Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 27 10:21:39.886297 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c6d1b138e19c4fba961ecb54054cd127. Sep 27 10:21:39.887700 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:21:39.888589 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3386/6722] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:39 2022] DELETE /identity/v3/projects/c6d1b138e19c4fba961ecb54054cd127 => generated 113 bytes in 118 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:21:39.894349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:39.895994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:39.921816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20ad2-a869-4c59-b57d-fa9086a0330b tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:39.927110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:39.927777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:39.928047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:39.928329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/projects/7f6c2ef9843d4f6396b05865c37d1e14` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:39.929581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7f6c2ef9843d4f6396b05865c37d1e14)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:39.931655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:39.991325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f6c2ef9843d4f6396b05865c37d1e14', 'request_id': 'req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:39.998685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f6c2ef9843d4f6396b05865c37d1e14', 'request_id': 'req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.013826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f6c2ef9843d4f6396b05865c37d1e14', 'request_id': 'req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.031066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] The token cache is being invalidate because project 7f6c2ef9843d4f6396b05865c37d1e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:40.031503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.032410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.039402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:40.040132 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3337/6723] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:39 2022] DELETE /identity/v3/projects/7f6c2ef9843d4f6396b05865c37d1e14 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:40.041754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:40.057491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b2cd7-01ec-4abd-b554-f11dfaa6f174 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:40.188878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:40.189579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:40.189855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:40.190128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/users/16afdc0fcd9f47e787bf61fdf2e78ba0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:40.191287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorizing `identity:delete_user(user_id=16afdc0fcd9f47e787bf61fdf2e78ba0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:40.193656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:40.244956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:40.254420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16afdc0fcd9f47e787bf61fdf2e78ba0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.256391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16afdc0fcd9f47e787bf61fdf2e78ba0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.268217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16afdc0fcd9f47e787bf61fdf2e78ba0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.279649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16afdc0fcd9f47e787bf61fdf2e78ba0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.280404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16afdc0fcd9f47e787bf61fdf2e78ba0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:40.300693 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3387/6724] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:40 2022] DELETE /identity/v3/users/16afdc0fcd9f47e787bf61fdf2e78ba0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:40.307467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:40.320478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca4aff-3a71-413b-86bc-8a0d7e15d6e7 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:40.390524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:40.438867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:40.443017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:40.443605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:40.444078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/6934953d209d4a10ba7952d956a4a39c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:40.452765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Could not find domain: 6934953d209d4a10ba7952d956a4a39c.: keystone.exception.DomainNotFound: Could not find domain: 6934953d209d4a10ba7952d956a4a39c. Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6934953d209d4a10ba7952d956a4a39c. Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:21:40.459231 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:21:40.460450 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 6934953d209d4a10ba7952d956a4a39c. Sep 27 10:21:40.461675 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:21:40.466816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3338/6725] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:40 2022] PATCH /identity/v3/domains/6934953d209d4a10ba7952d956a4a39c => generated 112 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:21:40.471089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:40.472790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:40.504408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b9ecd-8e9c-477d-9c01-7a893dffef20 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:40.513606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:40.514355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:40.516125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:40.516125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/95271a3806a543bfa206fbba960101c7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:40.550927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Could not find domain: 95271a3806a543bfa206fbba960101c7.: keystone.exception.DomainNotFound: Could not find domain: 95271a3806a543bfa206fbba960101c7. Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 95271a3806a543bfa206fbba960101c7. Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:21:40.583470 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:21:40.589613 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 95271a3806a543bfa206fbba960101c7. Sep 27 10:21:40.591035 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:21:40.597168 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3388/6726] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:40 2022] PATCH /identity/v3/domains/95271a3806a543bfa206fbba960101c7 => generated 112 bytes in 127 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:21:40.606290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:40.622424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07450b1-ffc0-406f-8737-87bb0640e8b0 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:40.642700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:40.643487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:40.643974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:40.644353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1d04045207d343c3ba8b3a3e06799e73` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:40.653719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Could not find domain: 1d04045207d343c3ba8b3a3e06799e73.: keystone.exception.DomainNotFound: Could not find domain: 1d04045207d343c3ba8b3a3e06799e73. Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 1d04045207d343c3ba8b3a3e06799e73. Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:21:40.659673 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:21:40.661456 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application with Lock( Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 1d04045207d343c3ba8b3a3e06799e73. Sep 27 10:21:40.662790 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:21:40.669449 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3339/6727] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:40 2022] PATCH /identity/v3/domains/1d04045207d343c3ba8b3a3e06799e73 => generated 112 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:21:40.670407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:40.687969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c017a-a3d0-4bae-baf2-365ae78e5f18 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:40.697705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4311771-5da6-4a60-976b-33a3bffd46fb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a982e7f9eaf4636a491718641ee0294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7acb7054d8945d28e96ed6aca92f2ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:40.698414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4311771-5da6-4a60-976b-33a3bffd46fb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:40.698817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4311771-5da6-4a60-976b-33a3bffd46fb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:40.699358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4311771-5da6-4a60-976b-33a3bffd46fb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] PATH_INFO: `/v3/domains/1dd998911e474be4b6a4fd49423c9115` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:40.745801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4311771-5da6-4a60-976b-33a3bffd46fb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b4311771-5da6-4a60-976b-33a3bffd46fb tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Could not find domain: 1dd998911e474be4b6a4fd49423c9115.: keystone.exception.DomainNotFound: Could not find domain: 1dd998911e474be4b6a4fd49423c9115. Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 1dd998911e474be4b6a4fd49423c9115. Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:21:40.761724 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:21:40.763424 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 1dd998911e474be4b6a4fd49423c9115. Sep 27 10:21:40.764984 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:21:40.772711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:40.774793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:40.783915 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3389/6728] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:21:40 2022] PATCH /identity/v3/domains/1dd998911e474be4b6a4fd49423c9115 => generated 112 bytes in 113 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:21:40.925739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2e4bb-73ea-4865-9e77-39a55776d618 tempest-TokensV3TestJSON-57681421 tempest-TokensV3TestJSON-57681421-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:40.933758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:40.934954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:40.935441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:40.935870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] PATH_INFO: `/v3/users/27bfd1734cd9400586d3da90d5cb9fed` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:40.937480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27bfd1734cd9400586d3da90d5cb9fed)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:40.940028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:41.032255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27bfd1734cd9400586d3da90d5cb9fed'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:41.033536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27bfd1734cd9400586d3da90d5cb9fed'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:41.059555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27bfd1734cd9400586d3da90d5cb9fed'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:41.079928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27bfd1734cd9400586d3da90d5cb9fed'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:41.144378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27bfd1734cd9400586d3da90d5cb9fed'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:41.163165 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3340/6729] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:40 2022] DELETE /identity/v3/users/27bfd1734cd9400586d3da90d5cb9fed => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:41.167781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:41.168015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:41.168359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:41.205128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:41.206715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:41.310087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3390/6730] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:41.328993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:41.331070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:41.388007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:41.404755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e09d-4e86-4f62-b443-a360127b30b8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:41.411049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:41.411978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:41.412459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:41.413245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:41.425456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:41.427315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:41.439177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3341/6731] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:42.257515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:42.259162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:42.279676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ed1b8-f054-4107-8f0c-84c3e8cf42f0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:42.286974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:42.287736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:42.288070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:42.288405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] PATH_INFO: `/v3/projects/ef5ecf11300a46d28205d24d59fadbc0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:42.289719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef5ecf11300a46d28205d24d59fadbc0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:42.291810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:42.387561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef5ecf11300a46d28205d24d59fadbc0', 'request_id': 'req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.401045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef5ecf11300a46d28205d24d59fadbc0', 'request_id': 'req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.408412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef5ecf11300a46d28205d24d59fadbc0', 'request_id': 'req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.421786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] The token cache is being invalidate because project ef5ecf11300a46d28205d24d59fadbc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:42.422364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.423134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.425244 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3391/6732] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:42 2022] DELETE /identity/v3/projects/ef5ecf11300a46d28205d24d59fadbc0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:42.429536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:42.440453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe8e8d-74aa-4e3c-9aec-dab3d6fd7f00 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:42.514697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:42.516131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:42.516715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:42.518815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] PATH_INFO: `/v3/users/2a982e7f9eaf4636a491718641ee0294` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:42.521142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a982e7f9eaf4636a491718641ee0294)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:42.525183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:42.585182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a982e7f9eaf4636a491718641ee0294'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.588791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a982e7f9eaf4636a491718641ee0294'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.605512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a982e7f9eaf4636a491718641ee0294'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.624237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a982e7f9eaf4636a491718641ee0294'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.659865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a982e7f9eaf4636a491718641ee0294'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:42.669807 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3342/6733] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:42 2022] DELETE /identity/v3/users/2a982e7f9eaf4636a491718641ee0294 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:43.409857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:43.412228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:43.435068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb7c7-6157-4fc9-82d6-5908dcef2bd7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:43.476930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:43.547507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:43.548644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:43.549211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:43.549670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] PATH_INFO: `/v3/projects/a7acb7054d8945d28e96ed6aca92f2ee` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:43.551991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7acb7054d8945d28e96ed6aca92f2ee)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:43.554975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:43.648906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7acb7054d8945d28e96ed6aca92f2ee', 'request_id': 'req-138318b7-8660-44d1-9490-688d3058640e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.660243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7acb7054d8945d28e96ed6aca92f2ee', 'request_id': 'req-138318b7-8660-44d1-9490-688d3058640e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.669650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7acb7054d8945d28e96ed6aca92f2ee', 'request_id': 'req-138318b7-8660-44d1-9490-688d3058640e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.687285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] The token cache is being invalidate because project a7acb7054d8945d28e96ed6aca92f2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:43.687577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.689127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-138318b7-8660-44d1-9490-688d3058640e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.693125 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3392/6734] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:43 2022] DELETE /identity/v3/projects/a7acb7054d8945d28e96ed6aca92f2ee => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:43.697433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:43.698950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:43.720774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046901f4-fa63-4bac-aa96-1b87c095a77a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:43.817314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:43.818500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:43.819150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:43.819608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] PATH_INFO: `/v3/users/770ce92d416741eb83285aca9d262adc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:43.822690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770ce92d416741eb83285aca9d262adc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:43.825773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:43.833937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:43.923186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770ce92d416741eb83285aca9d262adc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.924115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770ce92d416741eb83285aca9d262adc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.936525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770ce92d416741eb83285aca9d262adc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:43.955693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770ce92d416741eb83285aca9d262adc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:44.000925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770ce92d416741eb83285aca9d262adc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:44.017021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3343/6735] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:43 2022] DELETE /identity/v3/users/770ce92d416741eb83285aca9d262adc => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:44.081858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:44.082140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:44.082614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:44.144141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:44.145490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:44.246902 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3393/6736] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:44.262173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:44.264494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:44.394831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:44.396605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:44.423095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abab1e-4d49-4b8f-ac96-2fff5d3e05d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:44.441207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4434f4-865b-4914-af0f-ff0586b9f554 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:44.441926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4434f4-865b-4914-af0f-ff0586b9f554 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:44.443092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4434f4-865b-4914-af0f-ff0586b9f554 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:44.443437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4434f4-865b-4914-af0f-ff0586b9f554 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:44.463641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4434f4-865b-4914-af0f-ff0586b9f554 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:44.466372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4434f4-865b-4914-af0f-ff0586b9f554 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:44.479671 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3344/6737] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:21:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:21:44.519129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:44.528173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43ed69-627e-4216-baaf-3325fb5f6efa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:44.541065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:44.541728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:44.542407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:44.542534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] PATH_INFO: `/v3/projects/d2553392831841a88631ca974179bc68` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:44.544009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2553392831841a88631ca974179bc68)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:44.546682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:44.638110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2553392831841a88631ca974179bc68', 'request_id': 'req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:44.661214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2553392831841a88631ca974179bc68', 'request_id': 'req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:44.673115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2553392831841a88631ca974179bc68', 'request_id': 'req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:44.729628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] The token cache is being invalidate because project d2553392831841a88631ca974179bc68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:44.729930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:44.731320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99f4d45f-9bc0-4ef1-bbc4-bf73d23afb0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:44.737810 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3394/6738] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:44 2022] DELETE /identity/v3/projects/d2553392831841a88631ca974179bc68 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:44.744808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceff0a2-1b4c-4665-bc47-6e3268d1c6b4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:44.745653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceff0a2-1b4c-4665-bc47-6e3268d1c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:44.745653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceff0a2-1b4c-4665-bc47-6e3268d1c6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:44.794253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4ceff0a2-1b4c-4665-bc47-6e3268d1c6b4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:44.796966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ceff0a2-1b4c-4665-bc47-6e3268d1c6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:44.976621 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3345/6739] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:21:44.986112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a2a70-c142-4ca9-b0fa-0340bff6fb88 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:44.986743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a2a70-c142-4ca9-b0fa-0340bff6fb88 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:44.987197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a2a70-c142-4ca9-b0fa-0340bff6fb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:45.082119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ff8a2a70-c142-4ca9-b0fa-0340bff6fb88 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:45.097833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ff8a2a70-c142-4ca9-b0fa-0340bff6fb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:45.132621 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3395/6740] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:45.137214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceff0a2-1b4c-4665-bc47-6e3268d1c6b4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:45.155473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceff0a2-1b4c-4665-bc47-6e3268d1c6b4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:45.183599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:45.184354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:45.184634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:45.185218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:45.185891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:45.186404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:45.207394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:45.256885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a2a70-c142-4ca9-b0fa-0340bff6fb88 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:45.260864 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3346/6741] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:45.271676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a2a70-c142-4ca9-b0fa-0340bff6fb88 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:45.281884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:45.283651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:45.284439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:45.285120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:45.286200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:45.290415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:45.368317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3396/6742] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:45 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:45.370540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:45.376461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:45.386521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8253f7e-4016-4053-a84b-83e91f974427 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:45.566373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:45.567239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:45.567668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:45.568392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:45.569299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:45.571945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:45.637542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3347/6743] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:45 2022] POST /identity/v3/users => generated 300 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:45.639137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:45.662133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a32c9-f938-4dda-b8db-c865c726d469 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:45.678699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:45.681677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:45.682300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:45.688705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:45.690031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:45.693877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:45.732122 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3397/6744] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:45.738334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:45.764288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcff2a-f699-47b9-b4a3-417e6f015e05 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:45.775682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:45.779893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:45.781741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:45.783620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] PATH_INFO: `/v3/projects/e213621918394256855bdef588bd7e00/users/5b2b4a8ef1944cac8ba658bbc14fdb05/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:45.868266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e213621918394256855bdef588bd7e00, user_id=5b2b4a8ef1944cac8ba658bbc14fdb05, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:45.871326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:45.898937 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3348/6745] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:45 2022] PUT /identity/v3/projects/e213621918394256855bdef588bd7e00/users/5b2b4a8ef1944cac8ba658bbc14fdb05/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:45.906039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:45.932895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25492a91-f191-4dc4-b60a-e6885a1be533 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:46.162229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:46.164941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:46.165525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:46.166126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:46.167151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:46.170539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:46.189865 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3398/6746] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:46.190582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:46.215349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0ce49-5b73-4f54-8cfa-3beea3563097 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:46.223195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361275e-8ccb-4135-8312-67558daab891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:46.224519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361275e-8ccb-4135-8312-67558daab891 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:46.224519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361275e-8ccb-4135-8312-67558daab891 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:46.224828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361275e-8ccb-4135-8312-67558daab891 admin admin] PATH_INFO: `/v3/projects/e213621918394256855bdef588bd7e00/users/5b2b4a8ef1944cac8ba658bbc14fdb05/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:46.228964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361275e-8ccb-4135-8312-67558daab891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e213621918394256855bdef588bd7e00, user_id=5b2b4a8ef1944cac8ba658bbc14fdb05, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:46.234768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:46.236767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:46.264589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361275e-8ccb-4135-8312-67558daab891 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:46.309926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c207b5-ad7d-4caa-a8f6-b68e1152a1ca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:46.315526 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3349/6747] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:46 2022] PUT /identity/v3/projects/e213621918394256855bdef588bd7e00/users/5b2b4a8ef1944cac8ba658bbc14fdb05/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:46.323642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72632e1-990a-423e-b2cf-345267c6efdb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:46.324064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72632e1-990a-423e-b2cf-345267c6efdb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:46.324638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72632e1-990a-423e-b2cf-345267c6efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:46.420154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a72632e1-990a-423e-b2cf-345267c6efdb None None] MFA Rules not processed for user `5b2b4a8ef1944cac8ba658bbc14fdb05`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:46.422043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a72632e1-990a-423e-b2cf-345267c6efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:46.479506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:46.524242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:46.525279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:46.526505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:46.542552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] PATH_INFO: `/v3/users/1be0da58001e4b71976eaf46bd246b2d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:46.563545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be0da58001e4b71976eaf46bd246b2d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:46.567718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:46.596834 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3350/6748] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:46.600888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72632e1-990a-423e-b2cf-345267c6efdb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:46.624765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72632e1-990a-423e-b2cf-345267c6efdb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:46.636100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:46.638429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:46.641331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:46.641796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:46.642627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:46.647401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:46.722217 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3351/6749] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:46 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:46.726294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:46.779358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:46.782527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be0da58001e4b71976eaf46bd246b2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:46.784342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be0da58001e4b71976eaf46bd246b2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:46.808420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be0da58001e4b71976eaf46bd246b2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:46.832753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be0da58001e4b71976eaf46bd246b2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:46.834730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bf93324a-72ea-4957-b818-be2fdb7956e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be0da58001e4b71976eaf46bd246b2d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:46.891798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d978f89-259a-4f14-8a8c-b7ae9cc0eb52 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:46.892758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3399/6750] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:46 2022] DELETE /identity/v3/users/1be0da58001e4b71976eaf46bd246b2d => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:46.897336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:46.897757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:46.898236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:46.923911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:46.925734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:47.028200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:47.028871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:47.029155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:47.029428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:47.030117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:47.032025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:47.081302 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3352/6751] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:46 2022] POST /identity/v3/users => generated 294 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:21:47.082584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:47.091312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:47.112429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ae574-c8f6-443d-95eb-fdf9fbe14076 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:47.125463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:47.126481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:47.126954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:47.127522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:47.128368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:47.128753 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3400/6752] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:47.130621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:47.139940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:47.144388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:47.163726 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3353/6753] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:21:47.166884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:47.183316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648c38c-882c-4f74-8f22-1022ef2c1b02 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:47.194351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:47.194951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:47.195201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:47.195518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:47.197470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:47.197704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:47.226347 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3354/6754] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:47.227619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:47.250728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7d768-255f-429f-9019-60ec16d8dce9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:47.259108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec65e77-58f1-4f2e-a345-d98c02e9b06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:47.259804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec65e77-58f1-4f2e-a345-d98c02e9b06a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:47.260082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec65e77-58f1-4f2e-a345-d98c02e9b06a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:47.260355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec65e77-58f1-4f2e-a345-d98c02e9b06a admin admin] PATH_INFO: `/v3/projects/1416d99c6be94733ba35d51f7f5f148e/users/1de832435b22457e9da471b2c264061b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:47.297648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec65e77-58f1-4f2e-a345-d98c02e9b06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1416d99c6be94733ba35d51f7f5f148e, user_id=1de832435b22457e9da471b2c264061b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:47.305562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec65e77-58f1-4f2e-a345-d98c02e9b06a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:47.353551 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3355/6755] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:47 2022] PUT /identity/v3/projects/1416d99c6be94733ba35d51f7f5f148e/users/1de832435b22457e9da471b2c264061b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:21:47.357097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:47.358917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:47.359918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:47.396637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583d084-e260-45a1-a4c6-37f9fcb82bb2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:47.443229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:47.451437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] MFA Rules not processed for user `1de832435b22457e9da471b2c264061b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:47.456163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:47.551129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:47.552808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:47.553227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:47.553760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:47.645977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:47.668044 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3356/6756] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:47.686981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:47.692645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:47.789775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:47.792108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:47.818214 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3401/6757] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:47.885993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97780-b11c-4d16-ac72-96fd846be1a2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:47.894177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de617e9-5e74-44ac-8097-40c04eddae98 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:47.894841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de617e9-5e74-44ac-8097-40c04eddae98 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:47.895115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de617e9-5e74-44ac-8097-40c04eddae98 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:47.895487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de617e9-5e74-44ac-8097-40c04eddae98 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:47.904686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7de617e9-5e74-44ac-8097-40c04eddae98 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:48.042822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de617e9-5e74-44ac-8097-40c04eddae98 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:48.046815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de617e9-5e74-44ac-8097-40c04eddae98 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:48.066474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3357/6758] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:48.720047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:48.737426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37e15d-395c-43c5-807c-293aca615766 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:48.757647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:48.759050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:48.759623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:48.760142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] PATH_INFO: `/v3/users/5b2b4a8ef1944cac8ba658bbc14fdb05` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:48.762133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2b4a8ef1944cac8ba658bbc14fdb05)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:48.767846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:48.931845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2b4a8ef1944cac8ba658bbc14fdb05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:48.933357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2b4a8ef1944cac8ba658bbc14fdb05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:48.947207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2b4a8ef1944cac8ba658bbc14fdb05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:48.959389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2b4a8ef1944cac8ba658bbc14fdb05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:48.960761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2b4a8ef1944cac8ba658bbc14fdb05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:49.036113 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3402/6759] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:48 2022] DELETE /identity/v3/users/5b2b4a8ef1944cac8ba658bbc14fdb05 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:49.040903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:49.043491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:49.044007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:49.105758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:49.107052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:49.324619 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3358/6760] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:49.350409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:49.352815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:49.508084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:49.537210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb649c-990d-4e96-b7a1-d1392e815c04 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:49.547080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:49.547728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:49.548044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:49.548451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:49.570451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:49.572441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:49.597875 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3403/6761] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:51.262986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:51.264468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:51.290010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb0fd-d0b3-489e-8449-8e3df1eaea1f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:51.313857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:51.314553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:51.315057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:51.315513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] PATH_INFO: `/v3/projects/140e3033364a498c9752cae22f3108a6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:51.325688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=140e3033364a498c9752cae22f3108a6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:51.327654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:51.543155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:51.571704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140e3033364a498c9752cae22f3108a6', 'request_id': 'req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:51.596769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140e3033364a498c9752cae22f3108a6', 'request_id': 'req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:51.622880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140e3033364a498c9752cae22f3108a6', 'request_id': 'req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:51.701716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] The token cache is being invalidate because project 140e3033364a498c9752cae22f3108a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:51.701988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:51.702681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:51.709599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3359/6762] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:51 2022] DELETE /identity/v3/projects/140e3033364a498c9752cae22f3108a6 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:51.713409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:51.731473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c15a1-f5a4-401a-83b6-e902f74da67b service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:51.918667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:51.919351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:51.919641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:51.919915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] PATH_INFO: `/v3/users/7229e538689e42ed9b1f22e6464979ed` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:51.922246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7229e538689e42ed9b1f22e6464979ed)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:51.924835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:52.133698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7229e538689e42ed9b1f22e6464979ed'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.134600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7229e538689e42ed9b1f22e6464979ed'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.162487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7229e538689e42ed9b1f22e6464979ed'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.191598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7229e538689e42ed9b1f22e6464979ed'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.196507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7229e538689e42ed9b1f22e6464979ed'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.285190 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3404/6763] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:51 2022] DELETE /identity/v3/users/7229e538689e42ed9b1f22e6464979ed => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:52.513246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:52.515520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:52.558636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3c0b6-05e5-4e9d-bbe8-043bd8f7d77c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:52.752963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:52.754116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:52.756821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:52.757338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] PATH_INFO: `/v3/projects/e213621918394256855bdef588bd7e00` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:52.758921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e213621918394256855bdef588bd7e00)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:52.768949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:52.887894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e213621918394256855bdef588bd7e00', 'request_id': 'req-30e11597-5e99-4a97-9617-0fd002848f9e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.905195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e213621918394256855bdef588bd7e00', 'request_id': 'req-30e11597-5e99-4a97-9617-0fd002848f9e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.938587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e213621918394256855bdef588bd7e00', 'request_id': 'req-30e11597-5e99-4a97-9617-0fd002848f9e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.958406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] The token cache is being invalidate because project e213621918394256855bdef588bd7e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:52.958775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.962909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:52.966656 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3360/6764] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:52 2022] DELETE /identity/v3/projects/e213621918394256855bdef588bd7e00 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:52.976899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:52.980968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:53.021186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19393f64-d57a-46dd-bb04-3856d91bde99 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:53.209502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:53.210721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:53.210721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:53.210721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] PATH_INFO: `/v3/users/1de832435b22457e9da471b2c264061b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:53.213334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de832435b22457e9da471b2c264061b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:53.221603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:53.237604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:53.335536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de832435b22457e9da471b2c264061b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:53.337539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de832435b22457e9da471b2c264061b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:53.353606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de832435b22457e9da471b2c264061b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:53.380035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de832435b22457e9da471b2c264061b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:53.381952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3dd43338-3bde-4414-8a05-d2d15e870755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de832435b22457e9da471b2c264061b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:53.421037 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3405/6765] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:21:52 2022] DELETE /identity/v3/users/1de832435b22457e9da471b2c264061b => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:54.609435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:54.614762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:54.650048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e11597-5e99-4a97-9617-0fd002848f9e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:54.896554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:54.900135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:54.902172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:54.902970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] PATH_INFO: `/v3/projects/1416d99c6be94733ba35d51f7f5f148e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:54.908020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1416d99c6be94733ba35d51f7f5f148e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:54.921350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:54.925309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:55.109554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1416d99c6be94733ba35d51f7f5f148e', 'request_id': 'req-4421ad86-c095-45c2-8049-49ff7581ca49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.138900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1416d99c6be94733ba35d51f7f5f148e', 'request_id': 'req-4421ad86-c095-45c2-8049-49ff7581ca49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.169550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1416d99c6be94733ba35d51f7f5f148e', 'request_id': 'req-4421ad86-c095-45c2-8049-49ff7581ca49'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.196938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] The token cache is being invalidate because project 1416d99c6be94733ba35d51f7f5f148e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:55.198947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.202710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.205809 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3361/6766] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:54 2022] DELETE /identity/v3/projects/1416d99c6be94733ba35d51f7f5f148e => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:55.216059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c90797-2ce2-4b36-80d6-f5bc8da66b66 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:55.217443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c90797-2ce2-4b36-80d6-f5bc8da66b66 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:55.217876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c90797-2ce2-4b36-80d6-f5bc8da66b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:55.280735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-48c90797-2ce2-4b36-80d6-f5bc8da66b66 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:55.289446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-48c90797-2ce2-4b36-80d6-f5bc8da66b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:55.388558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:55.390693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:55.517124 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3406/6767] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:55.518650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50624f-4408-4a01-abc9-d37ae3ed3a4d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:55.524116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50624f-4408-4a01-abc9-d37ae3ed3a4d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:55.524394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50624f-4408-4a01-abc9-d37ae3ed3a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:55.603366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2b50624f-4408-4a01-abc9-d37ae3ed3a4d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:55.608415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2b50624f-4408-4a01-abc9-d37ae3ed3a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:55.636945 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3407/6768] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:55.640543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50624f-4408-4a01-abc9-d37ae3ed3a4d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:55.647102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421ad86-c095-45c2-8049-49ff7581ca49 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:55.661740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:55.663191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:55.664565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:55.664892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] PATH_INFO: `/v3/projects/ad7ae413ffc64977ab4ea87bc0362d76` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:55.666595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50624f-4408-4a01-abc9-d37ae3ed3a4d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:55.667678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad7ae413ffc64977ab4ea87bc0362d76)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:55.672797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:55.679931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:55.682924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:55.683721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:55.684385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:55.686492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:55.687707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:55.691171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:55.743028 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3408/6769] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:55.747455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:55.778690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638357-8388-4c39-b55f-1901e0458354 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:55.799619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:55.803646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:55.804882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:55.805614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:55.809305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:55.817671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:55.839610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad7ae413ffc64977ab4ea87bc0362d76', 'request_id': 'req-85e3f33f-d050-4980-9fe7-ba489b2b30fb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.862879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad7ae413ffc64977ab4ea87bc0362d76', 'request_id': 'req-85e3f33f-d050-4980-9fe7-ba489b2b30fb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.884136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad7ae413ffc64977ab4ea87bc0362d76', 'request_id': 'req-85e3f33f-d050-4980-9fe7-ba489b2b30fb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.900496 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3409/6770] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:55 2022] POST /identity/v3/projects => generated 391 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:55.904723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:55.927777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] The token cache is being invalidate because project ad7ae413ffc64977ab4ea87bc0362d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:21:55.928170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.929711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-85e3f33f-d050-4980-9fe7-ba489b2b30fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:21:55.937170 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3362/6771] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:21:55 2022] DELETE /identity/v3/projects/ad7ae413ffc64977ab4ea87bc0362d76 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:55.939547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c23f5-1d37-4abd-be4d-a3d6277c136f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:55.939933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c23f5-1d37-4abd-be4d-a3d6277c136f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:55.940170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c23f5-1d37-4abd-be4d-a3d6277c136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.006438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-977c23f5-1d37-4abd-be4d-a3d6277c136f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:56.009381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-977c23f5-1d37-4abd-be4d-a3d6277c136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:56.060458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:56.104516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:56.119102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cce85-0d08-4c5f-8cb0-16555bf69c65 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:56.126569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:56.128281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.131151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.132873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.134920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:56.140040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:56.219890 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3410/6772] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:55 2022] POST /identity/v3/users => generated 296 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:56.221395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:56.224797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:56.265510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9209-55c0-4549-ae65-043266b7a5ce admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:56.281926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:56.282828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3363/6773] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:21:56.284758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.286377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.286775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.287456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c625-6298-440f-9396-a015a23ace56 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.287748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c625-6298-440f-9396-a015a23ace56 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.288028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c625-6298-440f-9396-a015a23ace56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.290934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:56.297428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:56.327120 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3411/6774] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:21:56.334169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:56.340416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8216c625-6298-440f-9396-a015a23ace56 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:56.341756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8216c625-6298-440f-9396-a015a23ace56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:56.352468 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3364/6775] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:56.358500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216c625-6298-440f-9396-a015a23ace56 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:56.365742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a054e2-1f66-4911-a810-f17c8e29c2e6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:56.380080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216c625-6298-440f-9396-a015a23ace56 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:56.380361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:56.392688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.392688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.392688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.392688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:56.393195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:56.393195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.393195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.393195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.393563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:21:56.393563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:56.393737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:56.393831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:56.420828 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3412/6776] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:56.424560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:56.427541 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3365/6777] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:21:56.437237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:56.451164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1421c7-8cc8-480d-b925-d9ab5527b4dd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:56.453677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1712729-8161-4e80-811d-8cabc32171ca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:56.465572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:56.466590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.467626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c995b09-b6bf-4525-a549-b65f7e60725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:56.468535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.469033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.469799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c995b09-b6bf-4525-a549-b65f7e60725d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.470672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:56.471053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c995b09-b6bf-4525-a549-b65f7e60725d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.473060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c995b09-b6bf-4525-a549-b65f7e60725d admin admin] PATH_INFO: `/v3/projects/e99371cd2911482b96ba52a27b874a5c/users/3b2756fc7066452fb97e7a45cb0522ec/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.478703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:56.542531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c995b09-b6bf-4525-a549-b65f7e60725d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e99371cd2911482b96ba52a27b874a5c, user_id=3b2756fc7066452fb97e7a45cb0522ec, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:56.553606 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3366/6778] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:56 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:56.555738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:56.556528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c995b09-b6bf-4525-a549-b65f7e60725d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:56.600820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3413/6779] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:56 2022] PUT /identity/v3/projects/e99371cd2911482b96ba52a27b874a5c/users/3b2756fc7066452fb97e7a45cb0522ec/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:56.611918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6372-10a2-4e13-be38-23890d15a351 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.613040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6372-10a2-4e13-be38-23890d15a351 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.613329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6372-10a2-4e13-be38-23890d15a351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.751949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-973f6372-10a2-4e13-be38-23890d15a351 None None] MFA Rules not processed for user `3b2756fc7066452fb97e7a45cb0522ec`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:56.753240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-973f6372-10a2-4e13-be38-23890d15a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:56.758832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5fb9e-ea7c-4dd8-bf23-a4d485df6144 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:56.925660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:56.926223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:56.926509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:56.926905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:56.927712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:56.929905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:56.985105 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3367/6780] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:56 2022] POST /identity/v3/users => generated 289 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:56.986403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.000459 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3414/6781] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:57.003409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f6372-10a2-4e13-be38-23890d15a351 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.013707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebff3e-8637-47ab-8d7c-58897b7a987f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.031380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.032324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.032738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.033017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.035968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.040330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:57.074512 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3368/6782] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:21:57.075765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.078670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f6372-10a2-4e13-be38-23890d15a351 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.104961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841546-bb38-4530-ba93-3dd19031535b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.115912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.118245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.118743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.119532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.119876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.120160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.121110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.126385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:57.131301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.131900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:57.133318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.138935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.146110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:57.162881 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3369/6783] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:57.163968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.181487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b7a53-309d-430f-b256-d5c475533efb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.194967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a29ca-4a7c-4ef0-afc1-5b1757ec0319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.195760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a29ca-4a7c-4ef0-afc1-5b1757ec0319 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.200937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a29ca-4a7c-4ef0-afc1-5b1757ec0319 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.201570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a29ca-4a7c-4ef0-afc1-5b1757ec0319 admin admin] PATH_INFO: `/v3/projects/b8fae88fc8f8420eb7f5b18520f4a822/users/0d6b172b40364a4aba6f55e0cb24ea4e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.278774 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3415/6784] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:21:57 2022] POST /identity/v3/projects => generated 391 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:57.279692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a29ca-4a7c-4ef0-afc1-5b1757ec0319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8fae88fc8f8420eb7f5b18520f4a822, user_id=0d6b172b40364a4aba6f55e0cb24ea4e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.282554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a29ca-4a7c-4ef0-afc1-5b1757ec0319 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:57.286357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.329033 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3370/6785] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:57 2022] PUT /identity/v3/projects/b8fae88fc8f8420eb7f5b18520f4a822/users/0d6b172b40364a4aba6f55e0cb24ea4e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:57.345816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.347232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.347626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.420487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] MFA Rules not processed for user `0d6b172b40364a4aba6f55e0cb24ea4e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:57.421931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:57.449142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:57.497960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bf000-5b06-4929-a066-1fcc428381d9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.679296 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3371/6786] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:57.693106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.695511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:57.717263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.717891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.718166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.718436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.719070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.721031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:57.775234 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3416/6787] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:21:57 2022] POST /identity/v3/users => generated 302 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:21:57.778430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.801005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d114-027c-47cd-807e-f55075b2248a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.807052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.808163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.808619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.809139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.833779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.835261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:57.836381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be69cce-751d-48f2-998a-6c0d81f713c3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.859551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.860596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.861265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.862178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.863039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.863754 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3372/6788] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:21:57.881287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:57.904540 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3417/6789] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:21:57.909477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:57.931238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98aa3d-ba11-426d-8458-d52bc2a969b6 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:57.944285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:57.945100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:57.945283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:57.945564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] PATH_INFO: `/v3/projects/7a427c3b011b4febb48ade20231552a7/users/d182880b4c1646b2b2c1d54e0d620267/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:57.982172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a427c3b011b4febb48ade20231552a7, user_id=d182880b4c1646b2b2c1d54e0d620267, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:57.984927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:58.017677 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3373/6790] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:57 2022] PUT /identity/v3/projects/7a427c3b011b4febb48ade20231552a7/users/d182880b4c1646b2b2c1d54e0d620267/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:58.019299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:58.033449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ea9f4-461f-4d4c-8f30-8585259891cc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:58.167771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b5e51-68a9-40be-ae1b-843a24bed7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:58.168757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b5e51-68a9-40be-ae1b-843a24bed7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:58.169174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b5e51-68a9-40be-ae1b-843a24bed7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:58.169610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b5e51-68a9-40be-ae1b-843a24bed7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:58.170717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b5e51-68a9-40be-ae1b-843a24bed7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:58.172681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b5e51-68a9-40be-ae1b-843a24bed7b7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:58.186850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3418/6791] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:21:58.191722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:58.212380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a593dd2-5a6b-49c7-bc24-e6be47752ad3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:58.218464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:58.219422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:58.219842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:58.220276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] PATH_INFO: `/v3/projects/7a427c3b011b4febb48ade20231552a7/users/d182880b4c1646b2b2c1d54e0d620267/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:58.222880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a427c3b011b4febb48ade20231552a7, user_id=d182880b4c1646b2b2c1d54e0d620267, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:58.226593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:58.250735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:58.251256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:58.251740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:58.254897 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3374/6792] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:21:58 2022] PUT /identity/v3/projects/7a427c3b011b4febb48ade20231552a7/users/d182880b4c1646b2b2c1d54e0d620267/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:21:58.290901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] MFA Rules not processed for user `d182880b4c1646b2b2c1d54e0d620267`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:21:58.293209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:58.467521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:58.480148 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3419/6793] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:21:58.525728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:58.527920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:58.620566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:58.690278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56cc4e-c1f1-48b8-95ce-536d72096d77 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:58.699563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:58.702061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:58.702462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:58.702753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:58.857160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:58.859037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:58.882696 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3375/6794] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:21:59.602321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:21:59.612155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:59.852100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c0f7-16ee-47e9-85fb-19e02ebfeb2d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:21:59.859512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:21:59.860309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:21:59.860774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:21:59.861175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:21:59.863311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:21:59.891772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:21:59.894425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:21:59.919112 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3420/6795] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:21:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:03.905837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:03.907471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:03.940046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d9b95-1601-49bc-9dc3-515ff9309c85 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:03.944973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c32ad-8c6f-4578-be90-3233124ad8b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:03.946318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c32ad-8c6f-4578-be90-3233124ad8b1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:03.947100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c32ad-8c6f-4578-be90-3233124ad8b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:03.947555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c32ad-8c6f-4578-be90-3233124ad8b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:03.964473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c32ad-8c6f-4578-be90-3233124ad8b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:03.967047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c32ad-8c6f-4578-be90-3233124ad8b1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:03.985113 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3376/6796] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:07.322796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:07.429265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f1697-6029-41f9-878f-97baf4ccbf38 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:07.434271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:07.434921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:07.435195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:07.435460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] PATH_INFO: `/v3/users/0d6b172b40364a4aba6f55e0cb24ea4e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:07.436877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6b172b40364a4aba6f55e0cb24ea4e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:07.438947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:07.509842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6b172b40364a4aba6f55e0cb24ea4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:07.510665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6b172b40364a4aba6f55e0cb24ea4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:07.523174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6b172b40364a4aba6f55e0cb24ea4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:07.537047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6b172b40364a4aba6f55e0cb24ea4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:07.539043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6b172b40364a4aba6f55e0cb24ea4e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:07.566997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:07.577885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf16ab-488f-440a-86dd-854bdea5fe2e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:07.578945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf16ab-488f-440a-86dd-854bdea5fe2e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:07.579468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf16ab-488f-440a-86dd-854bdea5fe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:07.589314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3421/6797] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:07 2022] DELETE /identity/v3/users/0d6b172b40364a4aba6f55e0cb24ea4e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:07.625674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fbcf16ab-488f-440a-86dd-854bdea5fe2e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:07.625674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fbcf16ab-488f-440a-86dd-854bdea5fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:07.748259 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3377/6798] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:07.829436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:07.832251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:07.978857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b91a30-b9b3-4107-8f63-37f715ec12f9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:07.984973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:07.985713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:07.986079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:07.986427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:08.003362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:08.006011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:08.019326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3422/6799] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:08.526451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:08.526729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:08.527236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:08.562944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:08.564631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:08.672146 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3378/6800] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:08.697573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:08.699996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:08.829604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027da904-6ce1-48c2-9ef8-71933b6334fb service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:08.838884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:08.839594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:08.839963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:08.840468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:08.854973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:08.858481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:08.880402 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3423/6801] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:22:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:09.112528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:09.114361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:09.141708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b862a5-4cb8-4fe8-9b29-e77d56ed7f1f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:09.158225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:09.158925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:09.159151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:09.159595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] PATH_INFO: `/v3/users/3b2756fc7066452fb97e7a45cb0522ec` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:09.165643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2756fc7066452fb97e7a45cb0522ec)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:09.173465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:09.230366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:09.356422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2756fc7066452fb97e7a45cb0522ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.357243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2756fc7066452fb97e7a45cb0522ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.368422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:09.371017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:09.388468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2756fc7066452fb97e7a45cb0522ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.398393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdde5a-0f44-4459-8004-fe1e4ef5349f service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:09.411977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:09.413058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2756fc7066452fb97e7a45cb0522ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.414977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:09.415345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:09.415720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] PATH_INFO: `/v3/projects/b8fae88fc8f8420eb7f5b18520f4a822` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:09.420256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8fae88fc8f8420eb7f5b18520f4a822)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:09.426241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:09.438416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-835b5c5b-b226-4dbf-87a3-3fd10445530d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2756fc7066452fb97e7a45cb0522ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.455393 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3379/6802] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:09 2022] DELETE /identity/v3/users/3b2756fc7066452fb97e7a45cb0522ec => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:09.461662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:09.461979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:09.465225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:09.513439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:09.514816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:09.563828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8fae88fc8f8420eb7f5b18520f4a822', 'request_id': 'req-0b53acb0-6a2a-4c72-a3c9-740fb3648676'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.589370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8fae88fc8f8420eb7f5b18520f4a822', 'request_id': 'req-0b53acb0-6a2a-4c72-a3c9-740fb3648676'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.598795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8fae88fc8f8420eb7f5b18520f4a822', 'request_id': 'req-0b53acb0-6a2a-4c72-a3c9-740fb3648676'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.613723 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3380/6803] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:09.618139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] The token cache is being invalidate because project b8fae88fc8f8420eb7f5b18520f4a822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:09.618612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.619656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0b53acb0-6a2a-4c72-a3c9-740fb3648676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:09.624735 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3424/6804] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:09 2022] DELETE /identity/v3/projects/b8fae88fc8f8420eb7f5b18520f4a822 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:09.631940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:09.635429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:09.641008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a672876-80c0-436a-a4d2-a5d8a4aba2ee None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:09.641937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a672876-80c0-436a-a4d2-a5d8a4aba2ee None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:09.642542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a672876-80c0-436a-a4d2-a5d8a4aba2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:09.684369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9a672876-80c0-436a-a4d2-a5d8a4aba2ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:09.685634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9a672876-80c0-436a-a4d2-a5d8a4aba2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:09.698177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:09.781273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094c439-720a-4fa7-ad29-bc6e38f64bee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:09.813136 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3425/6805] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:09.822615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed2745-bdee-4c4b-ba58-a5840d56768f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:09.823265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed2745-bdee-4c4b-ba58-a5840d56768f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:09.823815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed2745-bdee-4c4b-ba58-a5840d56768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:09.877016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-88ed2745-bdee-4c4b-ba58-a5840d56768f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:09.879105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-88ed2745-bdee-4c4b-ba58-a5840d56768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:09.892246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:09.904122 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3426/6806] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:09.904460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:09.916214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:09.916891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:09.917338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:09.917616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed2745-bdee-4c4b-ba58-a5840d56768f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:09.920223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:09.938275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:09.940408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:09.941133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed2745-bdee-4c4b-ba58-a5840d56768f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:09.949611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:09.950540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:09.950823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:09.951096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:09.951760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:22:09.953015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:09.953716 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3381/6807] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:09.958549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:09.981579 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3427/6808] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:22:09.986205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:09.997163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ea095-18d0-4e92-96d3-92b0ed3a760c service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:10.005022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:10.005447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:10.005886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:10.006820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:10.007885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:10.011914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:10.054849 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3382/6809] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:09 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:10.056096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:10.075926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7401d58-352f-4898-8daf-9e25e4bb8c12 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:10.157387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:10.158223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:10.158602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:10.158997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:10.159879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:10.163179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:10.198449 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3428/6810] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:10 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:10.202392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:10.219035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef4ce0-77ef-4e6e-996a-b2d043f89bcd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:10.230081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:10.232007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:10.232934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:10.233379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:10.235665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:10.238222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:10.254357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3383/6811] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:10.258579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:10.270251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1bdb-2c13-4e7e-be63-4fc8f28f8b55 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:10.275749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9c053-92ff-49c5-a923-6b7d48eed6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:10.276392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9c053-92ff-49c5-a923-6b7d48eed6df admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:10.276731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9c053-92ff-49c5-a923-6b7d48eed6df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:10.277012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9c053-92ff-49c5-a923-6b7d48eed6df admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:10.277699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9c053-92ff-49c5-a923-6b7d48eed6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:10.279373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9c053-92ff-49c5-a923-6b7d48eed6df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:10.298592 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3429/6812] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:10.300178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:10.316056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4dcb9-18f8-487e-abbc-52e6f795d5ca admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:10.322882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:10.323642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:10.324152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:10.324443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] PATH_INFO: `/v3/projects/87e1be172b4845fe9ad5333d323fff8b/users/f2cc166dabae4adc8f28b755de18f684/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:10.348140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e1be172b4845fe9ad5333d323fff8b, user_id=f2cc166dabae4adc8f28b755de18f684, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:10.352735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:10.387161 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3384/6813] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:10 2022] PUT /identity/v3/projects/87e1be172b4845fe9ad5333d323fff8b/users/f2cc166dabae4adc8f28b755de18f684/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:22:10.400788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:10.401364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:10.401784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:10.437937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] MFA Rules not processed for user `f2cc166dabae4adc8f28b755de18f684`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:10.439976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:10.607909 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3430/6814] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:10.625483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:10.628012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:10.699837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb749-7a48-46e1-bbdf-518292f3946d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:10.704973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:10.705724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:10.706005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:10.707014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:10.716433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:10.718007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:10.728362 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3385/6815] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:22:10.947841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:10.949482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:10.956861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:11.058176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02a905-d07a-40f8-924a-170ec4a752db None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:11.064759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:11.065460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:11.065770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:11.066042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] PATH_INFO: `/v3/projects/e99371cd2911482b96ba52a27b874a5c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:11.068228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e99371cd2911482b96ba52a27b874a5c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:11.071084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:11.162804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e99371cd2911482b96ba52a27b874a5c', 'request_id': 'req-6673140a-ec2f-418f-9e21-e42445e902c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.179315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e99371cd2911482b96ba52a27b874a5c', 'request_id': 'req-6673140a-ec2f-418f-9e21-e42445e902c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.188975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e99371cd2911482b96ba52a27b874a5c', 'request_id': 'req-6673140a-ec2f-418f-9e21-e42445e902c3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.212897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] The token cache is being invalidate because project e99371cd2911482b96ba52a27b874a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:11.213202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.214044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.215947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3431/6816] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:10 2022] DELETE /identity/v3/projects/e99371cd2911482b96ba52a27b874a5c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:11.223030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:11.239604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f0b49-da7e-46f2-a374-dfd6b8911d73 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:11.349070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:11.349809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:11.350091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:11.350457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] PATH_INFO: `/v3/users/d182880b4c1646b2b2c1d54e0d620267` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:11.351960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d182880b4c1646b2b2c1d54e0d620267)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:11.354559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:11.399358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:11.419667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd182880b4c1646b2b2c1d54e0d620267'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.420860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd182880b4c1646b2b2c1d54e0d620267'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.435265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd182880b4c1646b2b2c1d54e0d620267'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.455960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd182880b4c1646b2b2c1d54e0d620267'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.477908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81bf68c7-547a-49f8-a0f6-93c8b017b196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd182880b4c1646b2b2c1d54e0d620267'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:11.488695 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3386/6817] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:11 2022] DELETE /identity/v3/users/d182880b4c1646b2b2c1d54e0d620267 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:12.776343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:12.777943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:12.804192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673140a-ec2f-418f-9e21-e42445e902c3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:12.929461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:12.930042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:12.931263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:12.931370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] PATH_INFO: `/v3/projects/7a427c3b011b4febb48ade20231552a7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:12.932696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a427c3b011b4febb48ade20231552a7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:12.934712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:13.005754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:13.018856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a427c3b011b4febb48ade20231552a7', 'request_id': 'req-30833712-2545-449e-931e-32975d2fd794'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.033959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a427c3b011b4febb48ade20231552a7', 'request_id': 'req-30833712-2545-449e-931e-32975d2fd794'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.044139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a427c3b011b4febb48ade20231552a7', 'request_id': 'req-30833712-2545-449e-931e-32975d2fd794'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.059846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] The token cache is being invalidate because project 7a427c3b011b4febb48ade20231552a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:13.060367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.061170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30833712-2545-449e-931e-32975d2fd794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.066561 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3432/6818] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:12 2022] DELETE /identity/v3/projects/7a427c3b011b4febb48ade20231552a7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:13.068472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5973c-a5b4-4927-b624-03c3a52f54c1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.068979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5973c-a5b4-4927-b624-03c3a52f54c1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.069591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5973c-a5b4-4927-b624-03c3a52f54c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:13.098905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9ae5973c-a5b4-4927-b624-03c3a52f54c1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:13.101471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9ae5973c-a5b4-4927-b624-03c3a52f54c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:13.236556 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3387/6819] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:13.238451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c9c4-1357-402d-962f-7cbd994c1106 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.238745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c9c4-1357-402d-962f-7cbd994c1106 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.239103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c9c4-1357-402d-962f-7cbd994c1106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:13.298689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ac45c9c4-1357-402d-962f-7cbd994c1106 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:13.300069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ac45c9c4-1357-402d-962f-7cbd994c1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:13.326600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5973c-a5b4-4927-b624-03c3a52f54c1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:13.337745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3433/6820] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:22:13.347336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5973c-a5b4-4927-b624-03c3a52f54c1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:13.362311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:13.362959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.363241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.363514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:13.365154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:22:13.365275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:13.366899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:13.404159 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3388/6821] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:22:13.407151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c9c4-1357-402d-962f-7cbd994c1106 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:13.430312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c9c4-1357-402d-962f-7cbd994c1106 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:13.444688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:13.448450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.449049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.449472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:13.450283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:13.456800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:13.519217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3434/6822] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:13 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:13.523297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:13.539634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1669-522d-45ed-a8c3-69ccb7011bff admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:13.738962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:13.740778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:13.741501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc784972-98b9-4a4c-8d8c-a09912a1667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:13.743339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784972-98b9-4a4c-8d8c-a09912a1667b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.743693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784972-98b9-4a4c-8d8c-a09912a1667b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.744972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784972-98b9-4a4c-8d8c-a09912a1667b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:13.747343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc784972-98b9-4a4c-8d8c-a09912a1667b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:13.753217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc784972-98b9-4a4c-8d8c-a09912a1667b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:13.768077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e769d7-cc81-4f43-96dc-b0153b814649 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:13.779917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:13.780528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.781253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.781534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] PATH_INFO: `/v3/users/f2cc166dabae4adc8f28b755de18f684` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:13.783031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2cc166dabae4adc8f28b755de18f684)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:13.785057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:13.794292 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3389/6823] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:13 2022] POST /identity/v3/users => generated 296 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:22:13.795832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc801-bfb3-43ec-95c9-d6fdec83434b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.796126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc801-bfb3-43ec-95c9-d6fdec83434b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.796408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc801-bfb3-43ec-95c9-d6fdec83434b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:13.840104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-129bc801-bfb3-43ec-95c9-d6fdec83434b None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:13.841477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-129bc801-bfb3-43ec-95c9-d6fdec83434b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:13.876877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2cc166dabae4adc8f28b755de18f684'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.882411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2cc166dabae4adc8f28b755de18f684'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.891796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:13.898797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2cc166dabae4adc8f28b755de18f684'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.910915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2cc166dabae4adc8f28b755de18f684'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.912632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2cc166dabae4adc8f28b755de18f684'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:13.944953 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3435/6824] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:13 2022] DELETE /identity/v3/users/f2cc166dabae4adc8f28b755de18f684 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:13.945977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:13.951566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:13.977433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3390/6825] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:13.981254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:13.981739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:13.982922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.039509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:14.041370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:14.109720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c74578-a9d7-4654-93fd-a0cfe9299986 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:14.122275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:14.122935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:14.123213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:14.123485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.124226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:14.130006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3391/6826] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:14.131161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:14.132626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:14.135211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:14.159590 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3436/6827] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:14.160593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:14.168929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:14.182928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:14.278163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96668d-e0f7-4ab6-b596-3570d4fb3fce admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:14.285028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:14.285664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:14.286012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:14.286289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.305858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:14.307347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:14.313178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2575d9-de30-4118-81fa-6360faa546ce None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:14.319391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:14.319877 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3437/6828] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:14.320230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:14.320598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:14.320988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.321098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:14.334959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f053079-ed3c-4577-9721-31e02f7fcfae service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:14.341097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:14.341880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:14.342158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:14.342830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.343736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:14.345436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:14.360894 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3438/6829] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:14.364855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:14.379466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fb44d-7927-4505-bb92-035a2683e92d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:14.386947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9440b-5838-4adb-96da-125557c1876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:14.387808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9440b-5838-4adb-96da-125557c1876c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:14.388214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9440b-5838-4adb-96da-125557c1876c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:14.388971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9440b-5838-4adb-96da-125557c1876c admin admin] PATH_INFO: `/v3/projects/d9a89937ac2c4f4db34c29070c651aa1/users/38180639051c4734b94afcf2686407a2/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.409991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9440b-5838-4adb-96da-125557c1876c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a89937ac2c4f4db34c29070c651aa1, user_id=38180639051c4734b94afcf2686407a2, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:14.412288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:14.412789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9440b-5838-4adb-96da-125557c1876c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:14.416564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:14.436920 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3439/6830] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:14 2022] PUT /identity/v3/projects/d9a89937ac2c4f4db34c29070c651aa1/users/38180639051c4734b94afcf2686407a2/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:22:14.441785 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3392/6831] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:14.448075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8634a-801e-43a6-9c0d-6f1f17a7074a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:14.448573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8634a-801e-43a6-9c0d-6f1f17a7074a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:14.449024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8634a-801e-43a6-9c0d-6f1f17a7074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.497001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-13d8634a-801e-43a6-9c0d-6f1f17a7074a None None] MFA Rules not processed for user `38180639051c4734b94afcf2686407a2`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:14.498259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-13d8634a-801e-43a6-9c0d-6f1f17a7074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:14.646132 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3440/6832] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:14.648987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:14.671512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f157c5-657c-4fe0-b87e-4b0ba3994769 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:14.714256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:14.915208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:14.916380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:14.917291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:14.917710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:14.919319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:14.922476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:15.078659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8634a-801e-43a6-9c0d-6f1f17a7074a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:15.079439 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3393/6833] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:14 2022] POST /identity/v3/projects => generated 387 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:15.098334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8634a-801e-43a6-9c0d-6f1f17a7074a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:15.242400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:15.243395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:15.243975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:15.244464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:15.245462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:15.251599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:15.268572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:15.273977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:15.291238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:15.348810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a911d99-8ca0-4901-a2b6-83caa994ae03 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:15.358728 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3441/6834] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:15 2022] POST /identity/v3/users => generated 300 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:15.365058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:15.374406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:15.375403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:15.377053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:15.377484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] PATH_INFO: `/v3/projects/87e1be172b4845fe9ad5333d323fff8b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:15.378854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e1be172b4845fe9ad5333d323fff8b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:15.383306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:15.404564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10066814-4ee9-431c-8697-c4a1dc3b8a95 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:15.422775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:15.425851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:15.429467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:15.430640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:15.433471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:15.442587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:15.482663 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3442/6835] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:15.486380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:15.517078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ba440-efb2-4944-8ddd-33d614141460 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:15.526229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e1be172b4845fe9ad5333d323fff8b', 'request_id': 'req-826de77f-1cbe-47e0-a96e-700b686e382e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:15.550304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e1be172b4845fe9ad5333d323fff8b', 'request_id': 'req-826de77f-1cbe-47e0-a96e-700b686e382e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:15.586843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e1be172b4845fe9ad5333d323fff8b', 'request_id': 'req-826de77f-1cbe-47e0-a96e-700b686e382e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:15.637167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] The token cache is being invalidate because project 87e1be172b4845fe9ad5333d323fff8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:15.638018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:15.639321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-826de77f-1cbe-47e0-a96e-700b686e382e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:15.643070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3394/6836] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:15 2022] DELETE /identity/v3/projects/87e1be172b4845fe9ad5333d323fff8b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:15.665108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fbf24-436a-4a5e-95e0-9e25e7557a0a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:15.665692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fbf24-436a-4a5e-95e0-9e25e7557a0a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:15.667042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fbf24-436a-4a5e-95e0-9e25e7557a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:15.667486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:15.668277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:15.668817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:15.668971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] PATH_INFO: `/v3/projects/5d466ba764d243d6af0280af36e5e25d/users/29665656fb22490f9306bcb058d07be1/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:15.715051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ec9fbf24-436a-4a5e-95e0-9e25e7557a0a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:15.718174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec9fbf24-436a-4a5e-95e0-9e25e7557a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:15.722188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d466ba764d243d6af0280af36e5e25d, user_id=29665656fb22490f9306bcb058d07be1, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:15.725715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:15.731670 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3395/6837] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:15.741155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26972a-d76a-412f-93d8-dcb48d2644b1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:15.741749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26972a-d76a-412f-93d8-dcb48d2644b1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:15.742371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26972a-d76a-412f-93d8-dcb48d2644b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:15.801740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4d26972a-d76a-412f-93d8-dcb48d2644b1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:15.811124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d26972a-d76a-412f-93d8-dcb48d2644b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:15.811548 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3443/6838] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:15 2022] PUT /identity/v3/projects/5d466ba764d243d6af0280af36e5e25d/users/29665656fb22490f9306bcb058d07be1/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:15.813609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:15.820754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:15.988434 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3396/6839] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:22:15.990880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26972a-d76a-412f-93d8-dcb48d2644b1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.001537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5b1c-8483-4784-b3b1-de5291730a8c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.005895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26972a-d76a-412f-93d8-dcb48d2644b1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.008483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.009454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.009963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.010526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.011339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.013019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.013506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.013690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.013978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.014301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.014970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:22:16.015271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.017737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.036189 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3444/6840] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:16.042630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.053733 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3397/6841] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:22:16.054911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.071040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:16.074787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8083f3d-72c9-4d32-9d7a-303958edd4f6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.079077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293a52-bb18-4f2e-afc4-2f2e7ebf21c2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.086684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63175cc2-bdf5-44a5-86df-7e6ee9d92dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.087181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.087648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63175cc2-bdf5-44a5-86df-7e6ee9d92dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.087992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63175cc2-bdf5-44a5-86df-7e6ee9d92dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.088351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63175cc2-bdf5-44a5-86df-7e6ee9d92dc5 admin admin] PATH_INFO: `/v3/projects/5d466ba764d243d6af0280af36e5e25d/users/29665656fb22490f9306bcb058d07be1/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.088975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.089430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.089996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.090840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.091112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63175cc2-bdf5-44a5-86df-7e6ee9d92dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d466ba764d243d6af0280af36e5e25d, user_id=29665656fb22490f9306bcb058d07be1, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.093256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.094680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63175cc2-bdf5-44a5-86df-7e6ee9d92dc5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.109786 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3445/6842] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:16 2022] PUT /identity/v3/projects/5d466ba764d243d6af0280af36e5e25d/users/29665656fb22490f9306bcb058d07be1/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:16.123932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034c7d8-6ddd-4437-ac90-1819c71d22f3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.124276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034c7d8-6ddd-4437-ac90-1819c71d22f3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.124644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034c7d8-6ddd-4437-ac90-1819c71d22f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.151295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3398/6843] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:16 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:16.152548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.171578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2034c7d8-6ddd-4437-ac90-1819c71d22f3 None None] MFA Rules not processed for user `29665656fb22490f9306bcb058d07be1`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:16.172997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2034c7d8-6ddd-4437-ac90-1819c71d22f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:16.282359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2f6e1-94e0-48d8-8745-da6f5dae01a5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.289142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:16.289639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.290773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.290773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.291187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.292304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.294741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.350023 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3399/6844] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:16 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:16.355177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.370241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623f7d7-fc7a-43d7-85b1-e9ff571558ab admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.377564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.378531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.379031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.379494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.380795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.382707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.396351 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3446/6845] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:16.403321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034c7d8-6ddd-4437-ac90-1819c71d22f3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.403722 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3400/6846] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:22:16.410757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.430217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034c7d8-6ddd-4437-ac90-1819c71d22f3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.430996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:16.439993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.440913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.441350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.441755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.442580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.444915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.472524 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3447/6847] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:16.476900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.497225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4f7cb-6b06-4c5a-8029-0fb437d3e056 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.515326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.516297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.517835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.519688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] PATH_INFO: `/v3/projects/ab5aec6b80a447bab8eb4f1adf06e61f/users/b652b17bea8942c4b4a30034c6c1f44c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.549364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d37cc-2a1f-43ef-9ba7-42bddaec7771 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.554059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9027864-7134-4cf2-a1e3-0124ecd8dad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.554865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027864-7134-4cf2-a1e3-0124ecd8dad2 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.555282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027864-7134-4cf2-a1e3-0124ecd8dad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.555838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027864-7134-4cf2-a1e3-0124ecd8dad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.565952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5aec6b80a447bab8eb4f1adf06e61f, user_id=b652b17bea8942c4b4a30034c6c1f44c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.571248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9027864-7134-4cf2-a1e3-0124ecd8dad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.573785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9027864-7134-4cf2-a1e3-0124ecd8dad2 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.580571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:16.615462 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3401/6848] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:16.624191 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3448/6849] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:16 2022] PUT /identity/v3/projects/ab5aec6b80a447bab8eb4f1adf06e61f/users/b652b17bea8942c4b4a30034c6c1f44c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:16.636296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.636578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.636910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.676016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] MFA Rules not processed for user `b652b17bea8942c4b4a30034c6c1f44c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:16.678603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:16.781391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:16.823332 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3402/6850] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:16.831320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:16.834707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:16.971441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4599e6-85d2-4096-a5bf-4e7a2aa34725 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:16.977782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:16.979642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:16.980126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:16.980566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:16.991742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:16.993902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:17.010229 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3449/6851] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:17.281388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:17.282878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:17.438993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3cef-4b57-4a42-a953-960511a83ac8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:17.449543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:17.450357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:17.450733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:17.451105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] PATH_INFO: `/v3/users/64c9f44e57164b858421b55114acf4cf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:17.452478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c9f44e57164b858421b55114acf4cf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:17.454762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:17.531942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:17.824118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dd261-e482-4422-8ff2-394e4ae45cef service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:17.831231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:17.832358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:17.833685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:17.835963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:17.841172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:17.881551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c9f44e57164b858421b55114acf4cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:17.882631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c9f44e57164b858421b55114acf4cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:17.916245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c9f44e57164b858421b55114acf4cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:17.955109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c9f44e57164b858421b55114acf4cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:17.983755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2e9fb344-1b98-4ade-9596-12c799569b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c9f44e57164b858421b55114acf4cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:17.997202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:18.004303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:18.009683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:18.017243 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3403/6852] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:17 2022] DELETE /identity/v3/users/64c9f44e57164b858421b55114acf4cf => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:18.019477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cd6d8-206d-40c7-b54b-a0407d9255bf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:18.019775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cd6d8-206d-40c7-b54b-a0407d9255bf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:18.020040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cd6d8-206d-40c7-b54b-a0407d9255bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:18.074144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3450/6853] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:22:18.103771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6b3cd6d8-206d-40c7-b54b-a0407d9255bf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:18.105053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6b3cd6d8-206d-40c7-b54b-a0407d9255bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:18.304887 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3404/6854] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:18.312496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:18.341353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc2e5-b32a-4d8f-bc92-54c81ac4d014 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:18.466172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:18.467068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:18.467416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:18.467687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:18.506196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:18.523312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:18.548288 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3451/6855] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:22:18.985311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3ea4b-d5c7-4ddf-93a9-e50e34d0aa54 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:18.985419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3ea4b-d5c7-4ddf-93a9-e50e34d0aa54 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:18.985702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3ea4b-d5c7-4ddf-93a9-e50e34d0aa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:19.013948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9fa3ea4b-d5c7-4ddf-93a9-e50e34d0aa54 None None] MFA Rules not processed for user `b652b17bea8942c4b4a30034c6c1f44c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:19.015499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9fa3ea4b-d5c7-4ddf-93a9-e50e34d0aa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:19.022169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:19.107421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3405/6856] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:19.446313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:19.448123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:19.473809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e86da3-48f7-4f56-a3b4-3710d591da52 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:19.492733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:19.493629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:19.494049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:19.494455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] PATH_INFO: `/v3/projects/932da48df2c44fc6a7dbd6c8daf9fc0d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:19.497990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932da48df2c44fc6a7dbd6c8daf9fc0d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:19.500587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:19.646379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'request_id': 'req-53e90e64-a62c-4fee-ab6a-876b82efaec2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:19.666805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'request_id': 'req-53e90e64-a62c-4fee-ab6a-876b82efaec2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:19.680399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'request_id': 'req-53e90e64-a62c-4fee-ab6a-876b82efaec2'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:19.710016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] The token cache is being invalidate because project 932da48df2c44fc6a7dbd6c8daf9fc0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:19.710724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:19.717413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:19.719671 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3452/6857] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:19 2022] DELETE /identity/v3/projects/932da48df2c44fc6a7dbd6c8daf9fc0d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:19.724468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3ea4b-d5c7-4ddf-93a9-e50e34d0aa54 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:19.759061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3ea4b-d5c7-4ddf-93a9-e50e34d0aa54 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:19.888120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:19.889419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:19.889779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:19.890127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] PATH_INFO: `/v3/users/f0f8930eeda742ea9f3483e3314f1a88` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:19.891642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f8930eeda742ea9f3483e3314f1a88)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:19.896615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:20.009596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f8930eeda742ea9f3483e3314f1a88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:20.010682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f8930eeda742ea9f3483e3314f1a88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:20.027230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f8930eeda742ea9f3483e3314f1a88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:20.040460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f8930eeda742ea9f3483e3314f1a88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:20.076157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f8930eeda742ea9f3483e3314f1a88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:20.091687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3406/6858] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:19 2022] DELETE /identity/v3/users/f0f8930eeda742ea9f3483e3314f1a88 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:20.821749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:20.823999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:20.846089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e90e64-a62c-4fee-ab6a-876b82efaec2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:20.972790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:20.974195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:20.974692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:20.975175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] PATH_INFO: `/v3/projects/430fffd9dbbb49ab90d73cafa9b64fe3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:20.976848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430fffd9dbbb49ab90d73cafa9b64fe3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:20.979859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:20.995313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:21.067369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:21.068894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:21.083248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430fffd9dbbb49ab90d73cafa9b64fe3', 'request_id': 'req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.091554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44564a8a-c297-4b89-a135-554b8d7bbaa3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:21.101508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430fffd9dbbb49ab90d73cafa9b64fe3', 'request_id': 'req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.108202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430fffd9dbbb49ab90d73cafa9b64fe3', 'request_id': 'req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.141545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] The token cache is being invalidate because project 430fffd9dbbb49ab90d73cafa9b64fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:21.142491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.144281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.147633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3453/6859] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:20 2022] DELETE /identity/v3/projects/430fffd9dbbb49ab90d73cafa9b64fe3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:21.153254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:21.155320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:21.159418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:21.210731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:21.213323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:21.214730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:21.215225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] PATH_INFO: `/v3/users/38180639051c4734b94afcf2686407a2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:21.217383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38180639051c4734b94afcf2686407a2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:21.220390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:21.271336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5a4f0-9866-4dca-9e53-f7924f700dc0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:21.277165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:21.280435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:21.280435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:21.280435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] PATH_INFO: `/v3/users/7499cab28aae45eabab969fcaf60cea3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:21.280435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7499cab28aae45eabab969fcaf60cea3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:21.282833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:21.388868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38180639051c4734b94afcf2686407a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.391718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38180639051c4734b94afcf2686407a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.401026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7499cab28aae45eabab969fcaf60cea3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.404099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7499cab28aae45eabab969fcaf60cea3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.427008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38180639051c4734b94afcf2686407a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.454133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7499cab28aae45eabab969fcaf60cea3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.470683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7499cab28aae45eabab969fcaf60cea3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.472974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7499cab28aae45eabab969fcaf60cea3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.475639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38180639051c4734b94afcf2686407a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.524551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-568b3e05-be75-4b87-8bfa-0d4158d1eb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38180639051c4734b94afcf2686407a2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:21.537946 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3454/6860] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:21 2022] DELETE /identity/v3/users/7499cab28aae45eabab969fcaf60cea3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:21.539248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:21.569754 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3407/6861] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:21 2022] DELETE /identity/v3/users/38180639051c4734b94afcf2686407a2 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:22:21.571991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:21.575103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:21.575636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:21.576344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:21.616395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:21.625235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:21.626836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:21.737374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbf523-d204-4514-83e5-0f3468248814 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:21.740399 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3408/6862] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:22:21.750224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:21.751878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:21.856265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb5c8-6146-4cea-991b-a1ce6cbbc4da None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:21.864520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:21.867740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:21.868375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:21.869405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:21.869673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:21.870501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:21.870786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:21.871225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:21.873242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:21.891138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:21.892005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:21.892967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:21.894553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:21.904517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:21.924609 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3409/6863] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:21.928326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3455/6864] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:22.316891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:22.319370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:22.384573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c603225-89dc-403d-bf08-a493b8cd004f service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:22.397474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:22.398262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:22.398608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:22.398939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] PATH_INFO: `/v3/projects/d9a89937ac2c4f4db34c29070c651aa1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:22.404728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a89937ac2c4f4db34c29070c651aa1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:22.410980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:22.683482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a89937ac2c4f4db34c29070c651aa1', 'request_id': 'req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:22.703022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a89937ac2c4f4db34c29070c651aa1', 'request_id': 'req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:22.726188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a89937ac2c4f4db34c29070c651aa1', 'request_id': 'req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:22.758472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] The token cache is being invalidate because project d9a89937ac2c4f4db34c29070c651aa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:22.759200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:22.762871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:22.763218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3410/6865] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:22 2022] DELETE /identity/v3/projects/d9a89937ac2c4f4db34c29070c651aa1 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:22.767008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:22.789959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be53a8a-d9df-4a4e-ac24-d235477b0c2c service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:22.937451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:22.938175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:22.938326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:22.938567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] PATH_INFO: `/v3/users/29665656fb22490f9306bcb058d07be1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:22.941076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=29665656fb22490f9306bcb058d07be1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:22.943204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:23.071845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29665656fb22490f9306bcb058d07be1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.075221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29665656fb22490f9306bcb058d07be1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.089646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29665656fb22490f9306bcb058d07be1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.091257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:23.104330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29665656fb22490f9306bcb058d07be1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.105028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f466904e-ebc9-439a-9206-d69e17a83e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29665656fb22490f9306bcb058d07be1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.184059 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3456/6866] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:22 2022] DELETE /identity/v3/users/29665656fb22490f9306bcb058d07be1 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:23.291316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:23.295896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:23.324541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0cc5d-1c85-4a68-ad48-6e4dd6b1d4c7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:23.447349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:23.448552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:23.449013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:23.449428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] PATH_INFO: `/v3/projects/e7b9670ac0de4e028fe2eb0d7636090c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:23.450975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b9670ac0de4e028fe2eb0d7636090c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:23.459095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:23.595033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b9670ac0de4e028fe2eb0d7636090c', 'request_id': 'req-21c99f5e-7f60-411d-ad1c-ae989dc872d9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.615383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b9670ac0de4e028fe2eb0d7636090c', 'request_id': 'req-21c99f5e-7f60-411d-ad1c-ae989dc872d9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.643157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b9670ac0de4e028fe2eb0d7636090c', 'request_id': 'req-21c99f5e-7f60-411d-ad1c-ae989dc872d9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.667673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] The token cache is being invalidate because project e7b9670ac0de4e028fe2eb0d7636090c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:23.667973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.673921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-21c99f5e-7f60-411d-ad1c-ae989dc872d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:23.675475 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3411/6867] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:23 2022] DELETE /identity/v3/projects/e7b9670ac0de4e028fe2eb0d7636090c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:23.680624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:23.680941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:23.681229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:23.727977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:23.729298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:23.882617 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3457/6868] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:23.890156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621f3a-dd26-4790-9eda-bfc344349e41 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:23.890418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621f3a-dd26-4790-9eda-bfc344349e41 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:23.890643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621f3a-dd26-4790-9eda-bfc344349e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:23.948494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-69621f3a-dd26-4790-9eda-bfc344349e41 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:23.953184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-69621f3a-dd26-4790-9eda-bfc344349e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:23.968439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:23.970007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:23.977350 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3412/6869] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:23.978524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621f3a-dd26-4790-9eda-bfc344349e41 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:23.987959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:23.994083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621f3a-dd26-4790-9eda-bfc344349e41 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.005018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717f3d6-c578-484d-af5e-972cb4c481f3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.007682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:24.008389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.008678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.012947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.022028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:22:24.022339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:24.025655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:24.033794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:24.035590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.038718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.039840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] PATH_INFO: `/v3/projects/5d466ba764d243d6af0280af36e5e25d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.042659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d466ba764d243d6af0280af36e5e25d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:24.048329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:24.050643 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3413/6870] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:22:24.051751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:24.054119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:24.241363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d466ba764d243d6af0280af36e5e25d', 'request_id': 'req-b5076006-1fec-470e-a3ab-9b97f3cf955c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:24.276642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d466ba764d243d6af0280af36e5e25d', 'request_id': 'req-b5076006-1fec-470e-a3ab-9b97f3cf955c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:24.293488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d466ba764d243d6af0280af36e5e25d', 'request_id': 'req-b5076006-1fec-470e-a3ab-9b97f3cf955c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:24.328858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] The token cache is being invalidate because project 5d466ba764d243d6af0280af36e5e25d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:24.329338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:24.330238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:24.332593 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3458/6871] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:23 2022] DELETE /identity/v3/projects/5d466ba764d243d6af0280af36e5e25d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:24.333393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:24.334620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aace8-355a-4f48-bda3-0c8ef237b6a6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.335098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:24.456119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42604a7-d8be-47f0-b5ff-b9bfb6789502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:24.457447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42604a7-d8be-47f0-b5ff-b9bfb6789502 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.457788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42604a7-d8be-47f0-b5ff-b9bfb6789502 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.458067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42604a7-d8be-47f0-b5ff-b9bfb6789502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.463481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d42604a7-d8be-47f0-b5ff-b9bfb6789502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:24.484686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:24.493161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5076006-1fec-470e-a3ab-9b97f3cf955c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.508182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:24.510046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.510177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42604a7-d8be-47f0-b5ff-b9bfb6789502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:24.510673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.512178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42604a7-d8be-47f0-b5ff-b9bfb6789502 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:24.513564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.515412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:24.521570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:24.543440 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3414/6872] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:24.550594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68983c2-3e0a-4da9-af8d-3ed70f8e693d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.551572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68983c2-3e0a-4da9-af8d-3ed70f8e693d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.551572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68983c2-3e0a-4da9-af8d-3ed70f8e693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.614347 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3459/6873] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:24 2022] POST /identity/v3/projects => generated 373 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:24.615133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:24.623196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c68983c2-3e0a-4da9-af8d-3ed70f8e693d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:24.624535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c68983c2-3e0a-4da9-af8d-3ed70f8e693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:24.762199 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3415/6874] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:24.776174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b71076-1583-4f80-bbef-6fe7128bda95 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.777513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b71076-1583-4f80-bbef-6fe7128bda95 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.778477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b71076-1583-4f80-bbef-6fe7128bda95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.796279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d013762-72be-4cd8-acfd-5d99acd9bc7d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.808579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:24.808891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e7b71076-1583-4f80-bbef-6fe7128bda95 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:24.809361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.809643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.809917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.810606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e7b71076-1583-4f80-bbef-6fe7128bda95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:24.811742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:24.816501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:24.820375 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3416/6875] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:24.825049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b71076-1583-4f80-bbef-6fe7128bda95 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:24.857922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b71076-1583-4f80-bbef-6fe7128bda95 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.873396 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3460/6876] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:24 2022] POST /identity/v3/users => generated 287 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:24.877429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:24.877898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:24.878535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.878812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.879082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.879841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:22:24.880146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:24.881905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:24.903192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471f17-6eb5-45cc-95ea-ef7b876bf7e7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.934200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:24.934936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:24.938025 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3417/6877] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:22:24.938339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:24.939887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:24.940475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:24.941106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:24.945763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:24.987678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc727-4e89-42ad-a553-4d69158fdb7e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:24.993597 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3461/6878] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:22:25.000690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:25.004099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:25.005843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.017129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.017546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.020845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:25.022961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:25.034212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031d28-39b9-415f-901d-6b655031c389 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:25.048175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:25.048998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.049360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.049726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.050740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:25.052526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:25.081403 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3462/6879] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:25.085020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3418/6880] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:24 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:25.089250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:25.089576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:25.108649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8762b7-9893-465b-9597-d1ec14592024 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:25.112765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:25.277672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f810d-4afd-47db-bdea-51c5f6e45efb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:25.286926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fd858-5104-418d-b340-6b5277eed88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:25.290439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fd858-5104-418d-b340-6b5277eed88c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.292511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fd858-5104-418d-b340-6b5277eed88c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.292959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fd858-5104-418d-b340-6b5277eed88c admin admin] PATH_INFO: `/v3/projects/4195812e6f274bbba026e71b7fc4e231/users/dc394c36e4bb436fabbecada1990b41e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.294708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:25.295675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.297189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.297603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.299140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:25.305756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:25.329511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fd858-5104-418d-b340-6b5277eed88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4195812e6f274bbba026e71b7fc4e231, user_id=dc394c36e4bb436fabbecada1990b41e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:25.334072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fd858-5104-418d-b340-6b5277eed88c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:25.365295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3419/6881] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:25 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:25.369989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:25.380232 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3463/6882] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:25 2022] PUT /identity/v3/projects/4195812e6f274bbba026e71b7fc4e231/users/dc394c36e4bb436fabbecada1990b41e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:25.386514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755a8d0-b62c-4a42-b0d4-4d8649218233 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.387086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755a8d0-b62c-4a42-b0d4-4d8649218233 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.387544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755a8d0-b62c-4a42-b0d4-4d8649218233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.398682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1d8ed-eead-49f2-877d-379083dc4567 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:25.438412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f755a8d0-b62c-4a42-b0d4-4d8649218233 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:25.441343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f755a8d0-b62c-4a42-b0d4-4d8649218233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:25.538680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:25.571633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:25.572270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.572545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.572841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.573496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:25.575141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:25.598417 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3420/6883] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:25.602624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:25.621849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3464/6884] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:25.622928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ec335-d05d-4af6-a2a5-3e61ce754670 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:25.628652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9d30c-840f-412a-9b08-b27c9d02c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:25.630132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9d30c-840f-412a-9b08-b27c9d02c367 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.630387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9d30c-840f-412a-9b08-b27c9d02c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.630662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9d30c-840f-412a-9b08-b27c9d02c367 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.631313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9d30c-840f-412a-9b08-b27c9d02c367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:25.632944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9d30c-840f-412a-9b08-b27c9d02c367 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:25.648083 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3421/6885] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:25.650986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755a8d0-b62c-4a42-b0d4-4d8649218233 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:25.670021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755a8d0-b62c-4a42-b0d4-4d8649218233 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:25.680076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:25.680946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.681218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.681491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] PATH_INFO: `/v3/projects/14a715df3201452e83b3cd98bcfcdd85/users/604bb3b24e0d404f9daa9ce5855a34bc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.706124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a715df3201452e83b3cd98bcfcdd85, user_id=604bb3b24e0d404f9daa9ce5855a34bc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:25.708880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:25.729462 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3465/6886] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:25 2022] PUT /identity/v3/projects/14a715df3201452e83b3cd98bcfcdd85/users/604bb3b24e0d404f9daa9ce5855a34bc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:25.740757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:25.741070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:25.741363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:25.800079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] MFA Rules not processed for user `604bb3b24e0d404f9daa9ce5855a34bc`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:25.801729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:25.959011 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3422/6887] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:25.987201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:25.989785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:26.107209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3bcd-5d34-4e05-be64-f3509dc5e4a7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:26.111020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:26.111641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:26.111919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:26.112278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:26.125404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:26.127748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:26.153252 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3466/6888] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:26.167436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:26.169044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:26.193832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8897b4-86c3-4db7-8bfc-165b0527fc60 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:26.201384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:26.202596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:26.202866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:26.203144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:26.219889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:26.326127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:26.329759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:26.344944 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3423/6889] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:27.956311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:27.959136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:28.205251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c89664-ba3c-4af6-90ce-beda540d87e5 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:28.232794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cef457-3d78-4673-8d44-447224c3008a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:28.233879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cef457-3d78-4673-8d44-447224c3008a service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:28.234922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cef457-3d78-4673-8d44-447224c3008a service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:28.236085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cef457-3d78-4673-8d44-447224c3008a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:28.240042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-88cef457-3d78-4673-8d44-447224c3008a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:28.270326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cef457-3d78-4673-8d44-447224c3008a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:28.272482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cef457-3d78-4673-8d44-447224c3008a service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:28.291892 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3467/6890] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:30.757364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:30.759253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:30.792458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504d076-2230-427a-a4df-2f5ab83b0693 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:30.808743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:30.809948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:30.810316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:30.810598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] PATH_INFO: `/v3/users/2e3ba57b36d34b40bcfa113cb71ef8bc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:30.812155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3ba57b36d34b40bcfa113cb71ef8bc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:30.820979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:30.948739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3ba57b36d34b40bcfa113cb71ef8bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:30.955891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3ba57b36d34b40bcfa113cb71ef8bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:30.968908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3ba57b36d34b40bcfa113cb71ef8bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:30.996692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3ba57b36d34b40bcfa113cb71ef8bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:31.043619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3ba57b36d34b40bcfa113cb71ef8bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:31.074697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3424/6891] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:30 2022] DELETE /identity/v3/users/2e3ba57b36d34b40bcfa113cb71ef8bc => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:31.079256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e48d-3993-41ff-97f3-94fe7a20b073 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:31.079713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e48d-3993-41ff-97f3-94fe7a20b073 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:31.079794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e48d-3993-41ff-97f3-94fe7a20b073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:31.107032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1f43e48d-3993-41ff-97f3-94fe7a20b073 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:31.108465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1f43e48d-3993-41ff-97f3-94fe7a20b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:31.176025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:31.242726 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3468/6892] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:31.623841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:31.627978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:31.830795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2985df-8511-47ca-b66e-7cd833d79d7d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:31.845240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:31.845897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:31.846180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:31.846461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:31.869429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:31.870899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:31.903117 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3425/6893] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:32.892013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e48d-3993-41ff-97f3-94fe7a20b073 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:32.909995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e48d-3993-41ff-97f3-94fe7a20b073 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:32.940796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf8fb2-2161-45b8-b593-57add8ac54a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:32.941613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8fb2-2161-45b8-b593-57add8ac54a2 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:32.941918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8fb2-2161-45b8-b593-57add8ac54a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:32.942224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8fb2-2161-45b8-b593-57add8ac54a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:32.944991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7bcf8fb2-2161-45b8-b593-57add8ac54a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:32.975178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf8fb2-2161-45b8-b593-57add8ac54a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:32.976686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf8fb2-2161-45b8-b593-57add8ac54a2 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:33.001114 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3469/6894] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:22:33.744954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:33.747185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:33.786346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa2437-3fc6-499a-99e7-db75f4deeb66 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:33.799627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:33.800623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:33.801155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:33.801684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] PATH_INFO: `/v3/projects/ff1e17cb29c54a1aabe7da7009c347c5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:33.804217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1e17cb29c54a1aabe7da7009c347c5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:33.814589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:33.857506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:33.941940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1e17cb29c54a1aabe7da7009c347c5', 'request_id': 'req-c6068a94-3046-4f42-a242-376c58a8466c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:33.958770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1e17cb29c54a1aabe7da7009c347c5', 'request_id': 'req-c6068a94-3046-4f42-a242-376c58a8466c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:33.973604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1e17cb29c54a1aabe7da7009c347c5', 'request_id': 'req-c6068a94-3046-4f42-a242-376c58a8466c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:33.997601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] The token cache is being invalidate because project ff1e17cb29c54a1aabe7da7009c347c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:33.998182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:33.999111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c6068a94-3046-4f42-a242-376c58a8466c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:34.001466 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3426/6895] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:33 2022] DELETE /identity/v3/projects/ff1e17cb29c54a1aabe7da7009c347c5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:34.011436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31a7d2-ec44-43a7-b1e2-cc4e05650760 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:34.012813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31a7d2-ec44-43a7-b1e2-cc4e05650760 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:34.013333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31a7d2-ec44-43a7-b1e2-cc4e05650760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:34.040742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0f31a7d2-ec44-43a7-b1e2-cc4e05650760 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:34.043031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0f31a7d2-ec44-43a7-b1e2-cc4e05650760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:34.192784 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3470/6896] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:34.202568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243659f8-58e9-4176-ab06-610ff8589b90 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:34.203160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243659f8-58e9-4176-ab06-610ff8589b90 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:34.203627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243659f8-58e9-4176-ab06-610ff8589b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:34.232944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-243659f8-58e9-4176-ab06-610ff8589b90 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:34.234813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-243659f8-58e9-4176-ab06-610ff8589b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:34.288117 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3427/6897] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:34.294296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31a7d2-ec44-43a7-b1e2-cc4e05650760 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:34.310907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31a7d2-ec44-43a7-b1e2-cc4e05650760 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:34.331588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:34.334595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:34.335122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:34.337015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:34.339616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:22:34.340216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:34.344756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:34.390534 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3471/6898] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:22:34.392015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243659f8-58e9-4176-ab06-610ff8589b90 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:34.406562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243659f8-58e9-4176-ab06-610ff8589b90 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:34.419395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:34.420091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:34.420466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:34.420851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:34.422184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:34.424268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:34.494366 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3428/6899] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:34 2022] POST /identity/v3/projects => generated 359 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:22:34.499296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:34.515075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ef79-6f6c-42e1-a6fe-8bd261cb10e8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:34.699752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:34.700621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:34.701039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:34.701994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:34.704438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:34.717791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:34.771309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3472/6900] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:34 2022] POST /identity/v3/users => generated 280 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:34.775858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:34.796476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4984b4-bd9b-4b96-9d82-5e9d7c765e52 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:34.810247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:34.811209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:34.811490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:34.812930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:34.813609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:34.819759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:34.846671 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3429/6901] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:34.850606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:34.867346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8abd8-aaa0-4d77-a130-e3324a9ba3c1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:34.885010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99b22f-d263-4cf6-a53b-64e0dff8c238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:34.889298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99b22f-d263-4cf6-a53b-64e0dff8c238 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:34.889609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99b22f-d263-4cf6-a53b-64e0dff8c238 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:34.891326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99b22f-d263-4cf6-a53b-64e0dff8c238 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:34.893519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99b22f-d263-4cf6-a53b-64e0dff8c238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:34.901381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99b22f-d263-4cf6-a53b-64e0dff8c238 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:34.955663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:34.971891 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3473/6902] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:34.976916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f22506-3f8b-45dc-9099-641d53272127 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:35.016104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:35.016841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:35.017122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:35.022668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] PATH_INFO: `/v3/projects/7c062933ff2d4cceba38c9245a1770ea/users/854702c021af4fcc9e2ade1547aaf3b6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:35.109483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c062933ff2d4cceba38c9245a1770ea, user_id=854702c021af4fcc9e2ade1547aaf3b6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:35.123984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:35.166052 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3430/6903] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:34 2022] PUT /identity/v3/projects/7c062933ff2d4cceba38c9245a1770ea/users/854702c021af4fcc9e2ade1547aaf3b6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:35.177108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:35.177933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:35.177933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:35.214506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] MFA Rules not processed for user `854702c021af4fcc9e2ade1547aaf3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:35.215957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:35.273105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:35.468187 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3474/6904] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:35.497097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:35.502286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:35.639511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e5c3e-9b7d-4e0d-943c-41433003265d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:35.645607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaa6b4-8d4c-4777-9caa-64b5541f6cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:35.646268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa6b4-8d4c-4777-9caa-64b5541f6cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:35.646590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa6b4-8d4c-4777-9caa-64b5541f6cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:35.646860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa6b4-8d4c-4777-9caa-64b5541f6cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:35.670296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaa6b4-8d4c-4777-9caa-64b5541f6cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:35.674539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaa6b4-8d4c-4777-9caa-64b5541f6cb3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:35.688830 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3431/6905] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:40.356182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:40.357846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:40.497707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2f645-af53-4885-b41b-dceb2ecd9db8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:40.518452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:40.519425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:40.519911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:40.520397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:40.534228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:40.561199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:40.562982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:40.599741 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3475/6906] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:43.596374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:43.599421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:43.600040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:43.646538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] MFA Rules not processed for user `dc394c36e4bb436fabbecada1990b41e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:43.648210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:43.755087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:43.888706 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3432/6907] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:43.892956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:43.894527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:43.918413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:43.920941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:44.034676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2e592-ae1e-4c12-8b8d-a66f998b24cb service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.059032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb3939-2740-42e2-add9-e73a9419ca62 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.059772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.060571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.061823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.062104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.063294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.067189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:44.073884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.075110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.075395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.075666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] PATH_INFO: `/v3/users/1d92f7d2ee2d47beaa4f5d43ac40cd82` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.079279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d92f7d2ee2d47beaa4f5d43ac40cd82)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.086520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:44.138659 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3476/6908] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:22:43 2022] POST /identity/v3/projects => generated 373 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:44.162167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:44.217617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d92f7d2ee2d47beaa4f5d43ac40cd82'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:44.219583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d92f7d2ee2d47beaa4f5d43ac40cd82'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:44.227542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:44.244221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d92f7d2ee2d47beaa4f5d43ac40cd82'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:44.261780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d92f7d2ee2d47beaa4f5d43ac40cd82'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:44.293681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-377d3157-7924-4969-b3d7-d088278876ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d92f7d2ee2d47beaa4f5d43ac40cd82'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:44.309809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f88e96-2f52-46fc-abac-e27e2d732549 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.314912 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3433/6909] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:43 2022] DELETE /identity/v3/users/1d92f7d2ee2d47beaa4f5d43ac40cd82 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:44.316607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.317009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.317009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.328573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.329261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.329533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.330319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.330426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.338610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:44.352404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:44.355236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:44.385865 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3477/6910] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:22:44 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:22:44.393496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:44.398287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:44.430921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0db7f-2f07-41bd-b981-a9d1162bc3bd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.498672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:44.540412 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3434/6911] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:22:44.549476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:44.550524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:44.595461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.595947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.596394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.596801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.597673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.600744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:44.628548 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3478/6912] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:44.635652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:44.660576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e41f-5b7b-486d-bc19-9300d88405b5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.679664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.680400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.681289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.682330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] PATH_INFO: `/v3/projects/29517ab16de8485a9044b547b7cf260c/users/37715b55759e4187b56197b342a84947/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.683658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e055be-8388-4a31-92a7-6067ecd981c1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.697329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.698221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.698661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.699072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.719374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.721124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:44.741543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=29517ab16de8485a9044b547b7cf260c, user_id=37715b55759e4187b56197b342a84947, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.745012 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3435/6913] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:22:44.757812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:44.801086 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3479/6914] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:44 2022] PUT /identity/v3/projects/29517ab16de8485a9044b547b7cf260c/users/37715b55759e4187b56197b342a84947/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:44.805550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:44.822065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842043e-c51a-4c4d-b991-7b13e122fc82 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.926118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.926926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.927564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.927564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.928530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.930212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:44.951102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3436/6915] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:22:44.954947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:44.972526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1668ba-1e47-4e21-9d8a-f4a65eb40e0b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:44.978623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb6784-eaac-4615-a776-8a247c4cf298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:44.979477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6784-eaac-4615-a776-8a247c4cf298 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:44.979881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6784-eaac-4615-a776-8a247c4cf298 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:44.980249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb6784-eaac-4615-a776-8a247c4cf298 admin admin] PATH_INFO: `/v3/projects/29517ab16de8485a9044b547b7cf260c/users/37715b55759e4187b56197b342a84947/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:44.985642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb6784-eaac-4615-a776-8a247c4cf298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29517ab16de8485a9044b547b7cf260c, user_id=37715b55759e4187b56197b342a84947, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:44.988848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb6784-eaac-4615-a776-8a247c4cf298 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:45.006133 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3480/6916] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:22:44 2022] PUT /identity/v3/projects/29517ab16de8485a9044b547b7cf260c/users/37715b55759e4187b56197b342a84947/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:46.180291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:46.182336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:46.205446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ecad5-d491-4da7-aa1f-fe6f12928f06 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:46.384750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:46.385420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:46.385646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:46.385914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] PATH_INFO: `/v3/projects/dbfb9f03e6de43a6bd5e7a6641ac8088` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:46.387691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbfb9f03e6de43a6bd5e7a6641ac8088)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:46.389651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:46.492411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:46.514366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:46.514906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:46.514906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:46.518676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'request_id': 'req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.529461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'request_id': 'req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.547503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:46.549401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'request_id': 'req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.557533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:46.569248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] The token cache is being invalidate because project dbfb9f03e6de43a6bd5e7a6641ac8088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:46.572541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.575803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.583675 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3437/6917] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:46 2022] DELETE /identity/v3/projects/dbfb9f03e6de43a6bd5e7a6641ac8088 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:46.585088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:46.589595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:46.732847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ebd09-ec5b-4877-a15d-103c3be6eb6d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:46.745079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:46.746322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:46.746624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:46.747046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] PATH_INFO: `/v3/users/c7b28035ce8c46969624d1d390741807` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:46.749038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7b28035ce8c46969624d1d390741807)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:46.752345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:46.781199 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3481/6918] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:22:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:22:46.782464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:46.787008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:46.846547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:46.871435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7b28035ce8c46969624d1d390741807'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.873684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7b28035ce8c46969624d1d390741807'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.898703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7b28035ce8c46969624d1d390741807'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.924354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7b28035ce8c46969624d1d390741807'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:46.994091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7b28035ce8c46969624d1d390741807'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:47.013714 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3438/6919] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:46 2022] DELETE /identity/v3/users/c7b28035ce8c46969624d1d390741807 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:47.026083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2482-418e-4467-950f-d4b5f144d2c9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:47.170968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3e92e-79ec-4add-a149-82c7c250f077 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:47.172043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3e92e-79ec-4add-a149-82c7c250f077 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:47.172536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3e92e-79ec-4add-a149-82c7c250f077 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:47.173092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3e92e-79ec-4add-a149-82c7c250f077 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:47.175212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4ff3e92e-79ec-4add-a149-82c7c250f077 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:47.291952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3e92e-79ec-4add-a149-82c7c250f077 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:47.294778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3e92e-79ec-4add-a149-82c7c250f077 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:47.306497 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3482/6920] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:22:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:49.355371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:49.356855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:49.468946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:49.509015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b626d15-d7c8-4880-bf51-d9ebbc8b4033 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:49.518648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:49.519315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:49.519597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:49.519908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] PATH_INFO: `/v3/projects/954717fecc8142178eb36ceb49b864ee` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:49.525513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=954717fecc8142178eb36ceb49b864ee)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:49.527605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:49.650956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '954717fecc8142178eb36ceb49b864ee', 'request_id': 'req-1c6246c0-8473-420d-92ab-f242311fdfdb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:49.665491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '954717fecc8142178eb36ceb49b864ee', 'request_id': 'req-1c6246c0-8473-420d-92ab-f242311fdfdb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:49.698292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '954717fecc8142178eb36ceb49b864ee', 'request_id': 'req-1c6246c0-8473-420d-92ab-f242311fdfdb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:49.720618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] The token cache is being invalidate because project 954717fecc8142178eb36ceb49b864ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:22:49.721188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:49.722017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:49.723884 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3439/6921] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:22:49 2022] DELETE /identity/v3/projects/954717fecc8142178eb36ceb49b864ee => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:52.183235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098d039-eddd-4b55-b63e-c0f14cd13f6b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:52.183721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098d039-eddd-4b55-b63e-c0f14cd13f6b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:52.184070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098d039-eddd-4b55-b63e-c0f14cd13f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:52.224743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b098d039-eddd-4b55-b63e-c0f14cd13f6b None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:52.226001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b098d039-eddd-4b55-b63e-c0f14cd13f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:52.375779 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3483/6922] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:22:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:22:52.400852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:52.403017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:52.648972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6246c0-8473-420d-92ab-f242311fdfdb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:52.660888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:52.661760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:52.662144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:52.662519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:52.685046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:52.686960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:52.706008 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3440/6923] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:22:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:22:53.839144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:53.839651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:53.839744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:53.886604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:53.887909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:53.916101 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3484/6924] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:53.924774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:53.942641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec642-f36a-4cad-976b-e9e61dea6a75 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:53.966227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:53.967913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:53.979660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:53.997064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e9975-cbb9-4e78-a405-5fc7b9252727 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:53.997716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9975-cbb9-4e78-a405-5fc7b9252727 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:54.001938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9975-cbb9-4e78-a405-5fc7b9252727 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:54.004408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9975-cbb9-4e78-a405-5fc7b9252727 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:54.035117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e9975-cbb9-4e78-a405-5fc7b9252727 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:54.041856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e9975-cbb9-4e78-a405-5fc7b9252727 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:54.072315 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3441/6925] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:22:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:54.270141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f31725-ed48-4ac2-bebe-6993e0ecf137 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:54.297003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:54.301581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:54.302081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:54.302497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:54.315655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:54.365549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:54.382160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:54.416768 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3485/6926] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:22:54.715322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc72fd-47df-4a4b-b5b1-21f127f8cd36 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:54.722366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc72fd-47df-4a4b-b5b1-21f127f8cd36 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:54.723071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc72fd-47df-4a4b-b5b1-21f127f8cd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:54.782413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1bfc72fd-47df-4a4b-b5b1-21f127f8cd36 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:54.789452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1bfc72fd-47df-4a4b-b5b1-21f127f8cd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:54.827771 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3442/6927] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:22:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:22:54.835266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:54.868690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d0553-4ee9-41a5-9aa3-281901d0b725 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:54.883117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:54.884225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:54.885756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:54.886341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:54.901331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:54.904499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:54.923131 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3486/6928] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:22:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:55.822371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:55.822942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:55.822942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:55.865145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:55.868834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:55.898002 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3443/6929] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:22:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:22:55.912438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:55.928892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8cfbc-1415-4c4f-b093-a06646e3f912 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:55.940942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bbf6d-aa94-418e-9fb1-4d5383197ca9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:55.946939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bbf6d-aa94-418e-9fb1-4d5383197ca9 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:55.947500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bbf6d-aa94-418e-9fb1-4d5383197ca9 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:55.947979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bbf6d-aa94-418e-9fb1-4d5383197ca9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:55.968199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1bbf6d-aa94-418e-9fb1-4d5383197ca9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:55.973717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1bbf6d-aa94-418e-9fb1-4d5383197ca9 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:55.995721 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3487/6930] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:22:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:59.543557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:22:59.545152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:22:59.652871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:59.688060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcd2f6-6b69-41b3-8d68-8877150e04d1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:22:59.709601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:22:59.710697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:59.711206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:59.711704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] PATH_INFO: `/v3/users/b652b17bea8942c4b4a30034c6c1f44c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:59.713304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b652b17bea8942c4b4a30034c6c1f44c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:22:59.715568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:22:59.830288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b652b17bea8942c4b4a30034c6c1f44c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:59.833441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b652b17bea8942c4b4a30034c6c1f44c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:59.844466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b652b17bea8942c4b4a30034c6c1f44c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:59.866181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b652b17bea8942c4b4a30034c6c1f44c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:59.930841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b652b17bea8942c4b4a30034c6c1f44c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:22:59.945986 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3444/6931] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:22:59 2022] DELETE /identity/v3/users/b652b17bea8942c4b4a30034c6c1f44c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:22:59.950803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:22:59.953008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:22:59.953008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:22:59.991466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:22:59.996498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:00.103674 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3488/6932] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:00.145634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:00.147625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:00.310458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e2fdc-d82c-466f-b701-df6b92e79bf6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:00.320237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa3792-8dba-4243-a334-1ca8dc2d224a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:00.323826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa3792-8dba-4243-a334-1ca8dc2d224a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:00.323826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa3792-8dba-4243-a334-1ca8dc2d224a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:00.324226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa3792-8dba-4243-a334-1ca8dc2d224a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:00.349156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa3792-8dba-4243-a334-1ca8dc2d224a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:00.355196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa3792-8dba-4243-a334-1ca8dc2d224a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:00.383034 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3445/6933] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:00.450935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:00.452431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:00.472844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c7d8a-f7b1-4ca0-9bc6-bce0e7dd5122 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:00.485101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:00.487681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:00.487971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:00.488250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] PATH_INFO: `/v3/users/404378dde74649c29604372480c0f97b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:00.495552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=404378dde74649c29604372480c0f97b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:00.498804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:00.645313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404378dde74649c29604372480c0f97b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:00.649201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404378dde74649c29604372480c0f97b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:00.673282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404378dde74649c29604372480c0f97b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:00.680420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:00.687602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404378dde74649c29604372480c0f97b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:00.689235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404378dde74649c29604372480c0f97b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:00.728447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3489/6934] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:00 2022] DELETE /identity/v3/users/404378dde74649c29604372480c0f97b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:00.733634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:00.735017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:00.735485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:00.772245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:00.773529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:00.909523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:00.943895 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3446/6935] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:01.003917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:01.006552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:01.159790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8503ceb-ebcd-4e5f-81ca-c81fb59069dd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:01.167355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb64592-15af-4a11-9ffa-b02d82dd04e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:01.168184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb64592-15af-4a11-9ffa-b02d82dd04e6 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:01.170096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb64592-15af-4a11-9ffa-b02d82dd04e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:01.170557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb64592-15af-4a11-9ffa-b02d82dd04e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:01.196346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb64592-15af-4a11-9ffa-b02d82dd04e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:01.198418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb64592-15af-4a11-9ffa-b02d82dd04e6 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:01.224383 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3490/6936] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:01.625398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:01.628242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:01.663069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39814160-bfd8-4d20-80c6-a483d49baa7b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:01.691062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:01.697562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:01.698198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:01.702631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] PATH_INFO: `/v3/projects/ab5aec6b80a447bab8eb4f1adf06e61f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:01.708935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5aec6b80a447bab8eb4f1adf06e61f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:01.719268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:01.852852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'request_id': 'req-30aec12b-c38e-44b5-910c-ade2a1285831'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:01.866123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'request_id': 'req-30aec12b-c38e-44b5-910c-ade2a1285831'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:01.897170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'request_id': 'req-30aec12b-c38e-44b5-910c-ade2a1285831'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:01.926252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] The token cache is being invalidate because project ab5aec6b80a447bab8eb4f1adf06e61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:01.927090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:01.927882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-30aec12b-c38e-44b5-910c-ade2a1285831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:01.929685 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3447/6937] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:01 2022] DELETE /identity/v3/projects/ab5aec6b80a447bab8eb4f1adf06e61f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:01.934958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688a51e-fcfc-45f6-bbf3-c1f57fe9c1ba None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:01.935662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688a51e-fcfc-45f6-bbf3-c1f57fe9c1ba None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:01.936042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688a51e-fcfc-45f6-bbf3-c1f57fe9c1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:01.978744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a688a51e-fcfc-45f6-bbf3-c1f57fe9c1ba None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:01.985801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a688a51e-fcfc-45f6-bbf3-c1f57fe9c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:02.137520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3491/6938] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:02.145317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8842c-2764-44ad-8938-876da6b0f5f0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:02.145997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8842c-2764-44ad-8938-876da6b0f5f0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:02.146491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8842c-2764-44ad-8938-876da6b0f5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:02.190586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b4a8842c-2764-44ad-8938-876da6b0f5f0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:02.192438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4a8842c-2764-44ad-8938-876da6b0f5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:02.210041 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3448/6939] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:02.214417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688a51e-fcfc-45f6-bbf3-c1f57fe9c1ba None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.228428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688a51e-fcfc-45f6-bbf3-c1f57fe9c1ba None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:02.242219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:02.243230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:02.243728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:02.244215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:02.245276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:02.250573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:02.253803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:02.287741 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3492/6940] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:23:02.292317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8842c-2764-44ad-8938-876da6b0f5f0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.311442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8842c-2764-44ad-8938-876da6b0f5f0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:02.319063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:02.320140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:02.320634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:02.321657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:02.326904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:02.329299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:02.397021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3449/6941] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:02 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:02.398239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.414521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5202de-7264-4d25-ad2a-54772ac779a0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:02.516271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.519104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:02.592078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:02.617299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:02.619665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:02.620192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:02.622302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:02.628503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:02.659852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:02.684962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3493/6942] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:02 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:02.687825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.712492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd0871-ee24-45a9-a792-7450a6cedc58 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:02.730589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:02.733885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:02.734385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:02.734818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:02.735709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:02.736144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6648f75-6b7e-45c2-8f06-e29d3ac65b92 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:02.738011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:02.749012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:02.750002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:02.751998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:02.752477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] PATH_INFO: `/v3/projects/05ee5f7ac3e0456ea3f3c84fa5c33266` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:02.754160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=05ee5f7ac3e0456ea3f3c84fa5c33266)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:02.760393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:02.768221 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3494/6943] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:02.782161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.806965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b612b-12d5-4e61-a09d-057310d10091 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:02.821849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:02.823959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:02.824356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:02.824642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:02.834115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:02.840711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:02.869914 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3495/6944] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:02.880827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.891106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05ee5f7ac3e0456ea3f3c84fa5c33266', 'request_id': 'req-386870fb-db88-4d68-9b41-63d9c65b9fee'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:02.906546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6947e72-7b85-4233-908e-ffdcf54b5d4b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:02.915483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05ee5f7ac3e0456ea3f3c84fa5c33266', 'request_id': 'req-386870fb-db88-4d68-9b41-63d9c65b9fee'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:02.920215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:02.935563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05ee5f7ac3e0456ea3f3c84fa5c33266', 'request_id': 'req-386870fb-db88-4d68-9b41-63d9c65b9fee'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:02.969195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] The token cache is being invalidate because project 05ee5f7ac3e0456ea3f3c84fa5c33266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:02.970255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:02.972677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:02.978102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3450/6945] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:02 2022] DELETE /identity/v3/projects/05ee5f7ac3e0456ea3f3c84fa5c33266 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:02.981958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:02.983721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:03.102540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac15d0-3ce8-47c6-8c0e-b5fd5d0bfc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:03.103650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac15d0-3ce8-47c6-8c0e-b5fd5d0bfc62 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:03.105245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac15d0-3ce8-47c6-8c0e-b5fd5d0bfc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:03.106495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac15d0-3ce8-47c6-8c0e-b5fd5d0bfc62 admin admin] PATH_INFO: `/v3/projects/1ef0c41846b645dcbfa9eef951b0b49e/users/b20edf0a7d764a85be532e5b07b5547c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:03.137045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386870fb-db88-4d68-9b41-63d9c65b9fee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:03.143646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac15d0-3ce8-47c6-8c0e-b5fd5d0bfc62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef0c41846b645dcbfa9eef951b0b49e, user_id=b20edf0a7d764a85be532e5b07b5547c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:03.147951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac15d0-3ce8-47c6-8c0e-b5fd5d0bfc62 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:03.151130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:03.153812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:03.154426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:03.154745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] PATH_INFO: `/v3/users/51d9a02f69ed4d5cb3368407284dc1cd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:03.159071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d9a02f69ed4d5cb3368407284dc1cd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:03.164905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:03.171872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3496/6946] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:02 2022] PUT /identity/v3/projects/1ef0c41846b645dcbfa9eef951b0b49e/users/b20edf0a7d764a85be532e5b07b5547c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:03.179177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:03.179848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:03.180368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:03.250678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d9a02f69ed4d5cb3368407284dc1cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:03.251413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d9a02f69ed4d5cb3368407284dc1cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:03.261353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d9a02f69ed4d5cb3368407284dc1cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:03.281652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d9a02f69ed4d5cb3368407284dc1cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:03.307614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:03.311277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:03.318993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d9a02f69ed4d5cb3368407284dc1cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:03.339696 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3451/6947] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:02 2022] DELETE /identity/v3/users/51d9a02f69ed4d5cb3368407284dc1cd => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:03.454756 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3497/6948] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:03.469357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:03.472156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:03.603681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d5d30-fcf4-4938-a981-f04d44de53e8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:03.607432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a7702-9a83-48b0-a233-7f2392ab6998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:03.608144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7702-9a83-48b0-a233-7f2392ab6998 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:03.608441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7702-9a83-48b0-a233-7f2392ab6998 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:03.608879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7702-9a83-48b0-a233-7f2392ab6998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:03.635388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:03.732670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a7702-9a83-48b0-a233-7f2392ab6998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:03.734657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a7702-9a83-48b0-a233-7f2392ab6998 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:03.772268 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3452/6949] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:05.480974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:05.483668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:05.517476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea077f7b-df5e-4fdd-82a5-3c8589f1cfd8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:05.531840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:05.532678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:05.533152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:05.537921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] PATH_INFO: `/v3/projects/526c4d6a63004031baef7e7b321a31ac` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:05.537921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=526c4d6a63004031baef7e7b321a31ac)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:05.537921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:05.678573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526c4d6a63004031baef7e7b321a31ac', 'request_id': 'req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:05.690739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:05.710850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526c4d6a63004031baef7e7b321a31ac', 'request_id': 'req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:05.721814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526c4d6a63004031baef7e7b321a31ac', 'request_id': 'req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:05.753783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] The token cache is being invalidate because project 526c4d6a63004031baef7e7b321a31ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:05.757179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:05.759076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-151f80cb-d2fb-45f7-ae7e-96858e5fc0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:05.762722 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3498/6950] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:05 2022] DELETE /identity/v3/projects/526c4d6a63004031baef7e7b321a31ac => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:05.768040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac1db9-8826-4a0a-9646-adb520194b21 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:05.768506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac1db9-8826-4a0a-9646-adb520194b21 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:05.768873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac1db9-8826-4a0a-9646-adb520194b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:05.808984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3aac1db9-8826-4a0a-9646-adb520194b21 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:05.813282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3aac1db9-8826-4a0a-9646-adb520194b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:05.946923 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3453/6951] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:05.954439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f70ea-7486-4ef5-8fa5-4c1280b7e83d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:05.954876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f70ea-7486-4ef5-8fa5-4c1280b7e83d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:05.955258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f70ea-7486-4ef5-8fa5-4c1280b7e83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:05.999974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e09f70ea-7486-4ef5-8fa5-4c1280b7e83d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:06.001415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e09f70ea-7486-4ef5-8fa5-4c1280b7e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:06.024482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac1db9-8826-4a0a-9646-adb520194b21 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:06.028800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3499/6952] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:06.040693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac1db9-8826-4a0a-9646-adb520194b21 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:06.058071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:06.059953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.060415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.060867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:06.061922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:06.063676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:06.065690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:06.100867 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3454/6953] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:23:06.101779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f70ea-7486-4ef5-8fa5-4c1280b7e83d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:06.116590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f70ea-7486-4ef5-8fa5-4c1280b7e83d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:06.139172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:06.140006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.140805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.142265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:06.143664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:06.156821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:06.233348 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3500/6954] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:06 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:06.237449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:06.250334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afac73-42d7-43af-ba9e-6cbe879d749b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:06.419524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:06.420271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.420560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.420877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:06.421520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:06.423838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:06.484852 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3455/6955] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:06 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:06.489916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:06.509821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9e152-c68d-45fa-920b-b17dcaa00266 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:06.527733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:06.530725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.531646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.532755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:06.534242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:06.540084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:06.593372 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3501/6956] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:06.593951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:06.639629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c1f8d-0835-4e9e-ba42-b4decad41061 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:06.651301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bae6a-c0a9-4a6d-8ab9-28119e108352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:06.651943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bae6a-c0a9-4a6d-8ab9-28119e108352 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.652211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bae6a-c0a9-4a6d-8ab9-28119e108352 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.652478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bae6a-c0a9-4a6d-8ab9-28119e108352 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:06.653408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bae6a-c0a9-4a6d-8ab9-28119e108352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:06.654845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bae6a-c0a9-4a6d-8ab9-28119e108352 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:06.697404 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3456/6957] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:06.698434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:06.713196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b8225-8856-4967-940a-ed18a31e40fd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:06.735166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cb883-bbd9-4d81-81ac-aaecc84566de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:06.736635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb883-bbd9-4d81-81ac-aaecc84566de admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.737176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb883-bbd9-4d81-81ac-aaecc84566de admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.737690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb883-bbd9-4d81-81ac-aaecc84566de admin admin] PATH_INFO: `/v3/projects/c212c50129dd4fd9a8f7c6cb346467ca/users/317b6ce53c1f45c590ad314b862b88dc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:06.769667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cb883-bbd9-4d81-81ac-aaecc84566de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c212c50129dd4fd9a8f7c6cb346467ca, user_id=317b6ce53c1f45c590ad314b862b88dc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:06.774945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cb883-bbd9-4d81-81ac-aaecc84566de admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:06.817168 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3502/6958] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:06 2022] PUT /identity/v3/projects/c212c50129dd4fd9a8f7c6cb346467ca/users/317b6ce53c1f45c590ad314b862b88dc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:06.827336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.827899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.827899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:06.905513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] MFA Rules not processed for user `317b6ce53c1f45c590ad314b862b88dc`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:06.907444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:06.924595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:06.983696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:06.983973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:06.984247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:07.045097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] MFA Rules not processed for user `37715b55759e4187b56197b342a84947`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:07.047845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:07.070302 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3457/6959] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:07.120296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:07.122501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:07.259283 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3503/6960] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:23:07.260459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e70f7-08cc-4c34-9011-031218bc08bd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:07.270786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88c5af-2bb3-4147-b27d-828304b908eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:07.271709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88c5af-2bb3-4147-b27d-828304b908eb service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:07.272133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88c5af-2bb3-4147-b27d-828304b908eb service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:07.272558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88c5af-2bb3-4147-b27d-828304b908eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:07.274787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:07.277030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:07.293076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88c5af-2bb3-4147-b27d-828304b908eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:07.294812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88c5af-2bb3-4147-b27d-828304b908eb service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:07.314925 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3458/6961] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:23:07.395993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:07.416211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8c91e-8994-4db7-ba04-9c51f74bd4e4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:07.423895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:07.425564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:07.426102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:07.427026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:07.429369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:07.607292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:07.612471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:07.651519 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3504/6962] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:08.217046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fccfd-6129-4a7a-84b5-9eec5ef49944 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:08.217922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fccfd-6129-4a7a-84b5-9eec5ef49944 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:08.218220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fccfd-6129-4a7a-84b5-9eec5ef49944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:08.292780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c99fccfd-6129-4a7a-84b5-9eec5ef49944 None None] MFA Rules not processed for user `74878b715fc94b23bb139cb30d8d20c0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:08.296397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c99fccfd-6129-4a7a-84b5-9eec5ef49944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:08.477508 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3459/6963] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:09.713455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:09.859903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc30d-3d99-4326-9039-d4217fd3de42 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:09.867294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:09.868300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:09.868896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:09.869387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] PATH_INFO: `/v3/users/317b6ce53c1f45c590ad314b862b88dc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:09.877603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=317b6ce53c1f45c590ad314b862b88dc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:09.880032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:09.974478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '317b6ce53c1f45c590ad314b862b88dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:09.975723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '317b6ce53c1f45c590ad314b862b88dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:09.991760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '317b6ce53c1f45c590ad314b862b88dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:10.010403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '317b6ce53c1f45c590ad314b862b88dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:10.012739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '317b6ce53c1f45c590ad314b862b88dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:10.050780 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3505/6964] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:09 2022] DELETE /identity/v3/users/317b6ce53c1f45c590ad314b862b88dc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:10.055403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331bb20-d28b-43dd-b94a-603c4aca7122 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:10.055956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331bb20-d28b-43dd-b94a-603c4aca7122 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:10.055956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331bb20-d28b-43dd-b94a-603c4aca7122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:10.092518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0331bb20-d28b-43dd-b94a-603c4aca7122 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:10.098991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0331bb20-d28b-43dd-b94a-603c4aca7122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:10.190423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:10.223710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3460/6965] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:10.268925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:10.270711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:10.311755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:10.364561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcb572-6b25-437a-b486-15b167dd3057 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:10.368690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:10.369408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:10.369782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:10.370171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:10.389069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:10.390524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:10.406734 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3506/6966] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:12.053717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10241d-3d02-4569-b71b-05bd3dbe684f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:12.054823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10241d-3d02-4569-b71b-05bd3dbe684f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:12.057332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10241d-3d02-4569-b71b-05bd3dbe684f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:12.087530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ff10241d-3d02-4569-b71b-05bd3dbe684f None None] MFA Rules not processed for user `dc394c36e4bb436fabbecada1990b41e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:12.091100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff10241d-3d02-4569-b71b-05bd3dbe684f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:12.219554 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3461/6967] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:12.736970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:12.740217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:12.764691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ca88-af61-427e-b429-21dcf2168195 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:12.780834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:12.781495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:12.781779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:12.782071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] PATH_INFO: `/v3/projects/c212c50129dd4fd9a8f7c6cb346467ca` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:12.783434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c212c50129dd4fd9a8f7c6cb346467ca)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:12.785461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:12.936743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c212c50129dd4fd9a8f7c6cb346467ca', 'request_id': 'req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:12.994793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c212c50129dd4fd9a8f7c6cb346467ca', 'request_id': 'req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:13.010165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c212c50129dd4fd9a8f7c6cb346467ca', 'request_id': 'req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:13.040986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] The token cache is being invalidate because project c212c50129dd4fd9a8f7c6cb346467ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:13.041411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:13.042476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bbf4c90b-fa82-469a-87de-7331dc5b9b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:13.052076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3507/6968] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:12 2022] DELETE /identity/v3/projects/c212c50129dd4fd9a8f7c6cb346467ca => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:13.062360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a80bd-bf5f-4318-99c6-ecbee8f930e6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:13.062897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a80bd-bf5f-4318-99c6-ecbee8f930e6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:13.062897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a80bd-bf5f-4318-99c6-ecbee8f930e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:13.120934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-693a80bd-bf5f-4318-99c6-ecbee8f930e6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:13.122441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-693a80bd-bf5f-4318-99c6-ecbee8f930e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:13.416889 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3462/6969] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:13.427440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fa7d9-e26c-4d4a-9eb0-1c23f0449b58 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:13.427440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fa7d9-e26c-4d4a-9eb0-1c23f0449b58 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:13.427440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fa7d9-e26c-4d4a-9eb0-1c23f0449b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:13.519796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6d5fa7d9-e26c-4d4a-9eb0-1c23f0449b58 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:13.525017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6d5fa7d9-e26c-4d4a-9eb0-1c23f0449b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:13.559833 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3508/6970] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:13.561612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a80bd-bf5f-4318-99c6-ecbee8f930e6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:13.573290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a80bd-bf5f-4318-99c6-ecbee8f930e6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:13.591744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:13.592396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:13.597329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:13.597766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:13.598559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:13.598875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:13.605466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:13.658255 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3463/6971] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:23:13.659470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fa7d9-e26c-4d4a-9eb0-1c23f0449b58 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:13.680992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fa7d9-e26c-4d4a-9eb0-1c23f0449b58 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:13.690792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:13.692532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:13.693093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:13.694099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:13.695011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:13.697242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:13.773395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3509/6972] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:13 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:13.777835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:13.795811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65909bea-40a2-45e6-8a11-43a018330b3d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:13.920266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:13.920985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:13.921452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:13.921740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:13.922383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:13.929354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:13.976056 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3464/6973] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:13 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:23:13.980746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:13.997689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960d0c4-987c-4d68-9be3-15a0b4f806bc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:14.027903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:14.028887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:14.029376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:14.029850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:14.030774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:14.033814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:14.083634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:14.092489 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3510/6974] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:14.102991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a24259-c03e-4e3c-8ee0-e99eb2bd1e68 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:14.110205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a7410-fec9-4858-9993-ec8fedff8aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:14.111643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a7410-fec9-4858-9993-ec8fedff8aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:14.113554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a7410-fec9-4858-9993-ec8fedff8aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:14.114470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a7410-fec9-4858-9993-ec8fedff8aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:14.117571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a7410-fec9-4858-9993-ec8fedff8aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:14.120084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a7410-fec9-4858-9993-ec8fedff8aa2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:14.123129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:14.142013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:14.150279 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3465/6975] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:14.160435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc9efb-7d77-4193-8107-ab176d7ab4f5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:14.177163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:14.178209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:14.178752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:14.179316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] PATH_INFO: `/v3/projects/cf32566466c440cf9dcfd1692db41da4/users/04f6acf6efbb42cb9fb9b18c0d08b83a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:14.218924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf32566466c440cf9dcfd1692db41da4, user_id=04f6acf6efbb42cb9fb9b18c0d08b83a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:14.226851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:14.271145 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3511/6976] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:14 2022] PUT /identity/v3/projects/cf32566466c440cf9dcfd1692db41da4/users/04f6acf6efbb42cb9fb9b18c0d08b83a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:14.272881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805b63-e96d-4697-8054-b470b09a940e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:14.272999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805b63-e96d-4697-8054-b470b09a940e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:14.273283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805b63-e96d-4697-8054-b470b09a940e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:14.337186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-53805b63-e96d-4697-8054-b470b09a940e None None] MFA Rules not processed for user `04f6acf6efbb42cb9fb9b18c0d08b83a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:14.338438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-53805b63-e96d-4697-8054-b470b09a940e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:14.530681 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3466/6977] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:14.644278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:14.646915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:14.772509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:14.803908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48611f97-eee7-447f-b863-4f6f15c66353 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:14.810587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:14.811361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:14.811744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:14.812119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:14.837854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:14.837854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:14.864209 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3512/6978] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:22.396419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35ae83-5c0e-4caa-9ccf-4eeae34db65a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:22.397252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35ae83-5c0e-4caa-9ccf-4eeae34db65a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:22.397628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35ae83-5c0e-4caa-9ccf-4eeae34db65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:22.442293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3d35ae83-5c0e-4caa-9ccf-4eeae34db65a None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:22.444006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3d35ae83-5c0e-4caa-9ccf-4eeae34db65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:22.513006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3467/6979] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:23:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:22.531056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:22.533296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:22.701523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637345c-68c6-485f-960d-2c4efb6ad5de service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:22.709789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:22.711538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:22.712338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:22.712905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:22.736765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:22.739504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:22.749781 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3513/6980] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:23:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:23:23.195132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f88b62-7e15-41f5-89e9-c650829ae5f8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:23.195514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f88b62-7e15-41f5-89e9-c650829ae5f8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:23.195891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f88b62-7e15-41f5-89e9-c650829ae5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:23.260918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e9f88b62-7e15-41f5-89e9-c650829ae5f8 None None] MFA Rules not processed for user `b20edf0a7d764a85be532e5b07b5547c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:23.262080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e9f88b62-7e15-41f5-89e9-c650829ae5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:23.434090 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3468/6981] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:23.438712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:23.440439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:23.606095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25035b92-b1ef-4b24-962b-9092a629274f service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:23.618588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:23.619968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:23.621991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:23.622448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:23.625104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:23.636219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:23.687893 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3514/6982] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:23 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:23.691640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f88b62-7e15-41f5-89e9-c650829ae5f8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:23.714138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f88b62-7e15-41f5-89e9-c650829ae5f8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:23.729151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ceeeb-2f24-488e-aa74-90dec3217e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:23.734303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ceeeb-2f24-488e-aa74-90dec3217e41 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:23.736836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ceeeb-2f24-488e-aa74-90dec3217e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:23.737382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ceeeb-2f24-488e-aa74-90dec3217e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:23.738328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ceeeb-2f24-488e-aa74-90dec3217e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:23.742406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ceeeb-2f24-488e-aa74-90dec3217e41 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:23.779376 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3469/6983] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:23.784749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:23.800895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2e532-7d2a-4a03-b48d-3b89231963a1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:23.816691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:23.817902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:23.818324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:23.818738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] PATH_INFO: `/v3/system/users/f32b18b562264365ae736772a207d959/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:23.869019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f32b18b562264365ae736772a207d959, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:23.872567 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:23:23.872567 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:23:23.872567 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:23:23.876418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:23.911759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3515/6984] 10.222.0.26 () {60 vars in 1378 bytes} [Tue Sep 27 10:23:23 2022] PUT /identity/v3/system/users/f32b18b562264365ae736772a207d959/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:23.911995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cea56d-a234-495a-8f96-2408c0510bec None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:23.912387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cea56d-a234-495a-8f96-2408c0510bec None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:23.912799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cea56d-a234-495a-8f96-2408c0510bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:23.920228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:23.993025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-88cea56d-a234-495a-8f96-2408c0510bec None None] MFA Rules not processed for user `f32b18b562264365ae736772a207d959`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:23.998332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-88cea56d-a234-495a-8f96-2408c0510bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:24.205822 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3470/6985] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:23 2022] POST /identity/v3/auth/tokens => generated 2415 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 10:23:24.209594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:24.336791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:24.454111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5844886-e972-498c-bff4-509df2ad26e2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:24.504025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f32b18b562264365ae736772a207d959', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:24.505662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:24.505939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:24.506259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:24.507071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:24.508516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:24.517798 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3516/6986] 10.222.0.26 () {58 vars in 1110 bytes} [Tue Sep 27 10:23:24 2022] GET /identity/v3/services => generated 2191 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:23:24.522565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cea56d-a234-495a-8f96-2408c0510bec None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:24.821164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cea56d-a234-495a-8f96-2408c0510bec None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:24.895727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:24.919433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f32b18b562264365ae736772a207d959', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:24.920184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:24.920543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:24.920993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] PATH_INFO: `/v3/limits` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:24.922200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:24.924354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Sep 27 10:23:24.996032 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.NoLimitReference Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 27 10:23:24.997205 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:23:25.003455 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3471/6987] 10.222.0.26 () {60 vars in 1126 bytes} [Tue Sep 27 10:23:24 2022] POST /identity/v3/limits => generated 124 bytes in 481 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:23:25.007817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:25.288521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5fc1e-513f-4c80-bdba-f8beeb638015 None tempest-ImageQuotaTest-1728173280-system-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:25.294654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:25.367704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f32b18b562264365ae736772a207d959', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:25.368887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:25.369319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:25.371097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:25.371902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:25.373225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:25.390475 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3517/6988] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:23:25 2022] GET /identity/v3/registered_limits => generated 129 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:23:25.398697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:25.667999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f4e93-2dcc-4042-b0ff-ea6ad8e5235c None tempest-ImageQuotaTest-1728173280-system-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:25.748267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccc11f-0393-43fb-8962-46782cf17726 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f32b18b562264365ae736772a207d959', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:25.749408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccc11f-0393-43fb-8962-46782cf17726 None tempest-ImageQuotaTest-1728173280-system-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:25.749901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccc11f-0393-43fb-8962-46782cf17726 None tempest-ImageQuotaTest-1728173280-system-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:25.750400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccc11f-0393-43fb-8962-46782cf17726 None tempest-ImageQuotaTest-1728173280-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:25.751903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccc11f-0393-43fb-8962-46782cf17726 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:25.753359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccc11f-0393-43fb-8962-46782cf17726 None tempest-ImageQuotaTest-1728173280-system-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:25.763446 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3472/6989] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:23:25 2022] GET /identity/v3/registered_limits => generated 129 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:23:33.642551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:33.664524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230e44-56bd-4d14-8986-2d4eba9c92e1 None tempest-ImageQuotaTest-1728173280-system-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:33.682120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:33.682984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:33.685598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:33.685598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] PATH_INFO: `/v3/users/04f6acf6efbb42cb9fb9b18c0d08b83a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:33.688456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=04f6acf6efbb42cb9fb9b18c0d08b83a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:33.690470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:33.922622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04f6acf6efbb42cb9fb9b18c0d08b83a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:33.923382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04f6acf6efbb42cb9fb9b18c0d08b83a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:33.942426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04f6acf6efbb42cb9fb9b18c0d08b83a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:33.963753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04f6acf6efbb42cb9fb9b18c0d08b83a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:33.964633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04f6acf6efbb42cb9fb9b18c0d08b83a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:34.005061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3518/6990] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:33 2022] DELETE /identity/v3/users/04f6acf6efbb42cb9fb9b18c0d08b83a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:34.009702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:34.010236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:34.010236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:34.046886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:34.048507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:34.131687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:34.169309 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3473/6991] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:34.251794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:34.254067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:34.398102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1a608-5ecb-49aa-8d2d-93da042339ae admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:34.409090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d83ac-8211-4f9e-9d13-f5f0ae4e8e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:34.409936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d83ac-8211-4f9e-9d13-f5f0ae4e8e95 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:34.410345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d83ac-8211-4f9e-9d13-f5f0ae4e8e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:34.410802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d83ac-8211-4f9e-9d13-f5f0ae4e8e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:34.441951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d83ac-8211-4f9e-9d13-f5f0ae4e8e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:34.443848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d83ac-8211-4f9e-9d13-f5f0ae4e8e95 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:34.475315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3519/6992] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:35.709230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:35.713426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:35.739450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12ddb-ad0b-4bea-b96e-25cd7aaf417e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:35.756801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:35.757470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:35.757741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:35.758171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] PATH_INFO: `/v3/projects/cf32566466c440cf9dcfd1692db41da4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:35.760922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf32566466c440cf9dcfd1692db41da4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:35.763009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:35.876256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf32566466c440cf9dcfd1692db41da4', 'request_id': 'req-6a2eabba-2287-4a7c-8640-9309d8995e03'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:35.896050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf32566466c440cf9dcfd1692db41da4', 'request_id': 'req-6a2eabba-2287-4a7c-8640-9309d8995e03'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:35.914580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf32566466c440cf9dcfd1692db41da4', 'request_id': 'req-6a2eabba-2287-4a7c-8640-9309d8995e03'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:35.944013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] The token cache is being invalidate because project cf32566466c440cf9dcfd1692db41da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:35.944538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:35.945500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a2eabba-2287-4a7c-8640-9309d8995e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:35.947821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3474/6993] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:35 2022] DELETE /identity/v3/projects/cf32566466c440cf9dcfd1692db41da4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:35.954132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389bc6a-7bce-4f81-b76e-9d073e2188e9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:35.955288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389bc6a-7bce-4f81-b76e-9d073e2188e9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:35.955288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389bc6a-7bce-4f81-b76e-9d073e2188e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:35.995079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3389bc6a-7bce-4f81-b76e-9d073e2188e9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:35.999381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3389bc6a-7bce-4f81-b76e-9d073e2188e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:36.056282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:36.118778 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3520/6994] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:36.121136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425f5fb-a083-42ae-a1c6-1552cf4fdc51 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.121451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425f5fb-a083-42ae-a1c6-1552cf4fdc51 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.121752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425f5fb-a083-42ae-a1c6-1552cf4fdc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:36.188546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d425f5fb-a083-42ae-a1c6-1552cf4fdc51 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:36.190790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d425f5fb-a083-42ae-a1c6-1552cf4fdc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:36.213930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3475/6995] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:36.218393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389bc6a-7bce-4f81-b76e-9d073e2188e9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:36.233233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389bc6a-7bce-4f81-b76e-9d073e2188e9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:36.250062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:36.251552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.251552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.251552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:36.252321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:36.252994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:36.254940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:36.292758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3521/6996] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:23:36.297622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425f5fb-a083-42ae-a1c6-1552cf4fdc51 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:36.313225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425f5fb-a083-42ae-a1c6-1552cf4fdc51 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:36.327681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:36.328471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.329377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.329666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:36.330325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:36.332439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:36.410082 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3476/6997] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:36 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:23:36.414465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:36.430192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa911-a555-4f5b-9c17-175635a3facc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:36.570480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:36.571235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.571584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.575250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:36.576636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:36.588948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:36.684905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:36.695606 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3522/6998] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:36 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:23:36.704515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade1078-e00f-44f8-882f-6b94b459dbe9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:36.720319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:36.721056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.721292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.721564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:36.722237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:36.724004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:36.756980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:36.771468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaba015-aeda-4332-a8be-c9848196d26a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:36.772470 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3477/6999] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:36.785231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835b3c2-67ac-4ab4-abae-e83257d52caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:36.785967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b3c2-67ac-4ab4-abae-e83257d52caf admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.786417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b3c2-67ac-4ab4-abae-e83257d52caf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.786702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b3c2-67ac-4ab4-abae-e83257d52caf admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:36.787360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835b3c2-67ac-4ab4-abae-e83257d52caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:36.789032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835b3c2-67ac-4ab4-abae-e83257d52caf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:36.821375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3523/7000] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:36.825794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:36.840562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b316203-acc5-4210-bcfd-d027198c4508 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:36.856922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:36.857734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.864395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.864725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] PATH_INFO: `/v3/projects/2cbdfce5a2614714995c598907d4aa69/users/05f2af6358034ac6ab709b3a22d1bfbf/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:36.904716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbdfce5a2614714995c598907d4aa69, user_id=05f2af6358034ac6ab709b3a22d1bfbf, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:36.914508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:36.956282 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3478/7001] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:36 2022] PUT /identity/v3/projects/2cbdfce5a2614714995c598907d4aa69/users/05f2af6358034ac6ab709b3a22d1bfbf/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:36.968381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07307e-a0a9-485a-b047-4b05c43ec9f7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:36.969547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07307e-a0a9-485a-b047-4b05c43ec9f7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:36.970087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07307e-a0a9-485a-b047-4b05c43ec9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:37.075017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5b07307e-a0a9-485a-b047-4b05c43ec9f7 None None] MFA Rules not processed for user `05f2af6358034ac6ab709b3a22d1bfbf`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:37.096751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5b07307e-a0a9-485a-b047-4b05c43ec9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:37.255102 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3524/7002] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:37.269073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:37.271549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:37.459190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cc8a6-4ad0-4e2f-9109-fcb43cf85af4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:37.466140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eff1a-1f7c-4f98-b9c7-d6325c605bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:37.466795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eff1a-1f7c-4f98-b9c7-d6325c605bde service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:37.467537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eff1a-1f7c-4f98-b9c7-d6325c605bde service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:37.467822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eff1a-1f7c-4f98-b9c7-d6325c605bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:37.478766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eff1a-1f7c-4f98-b9c7-d6325c605bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:37.482194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eff1a-1f7c-4f98-b9c7-d6325c605bde service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:37.496042 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3479/7003] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:38.445261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07307e-a0a9-485a-b047-4b05c43ec9f7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:38.574257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:38.720121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07307e-a0a9-485a-b047-4b05c43ec9f7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:38.757259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:38.758323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:38.761356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:38.769646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] PATH_INFO: `/v3/users/05f2af6358034ac6ab709b3a22d1bfbf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:38.778079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05f2af6358034ac6ab709b3a22d1bfbf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:38.786183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:38.987844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05f2af6358034ac6ab709b3a22d1bfbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:38.989028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05f2af6358034ac6ab709b3a22d1bfbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:39.032471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05f2af6358034ac6ab709b3a22d1bfbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:39.056646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05f2af6358034ac6ab709b3a22d1bfbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:39.062607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05f2af6358034ac6ab709b3a22d1bfbf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:39.125133 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3525/7004] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:38 2022] DELETE /identity/v3/users/05f2af6358034ac6ab709b3a22d1bfbf => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:39.129706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:39.130225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:39.130225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:39.167344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:39.168216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:39.178585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:39.334939 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3480/7005] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:39.382076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:39.383878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:39.529249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6771ab-b7bb-4bb2-b8f5-1de78acb0b62 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:39.536337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:39.537474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:39.537992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:39.538633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:39.552762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:39.555439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:39.584628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3526/7006] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:23:41.085130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:41.087285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:41.113613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36187d96-4c3d-4c5c-aa3e-5387375a0616 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:41.128705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:41.130015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:41.131406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:41.131905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] PATH_INFO: `/v3/users/b20edf0a7d764a85be532e5b07b5547c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:41.136584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20edf0a7d764a85be532e5b07b5547c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:41.140134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:41.282675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20edf0a7d764a85be532e5b07b5547c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.283605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20edf0a7d764a85be532e5b07b5547c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.289446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:41.293227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:41.306943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20edf0a7d764a85be532e5b07b5547c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.316603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103ab52-8b24-4a63-8d4f-cf38e9629f1f service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:41.332363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20edf0a7d764a85be532e5b07b5547c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.336589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:41.337774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:41.338276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:41.338851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] PATH_INFO: `/v3/projects/2cbdfce5a2614714995c598907d4aa69` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:41.342886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbdfce5a2614714995c598907d4aa69)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:41.345896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:41.367585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-49d19fa7-5ce6-427d-aab2-77bf518499af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20edf0a7d764a85be532e5b07b5547c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.396577 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3481/7007] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:41 2022] DELETE /identity/v3/users/b20edf0a7d764a85be532e5b07b5547c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:41.461312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbdfce5a2614714995c598907d4aa69', 'request_id': 'req-049df286-8a74-4bbd-866f-35a1b6c9245b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.463239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:41.482278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbdfce5a2614714995c598907d4aa69', 'request_id': 'req-049df286-8a74-4bbd-866f-35a1b6c9245b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.495109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbdfce5a2614714995c598907d4aa69', 'request_id': 'req-049df286-8a74-4bbd-866f-35a1b6c9245b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.519262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] The token cache is being invalidate because project 2cbdfce5a2614714995c598907d4aa69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:41.519644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.520250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-049df286-8a74-4bbd-866f-35a1b6c9245b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:41.525115 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3527/7008] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:41 2022] DELETE /identity/v3/projects/2cbdfce5a2614714995c598907d4aa69 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:41.534362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb81a5-56b2-48e8-89d0-bd4cab634d46 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:41.534649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb81a5-56b2-48e8-89d0-bd4cab634d46 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:41.534931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb81a5-56b2-48e8-89d0-bd4cab634d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:41.583919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-18cb81a5-56b2-48e8-89d0-bd4cab634d46 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:41.585213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-18cb81a5-56b2-48e8-89d0-bd4cab634d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:41.770970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac632f-bf28-4d60-ac43-4f290b3d36e3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:41.770970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac632f-bf28-4d60-ac43-4f290b3d36e3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:41.771189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac632f-bf28-4d60-ac43-4f290b3d36e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:41.774992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3482/7009] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:41.799321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3cac632f-bf28-4d60-ac43-4f290b3d36e3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:41.800758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3cac632f-bf28-4d60-ac43-4f290b3d36e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:41.810853 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3528/7010] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:41.815065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb81a5-56b2-48e8-89d0-bd4cab634d46 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:41.826865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb81a5-56b2-48e8-89d0-bd4cab634d46 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:41.831881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:41.832684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:41.833003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:41.833615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:41.834283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:41.834584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:41.836826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:41.858426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3483/7011] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:23:41.863211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac632f-bf28-4d60-ac43-4f290b3d36e3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:41.877936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac632f-bf28-4d60-ac43-4f290b3d36e3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:41.894273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:41.894930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:41.895291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:41.895564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:41.896403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:41.898622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:41.977738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:41.985294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3529/7012] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:41 2022] POST /identity/v3/projects => generated 351 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:41.993336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23656eb8-c38f-46b5-8924-69180c454f5d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:41.999369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:42.180275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:42.181014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:42.181204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:42.181487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:42.182186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:42.184306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:42.252607 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3484/7013] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:41 2022] POST /identity/v3/users => generated 276 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:42.257213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:42.281905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2981cc-430f-408e-8954-6507ad197b32 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:42.295490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:42.296960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:42.297385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:42.297837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:42.299361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:42.301181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:42.330436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:42.332176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3530/7014] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:42.354785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145afcf-debb-4463-a9a5-032782ba3f32 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:42.377806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:42.381016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:42.381338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:42.382977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:42.383682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:42.387879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:42.443554 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3485/7015] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:23:42.447800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:42.461410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a1c35-0c1a-468f-8bbe-ec5706b18845 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:42.479957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe9764-a0cd-4018-8ac5-111108ff88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:42.480989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe9764-a0cd-4018-8ac5-111108ff88f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:42.481434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe9764-a0cd-4018-8ac5-111108ff88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:42.481876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe9764-a0cd-4018-8ac5-111108ff88f5 admin admin] PATH_INFO: `/v3/projects/3f2965a6eebb49bebed0ccc9c2cc458f/users/0ca5d19fe78947a38ab6e79a050ecadb/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:42.534925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:42.537244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:42.561539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe9764-a0cd-4018-8ac5-111108ff88f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2965a6eebb49bebed0ccc9c2cc458f, user_id=0ca5d19fe78947a38ab6e79a050ecadb, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:42.565653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe9764-a0cd-4018-8ac5-111108ff88f5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:42.623045 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3531/7016] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:42 2022] PUT /identity/v3/projects/3f2965a6eebb49bebed0ccc9c2cc458f/users/0ca5d19fe78947a38ab6e79a050ecadb/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:42.624800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:42.628496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:42.631060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:42.658513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1c0d-3cf0-43e0-bfb8-38c990806f60 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:42.755953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] MFA Rules not processed for user `0ca5d19fe78947a38ab6e79a050ecadb`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:42.763798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:42.799317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:42.800462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:42.800945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:42.801462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:42.804448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:43.037019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:43.045668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:43.047175 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3532/7017] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:43.056683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:43.063237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:43.068022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:43.099192 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3486/7018] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:43.100126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:43.103689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:43.131988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0bb64-4bef-419b-b853-2395807709a3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:43.143509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:43.144145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:43.144432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:43.144701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:43.160692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:43.162155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:43.184097 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3487/7019] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:43.263676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740dfdc-53b9-486e-bbee-8d6e0369c5f0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:43.271305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:43.273285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:43.273951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:43.274447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] PATH_INFO: `/v3/projects/1ef0c41846b645dcbfa9eef951b0b49e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:43.275868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef0c41846b645dcbfa9eef951b0b49e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:43.286628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:43.298731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:43.431906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef0c41846b645dcbfa9eef951b0b49e', 'request_id': 'req-dd46e2c1-7890-415a-8648-7685274f834b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.463867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef0c41846b645dcbfa9eef951b0b49e', 'request_id': 'req-dd46e2c1-7890-415a-8648-7685274f834b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.478540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef0c41846b645dcbfa9eef951b0b49e', 'request_id': 'req-dd46e2c1-7890-415a-8648-7685274f834b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.507083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] The token cache is being invalidate because project 1ef0c41846b645dcbfa9eef951b0b49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:43.507640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.513525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-dd46e2c1-7890-415a-8648-7685274f834b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.516226 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3533/7020] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:43 2022] DELETE /identity/v3/projects/1ef0c41846b645dcbfa9eef951b0b49e => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:43.520151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:43.535212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54903f29-c8c9-4760-a5c5-f2f22b4b314b service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:43.732135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:43.733855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:43.734138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:43.734410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] PATH_INFO: `/v3/users/f32b18b562264365ae736772a207d959` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:43.738225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32b18b562264365ae736772a207d959)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:43.740519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:43.935044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32b18b562264365ae736772a207d959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.936284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32b18b562264365ae736772a207d959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.958927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32b18b562264365ae736772a207d959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:43.984689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32b18b562264365ae736772a207d959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:44.017345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-86755a11-89f4-4758-b169-dacb6aef6ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32b18b562264365ae736772a207d959'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:44.026764 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3488/7021] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:43 2022] DELETE /identity/v3/users/f32b18b562264365ae736772a207d959 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:44.032579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26740bc-0d00-473f-88d8-288987956312 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.033305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26740bc-0d00-473f-88d8-288987956312 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.033788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26740bc-0d00-473f-88d8-288987956312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.064816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f26740bc-0d00-473f-88d8-288987956312 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:44.066544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f26740bc-0d00-473f-88d8-288987956312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:44.191962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3534/7022] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:23:44.203451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f87ab-2122-41d7-b130-c9da0981fbdb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.206739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f87ab-2122-41d7-b130-c9da0981fbdb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.207609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f87ab-2122-41d7-b130-c9da0981fbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.302922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ef8f87ab-2122-41d7-b130-c9da0981fbdb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:44.307087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ef8f87ab-2122-41d7-b130-c9da0981fbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:44.354916 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3489/7023] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:44.359431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26740bc-0d00-473f-88d8-288987956312 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:44.375314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26740bc-0d00-473f-88d8-288987956312 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:44.383231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:44.384292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.385624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.386105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.387295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:44.387854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:44.390383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:44.416112 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3535/7024] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:23:44.421462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f87ab-2122-41d7-b130-c9da0981fbdb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:44.436164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f87ab-2122-41d7-b130-c9da0981fbdb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:44.452893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:44.454103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.454647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.455182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.456339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:44.459640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:44.530975 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3490/7025] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:44 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:23:44.534470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:44.550804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c3ee-56f5-4603-8520-b77ca0e3a4d1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:44.727705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:44.728634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.729186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.729684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.730588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:44.732845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:44.769595 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3536/7026] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:44 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:44.773680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:44.788400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5746c32-afba-46a7-a9ff-3ac29a65a102 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:44.794997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:44.796044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.796435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.796811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.797540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:44.799223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:44.812871 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3491/7027] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:44.817422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:44.832607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e73a1-07ae-44c1-925a-5548c8e6c14d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:44.843071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafde64b-d564-4987-96f2-88a016e6718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:44.845105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafde64b-d564-4987-96f2-88a016e6718f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.845613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafde64b-d564-4987-96f2-88a016e6718f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.846083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafde64b-d564-4987-96f2-88a016e6718f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.847012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafde64b-d564-4987-96f2-88a016e6718f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:44.849349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafde64b-d564-4987-96f2-88a016e6718f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:44.856816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:44.866650 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3537/7028] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:44.870793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:44.884896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15157b1-c435-4429-bf2d-05506e031655 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:44.893195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:44.894883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.897076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.897456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] PATH_INFO: `/v3/projects/64f8fe93425d4e97b1ae5661172f4846/users/178956f6c8154a06a0710b295aff3086/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:44.906420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:44.946540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f8fe93425d4e97b1ae5661172f4846, user_id=178956f6c8154a06a0710b295aff3086, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:44.952256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:44.984783 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3492/7029] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:44 2022] PUT /identity/v3/projects/64f8fe93425d4e97b1ae5661172f4846/users/178956f6c8154a06a0710b295aff3086/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:44.990288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:44.991081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:44.991598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:45.055864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:45.059497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:45.220404 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3538/7030] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:45.239798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:45.242193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:45.446299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22032e33-77dc-4f18-8f35-4f07557756be admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:45.456032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f82a03-7763-4d3b-8c4c-2eb6c7d62d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:45.456893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f82a03-7763-4d3b-8c4c-2eb6c7d62d06 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:45.457344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f82a03-7763-4d3b-8c4c-2eb6c7d62d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:45.457823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f82a03-7763-4d3b-8c4c-2eb6c7d62d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:45.505670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f82a03-7763-4d3b-8c4c-2eb6c7d62d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:45.507212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f82a03-7763-4d3b-8c4c-2eb6c7d62d06 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:45.536031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3493/7031] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:47.507512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:47.510343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:47.533015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58f7db-74b3-426a-86b5-cee28b00dc90 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:47.550258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:47.551147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:47.551668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:47.553071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] PATH_INFO: `/v3/users/0ca5d19fe78947a38ab6e79a050ecadb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:47.554518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca5d19fe78947a38ab6e79a050ecadb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:47.561059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:47.674968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca5d19fe78947a38ab6e79a050ecadb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:47.676473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca5d19fe78947a38ab6e79a050ecadb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:47.703589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca5d19fe78947a38ab6e79a050ecadb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:47.724556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca5d19fe78947a38ab6e79a050ecadb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:47.724726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca5d19fe78947a38ab6e79a050ecadb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:47.764987 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3539/7032] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:47 2022] DELETE /identity/v3/users/0ca5d19fe78947a38ab6e79a050ecadb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:47.769473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:47.770041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:47.770143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:47.808301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:47.813993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:47.959399 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3494/7033] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:47.979213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:47.980651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:48.093408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0045a89-de49-4f9b-a25b-b6d73492663b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:48.102243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88bff3-5ff7-43e2-80e2-aff8bbb509e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:48.104387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88bff3-5ff7-43e2-80e2-aff8bbb509e7 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.104696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88bff3-5ff7-43e2-80e2-aff8bbb509e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.106432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88bff3-5ff7-43e2-80e2-aff8bbb509e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:48.133599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88bff3-5ff7-43e2-80e2-aff8bbb509e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:48.138592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88bff3-5ff7-43e2-80e2-aff8bbb509e7 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:48.164089 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3540/7034] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:48.416758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:48.418212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:48.435949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95fc2b-2bae-4be6-8b67-cbf4279ffec6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:48.455954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:48.459253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.459253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.459253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] PATH_INFO: `/v3/projects/3f2965a6eebb49bebed0ccc9c2cc458f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:48.459253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2965a6eebb49bebed0ccc9c2cc458f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:48.463460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:48.586751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2965a6eebb49bebed0ccc9c2cc458f', 'request_id': 'req-94900325-1c04-411b-9c67-c21c3a4ef88c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:48.594157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2965a6eebb49bebed0ccc9c2cc458f', 'request_id': 'req-94900325-1c04-411b-9c67-c21c3a4ef88c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:48.608883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2965a6eebb49bebed0ccc9c2cc458f', 'request_id': 'req-94900325-1c04-411b-9c67-c21c3a4ef88c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:48.633033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] The token cache is being invalidate because project 3f2965a6eebb49bebed0ccc9c2cc458f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:48.633423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:48.634181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94900325-1c04-411b-9c67-c21c3a4ef88c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:48.642869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3495/7035] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:48 2022] DELETE /identity/v3/projects/3f2965a6eebb49bebed0ccc9c2cc458f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:48.643611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de340b6-fe3f-41a0-8431-e8bd404260ef None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.644072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de340b6-fe3f-41a0-8431-e8bd404260ef None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.644265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de340b6-fe3f-41a0-8431-e8bd404260ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:48.680426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7de340b6-fe3f-41a0-8431-e8bd404260ef None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:48.682436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7de340b6-fe3f-41a0-8431-e8bd404260ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:48.703276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:48.721490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.721983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.722480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:48.749310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:48.750119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:48.756948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:48.788524 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3541/7036] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:23:48.798355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0968bf-971b-43a1-a6ef-dd8fc772acca None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.798737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0968bf-971b-43a1-a6ef-dd8fc772acca None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.799109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0968bf-971b-43a1-a6ef-dd8fc772acca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:48.829329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fb0968bf-971b-43a1-a6ef-dd8fc772acca None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:48.830761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb0968bf-971b-43a1-a6ef-dd8fc772acca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:48.844887 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3496/7037] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:48.848671 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3542/7038] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:48.849917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0968bf-971b-43a1-a6ef-dd8fc772acca None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:48.853522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:48.859321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:48.864831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0968bf-971b-43a1-a6ef-dd8fc772acca None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:48.877439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:48.881096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.881379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.882090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:48.882310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:48.882615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:48.885893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:48.911364 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3543/7039] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:23:48.917740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:48.934876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43468-3bb1-4375-ac01-588fbea869a9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:48.943130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:48.944249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.944791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.945137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:48.946106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:48.948460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:48.987197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae96387-0f8a-45d8-a920-987eac502c50 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:48.992765 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3544/7040] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:48 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:48.996342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:48.997874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:48.998684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:48.999028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:48.999320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:49.016039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:49.022912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:49.055166 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3497/7041] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:23:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:23:49.124281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622ea6f-5181-4a5d-b05d-0d959b3950cb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:49.129577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:49.130465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:49.130465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:49.130743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:49.136214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:49.138412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:49.172977 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3545/7042] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:48 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:23:49.173775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:49.196048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7415b77-1b6b-43b1-94d2-edb6958dc359 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:49.201974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:49.202726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:49.203075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:49.203898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:49.204717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:49.206706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:49.223845 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3498/7043] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:49.230376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:49.259377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f6fb5-0762-4cd9-a5a2-2792492aac3e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:49.270350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ac48e-3bbe-414f-9d46-165b3e56ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:49.270998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ac48e-3bbe-414f-9d46-165b3e56ec75 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:49.271283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ac48e-3bbe-414f-9d46-165b3e56ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:49.271556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ac48e-3bbe-414f-9d46-165b3e56ec75 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:49.272341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ac48e-3bbe-414f-9d46-165b3e56ec75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:49.274478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ac48e-3bbe-414f-9d46-165b3e56ec75 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:49.300709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3546/7044] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:49.301426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:49.317177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efff8cf-384a-433c-9a68-84702dc0d06d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:49.331119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:49.332091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:49.332883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:49.333026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] PATH_INFO: `/v3/projects/eadfe59250434a348903cf3bcd079465/users/e9e5c74a1e004d2e9a2dde916c31a795/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:49.372023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eadfe59250434a348903cf3bcd079465, user_id=e9e5c74a1e004d2e9a2dde916c31a795, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:49.376091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:49.421735 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3499/7045] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:49 2022] PUT /identity/v3/projects/eadfe59250434a348903cf3bcd079465/users/e9e5c74a1e004d2e9a2dde916c31a795/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:49.423165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d0f3f-77e3-4668-9030-eb0817435311 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:49.423453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d0f3f-77e3-4668-9030-eb0817435311 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:49.423733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d0f3f-77e3-4668-9030-eb0817435311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:49.532007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf8d0f3f-77e3-4668-9030-eb0817435311 None None] MFA Rules not processed for user `e9e5c74a1e004d2e9a2dde916c31a795`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:49.536984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf8d0f3f-77e3-4668-9030-eb0817435311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:49.655361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:49.714693 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3547/7046] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:49.729547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:49.731590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:49.815013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331da9c-62d5-48c3-8456-8eefd6ad2155 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:49.820787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc066d4d-36df-4ebd-8fad-31b18a674d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:49.821748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc066d4d-36df-4ebd-8fad-31b18a674d8e service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:49.823233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc066d4d-36df-4ebd-8fad-31b18a674d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:49.823747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc066d4d-36df-4ebd-8fad-31b18a674d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:49.836099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc066d4d-36df-4ebd-8fad-31b18a674d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:49.838209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc066d4d-36df-4ebd-8fad-31b18a674d8e service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:49.851797 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3500/7047] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:49.906218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d0f3f-77e3-4668-9030-eb0817435311 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:50.065414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d0f3f-77e3-4668-9030-eb0817435311 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:50.078847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:50.079505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:50.079781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:50.080046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] PATH_INFO: `/v3/users/e9e5c74a1e004d2e9a2dde916c31a795` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:50.081854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9e5c74a1e004d2e9a2dde916c31a795)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:50.085649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:50.169617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9e5c74a1e004d2e9a2dde916c31a795'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:50.170693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9e5c74a1e004d2e9a2dde916c31a795'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:50.186940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9e5c74a1e004d2e9a2dde916c31a795'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:50.204286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9e5c74a1e004d2e9a2dde916c31a795'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:50.206384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9e5c74a1e004d2e9a2dde916c31a795'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:50.250966 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3548/7048] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:49 2022] DELETE /identity/v3/users/e9e5c74a1e004d2e9a2dde916c31a795 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:50.255619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a892878-1347-44f6-8457-4688fca44c62 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:50.255905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a892878-1347-44f6-8457-4688fca44c62 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:50.256120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a892878-1347-44f6-8457-4688fca44c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:50.299603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7a892878-1347-44f6-8457-4688fca44c62 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:50.305317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7a892878-1347-44f6-8457-4688fca44c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:50.323929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:50.425126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3501/7049] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:50.484226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:50.486305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:50.643295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85a8e4-9d60-4796-82c0-770d5cf4aacf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:50.643583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85a8e4-9d60-4796-82c0-770d5cf4aacf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:50.644542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85a8e4-9d60-4796-82c0-770d5cf4aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:50.670414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686d9db-bbea-4771-8a07-d0ff424fd3ec admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:50.675868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:50.677002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:50.677494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:50.678030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:50.689975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:50.692581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:50.701574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ec85a8e4-9d60-4796-82c0-770d5cf4aacf None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:50.704054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec85a8e4-9d60-4796-82c0-770d5cf4aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:50.716201 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3549/7050] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:50.880611 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3502/7051] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:23:50.894287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:50.895829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:50.907850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:51.074235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcce6-702b-41fe-9c01-6917774f035c service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:51.081156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:51.082515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:51.083145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:51.084108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:51.104785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:51.115965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:51.129848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3550/7052] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:23:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:23:51.807286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:51.807896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:51.808177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:51.846912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:51.848464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:51.871781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3503/7053] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:51.889478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:51.914940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde9321-de9a-4114-b869-d3a2f357ea70 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:51.924116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc31a7-5f3a-4198-85a9-59207b8a4f90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:51.925627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc31a7-5f3a-4198-85a9-59207b8a4f90 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:51.926279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc31a7-5f3a-4198-85a9-59207b8a4f90 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:51.926955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc31a7-5f3a-4198-85a9-59207b8a4f90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:51.941017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc31a7-5f3a-4198-85a9-59207b8a4f90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:51.945625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc31a7-5f3a-4198-85a9-59207b8a4f90 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:51.985556 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3551/7054] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:23:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:23:52.448393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:52.451918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:52.471407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b23103-fe57-4242-a6fe-6026641bac98 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:52.477715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:52.478803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:52.479490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:52.479914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] PATH_INFO: `/v3/projects/eadfe59250434a348903cf3bcd079465` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:52.481525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eadfe59250434a348903cf3bcd079465)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:52.485641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:52.616134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eadfe59250434a348903cf3bcd079465', 'request_id': 'req-47e683cc-41c9-42f8-8cc2-21e508891d52'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:52.633545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eadfe59250434a348903cf3bcd079465', 'request_id': 'req-47e683cc-41c9-42f8-8cc2-21e508891d52'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:52.650362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eadfe59250434a348903cf3bcd079465', 'request_id': 'req-47e683cc-41c9-42f8-8cc2-21e508891d52'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:52.682196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] The token cache is being invalidate because project eadfe59250434a348903cf3bcd079465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:23:52.688935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:52.692599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-47e683cc-41c9-42f8-8cc2-21e508891d52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:52.695352 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3504/7055] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:23:52 2022] DELETE /identity/v3/projects/eadfe59250434a348903cf3bcd079465 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:52.700419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40207-c354-4b06-8f5a-2c4e4e1b6dde None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:52.700980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40207-c354-4b06-8f5a-2c4e4e1b6dde None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:52.701395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40207-c354-4b06-8f5a-2c4e4e1b6dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:52.728198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a4d40207-c354-4b06-8f5a-2c4e4e1b6dde None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:52.729601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a4d40207-c354-4b06-8f5a-2c4e4e1b6dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:52.810645 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3552/7056] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:52.818524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502e63-ee34-48e9-adad-82ddb078bc89 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:52.819464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502e63-ee34-48e9-adad-82ddb078bc89 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:52.819946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502e63-ee34-48e9-adad-82ddb078bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:52.844778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e3502e63-ee34-48e9-adad-82ddb078bc89 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:52.846347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e3502e63-ee34-48e9-adad-82ddb078bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:52.858956 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3505/7057] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:52.863590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d40207-c354-4b06-8f5a-2c4e4e1b6dde None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:52.874768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d40207-c354-4b06-8f5a-2c4e4e1b6dde None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:52.880193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:52.880919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:52.881200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:52.881471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:52.885805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:23:52.885805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:52.885805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:52.903385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3553/7058] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:23:52.905086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502e63-ee34-48e9-adad-82ddb078bc89 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:52.918928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502e63-ee34-48e9-adad-82ddb078bc89 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:52.925082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:52.925897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:52.926357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:52.926751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:52.928225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:52.931921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:52.986291 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3506/7059] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:52 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:23:52.991046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:53.006507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31419c44-7f0f-4e53-815e-42c7aed3aef2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:53.122880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:53.125912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:53.126606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:53.127119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:53.127982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:53.130742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:53.165181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:53.179962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3554/7060] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:52 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:53.180233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:53.198569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d16f4-c031-4f86-9d3c-813cf528d84d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:53.212055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:53.212773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:53.213057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:53.213334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:53.214050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:53.215704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:53.217635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:53.231505 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3507/7061] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:23:53.239924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:53.256836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06621519-a15e-468a-b2f9-fa12958b6f7f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:53.265756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35721c97-0139-4d8b-958c-8089bccd405d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:53.267075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721c97-0139-4d8b-958c-8089bccd405d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:53.267861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721c97-0139-4d8b-958c-8089bccd405d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:53.268394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35721c97-0139-4d8b-958c-8089bccd405d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:53.269390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35721c97-0139-4d8b-958c-8089bccd405d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:53.272022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35721c97-0139-4d8b-958c-8089bccd405d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:53.329197 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3555/7062] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:53.331124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:53.345786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386f50b-fd9d-4359-84d1-17756c0d79b1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:53.355890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:53.356751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:53.357155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:53.357548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] PATH_INFO: `/v3/projects/8f280feb1d96433dbbfb114f3d5bf8d1/users/b1a233f92d6743c0a7e55730ac0af920/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:53.380580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f280feb1d96433dbbfb114f3d5bf8d1, user_id=b1a233f92d6743c0a7e55730ac0af920, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:53.383984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:53.407767 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3508/7063] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:53 2022] PUT /identity/v3/projects/8f280feb1d96433dbbfb114f3d5bf8d1/users/b1a233f92d6743c0a7e55730ac0af920/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:53.565643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d94d-4ae0-4ecd-9beb-91c7d9c8ca90 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:53.566166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d94d-4ae0-4ecd-9beb-91c7d9c8ca90 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:53.566689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d94d-4ae0-4ecd-9beb-91c7d9c8ca90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:53.619492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-54c8d94d-4ae0-4ecd-9beb-91c7d9c8ca90 None None] MFA Rules not processed for user `b1a233f92d6743c0a7e55730ac0af920`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:53.621343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-54c8d94d-4ae0-4ecd-9beb-91c7d9c8ca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:53.699248 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3556/7064] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:23:53.703907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:53.793435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa081a-afc6-4b90-bd55-66e9a4573b51 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:53.810910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:53.812369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:53.812926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:53.813258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:53.814544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:53.817434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:53.890720 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3509/7065] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:53 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:53.894781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d94d-4ae0-4ecd-9beb-91c7d9c8ca90 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:53.914575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d94d-4ae0-4ecd-9beb-91c7d9c8ca90 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:54.001766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:54.002955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:54.003419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:54.003952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:54.005701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:54.008110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:54.052173 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3557/7066] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:53 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:54.052402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:54.068585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd699d-1575-4b76-a0fd-65afbc177799 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:54.073905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:54.074813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:54.075398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:54.075884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:54.076607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:54.078482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:54.094984 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3510/7067] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:54.099481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:54.111333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248e3c8-0b4e-4d50-8ee0-51e5642db7de admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:54.117062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:54.117753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:54.118034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:54.118465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] PATH_INFO: `/v3/projects/e855786c15db4583b3d426dedef509b9/users/dcf9dcc133bb43bbb0a96e706e9dfa60/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:54.163713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e855786c15db4583b3d426dedef509b9, user_id=dcf9dcc133bb43bbb0a96e706e9dfa60, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:54.166719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:54.196046 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3558/7068] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:54 2022] PUT /identity/v3/projects/e855786c15db4583b3d426dedef509b9/users/dcf9dcc133bb43bbb0a96e706e9dfa60/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:54.200398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:54.219318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c9e54-4d78-4c3d-a2c6-bbb876ff9ec4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:54.349234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4821b-ddbf-4382-8f94-5a8700127a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:54.350104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4821b-ddbf-4382-8f94-5a8700127a13 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:54.350481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4821b-ddbf-4382-8f94-5a8700127a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:54.351054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4821b-ddbf-4382-8f94-5a8700127a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:54.351812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4821b-ddbf-4382-8f94-5a8700127a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:54.355020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4821b-ddbf-4382-8f94-5a8700127a13 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:54.371427 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3511/7069] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:54.375530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:54.399617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daecf4-f932-487f-8825-e2528c3499cd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:54.412923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:54.413629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:54.413912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:54.414184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] PATH_INFO: `/v3/projects/e855786c15db4583b3d426dedef509b9/users/dcf9dcc133bb43bbb0a96e706e9dfa60/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:54.421285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e855786c15db4583b3d426dedef509b9, user_id=dcf9dcc133bb43bbb0a96e706e9dfa60, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:54.433665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:54.455028 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3559/7070] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:54 2022] PUT /identity/v3/projects/e855786c15db4583b3d426dedef509b9/users/dcf9dcc133bb43bbb0a96e706e9dfa60/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:54.466195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fde54-c962-457a-b918-178920f91899 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:54.466659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fde54-c962-457a-b918-178920f91899 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:54.467132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fde54-c962-457a-b918-178920f91899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:54.526510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-891fde54-c962-457a-b918-178920f91899 None None] MFA Rules not processed for user `dcf9dcc133bb43bbb0a96e706e9dfa60`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:54.528029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-891fde54-c962-457a-b918-178920f91899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:54.590416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:54.732522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3512/7071] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:54.739151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:54.743381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:54.824156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:54.860081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3d845-983a-409f-9bd6-62cb7f40c0e3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:54.864880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:54.866874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:54.867935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:54.868416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:54.983841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:54.988205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:55.016596 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3560/7072] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:23:57.445366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f8559-10c8-4d19-8eb2-cfd01407daf3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:57.448262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f8559-10c8-4d19-8eb2-cfd01407daf3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:57.449979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f8559-10c8-4d19-8eb2-cfd01407daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:57.503657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6e3f8559-10c8-4d19-8eb2-cfd01407daf3 None None] MFA Rules not processed for user `178956f6c8154a06a0710b295aff3086`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:57.509257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6e3f8559-10c8-4d19-8eb2-cfd01407daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:57.745376 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3513/7073] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:57.750345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:57.757520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:57.977842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2cc3a-10f6-45a5-bcea-3815bcdc45d9 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:57.985937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:57.986582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:57.986816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:57.987136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:57.990396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:57.995367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:58.057439 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3561/7074] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:23:57 2022] POST /identity/v3/projects => generated 379 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:58.062009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f8559-10c8-4d19-8eb2-cfd01407daf3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:58.075629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f8559-10c8-4d19-8eb2-cfd01407daf3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:58.237298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:58.237992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:58.238219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:58.238565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:58.240134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:58.242879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:58.252696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:58.272501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35540266-c214-4249-9fdd-fc9deeda96c6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:58.290447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:58.292716 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3514/7075] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:23:58 2022] POST /identity/v3/users => generated 296 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:23:58.294220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:58.295545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:58.296344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] PATH_INFO: `/v3/users/b1a233f92d6743c0a7e55730ac0af920` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:58.297255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:58.300393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a233f92d6743c0a7e55730ac0af920)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:58.306182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:58.351441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c3fc-007e-4d08-ab0e-f6734278b56a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:58.370016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:58.372036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:58.372318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:58.372590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:58.387007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:58.389609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:58.429799 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3515/7076] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:23:58.432614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:58.457807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022880a-71eb-40de-984f-9c874dcd893c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:58.468804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:58.471765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:58.472057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:58.481055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] PATH_INFO: `/v3/projects/8b8299ae80f441c5850363b0794c28bf/users/220cd4bdb57b402fb15667386d90dc58/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:58.509323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a233f92d6743c0a7e55730ac0af920'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:58.510047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a233f92d6743c0a7e55730ac0af920'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:58.531810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a233f92d6743c0a7e55730ac0af920'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:58.534499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8299ae80f441c5850363b0794c28bf, user_id=220cd4bdb57b402fb15667386d90dc58, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:58.541004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:58.550352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a233f92d6743c0a7e55730ac0af920'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:58.552545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4d059772-6c96-4ce6-b7c8-50b47a7f8083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a233f92d6743c0a7e55730ac0af920'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:23:58.554519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:58.589595 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3516/7077] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:58 2022] PUT /identity/v3/projects/8b8299ae80f441c5850363b0794c28bf/users/220cd4bdb57b402fb15667386d90dc58/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:58.593893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:58.595860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:58.621073 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3562/7078] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:23:58 2022] DELETE /identity/v3/users/b1a233f92d6743c0a7e55730ac0af920 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:58.625739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:58.626026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:58.626295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:58.631113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:58.662307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:23:58.663573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:58.774806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c86a4a-7a86-495f-8273-c2399037532e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:58.874694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3563/7079] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:23:58.930675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:58.934500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:23:58.994113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:58.995031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:58.995316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:58.996142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:58.996278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:59.002006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:59.034629 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3517/7080] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:23:59.037723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:23:59.071168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352b73a-ce1e-4cc2-b57e-2c329c4e6653 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:59.088892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:59.089699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:59.089871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:59.090131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] PATH_INFO: `/v3/projects/8b8299ae80f441c5850363b0794c28bf/users/220cd4bdb57b402fb15667386d90dc58/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:59.093172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8299ae80f441c5850363b0794c28bf, user_id=220cd4bdb57b402fb15667386d90dc58, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:59.111430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:59.140512 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3518/7081] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:23:59 2022] PUT /identity/v3/projects/8b8299ae80f441c5850363b0794c28bf/users/220cd4bdb57b402fb15667386d90dc58/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:23:59.144943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524ba5c-234a-468f-af1e-33f16b32a328 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:23:59.283520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:23:59.284615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:23:59.285116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:23:59.285598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:23:59.371453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:59.396869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:23:59.399009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:23:59.418621 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3564/7082] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:00.780364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:00.783343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:00.826697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc646364-44c3-4916-a879-3132fc06118f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:00.841777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:00.843970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:00.844320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:00.844607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] PATH_INFO: `/v3/projects/8f280feb1d96433dbbfb114f3d5bf8d1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:00.846030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f280feb1d96433dbbfb114f3d5bf8d1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:00.868830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:01.091928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f280feb1d96433dbbfb114f3d5bf8d1', 'request_id': 'req-f1b0a629-9771-4046-8927-aace93b80933'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.100246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f280feb1d96433dbbfb114f3d5bf8d1', 'request_id': 'req-f1b0a629-9771-4046-8927-aace93b80933'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.117746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f280feb1d96433dbbfb114f3d5bf8d1', 'request_id': 'req-f1b0a629-9771-4046-8927-aace93b80933'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.151265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] The token cache is being invalidate because project 8f280feb1d96433dbbfb114f3d5bf8d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:01.151986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.153020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b0a629-9771-4046-8927-aace93b80933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.155116 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3519/7083] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:00 2022] DELETE /identity/v3/projects/8f280feb1d96433dbbfb114f3d5bf8d1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:01.159994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:01.176512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbfa40-1460-4ab8-a8d5-21c65430d884 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:01.329365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:01.330453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:01.330940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:01.331430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] PATH_INFO: `/v3/users/dcf9dcc133bb43bbb0a96e706e9dfa60` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:01.334189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf9dcc133bb43bbb0a96e706e9dfa60)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:01.336622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:01.481459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf9dcc133bb43bbb0a96e706e9dfa60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.486770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf9dcc133bb43bbb0a96e706e9dfa60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.512507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf9dcc133bb43bbb0a96e706e9dfa60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.529734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf9dcc133bb43bbb0a96e706e9dfa60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.531827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf9dcc133bb43bbb0a96e706e9dfa60'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:01.581514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3565/7084] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:01 2022] DELETE /identity/v3/users/dcf9dcc133bb43bbb0a96e706e9dfa60 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:02.031827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:02.032908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:02.033153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:02.059142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] MFA Rules not processed for user `97c769dbba794531b0a9db8e764ff097`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:02.060704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:02.123845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:02.202791 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3520/7085] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:02.351681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:02.353960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:02.489203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6868-a3ac-429c-9f16-632fcdae0cd2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:02.498956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:02.500497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:02.501792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:02.502378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] PATH_INFO: `/v3/users/854702c021af4fcc9e2ade1547aaf3b6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:02.505580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] RBAC: Authorizing `identity:delete_user(user_id=854702c021af4fcc9e2ade1547aaf3b6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:02.510453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:02.524694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:02.527834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:02.528830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:02.572257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd6df1-c774-47c1-82ed-3f5dcfe0707f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:02.591945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:02.592688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:02.594271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:02.595248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] PATH_INFO: `/v3/projects/e855786c15db4583b3d426dedef509b9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:02.602830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e855786c15db4583b3d426dedef509b9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:02.605559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:02.646378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854702c021af4fcc9e2ade1547aaf3b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.647275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854702c021af4fcc9e2ade1547aaf3b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.677208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854702c021af4fcc9e2ade1547aaf3b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.687286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854702c021af4fcc9e2ade1547aaf3b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.688410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7905eef2-b0ba-4fa0-ba38-395e77d5268a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854702c021af4fcc9e2ade1547aaf3b6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.722870 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3566/7086] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:02 2022] DELETE /identity/v3/users/854702c021af4fcc9e2ade1547aaf3b6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:02.727971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:02.728395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:02.728908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:02.745526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e855786c15db4583b3d426dedef509b9', 'request_id': 'req-ea707f45-11ce-4a84-a792-adc2449598c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.755708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:02.758117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:02.765766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e855786c15db4583b3d426dedef509b9', 'request_id': 'req-ea707f45-11ce-4a84-a792-adc2449598c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.783353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e855786c15db4583b3d426dedef509b9', 'request_id': 'req-ea707f45-11ce-4a84-a792-adc2449598c4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.805050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] The token cache is being invalidate because project e855786c15db4583b3d426dedef509b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:02.805679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.807227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ea707f45-11ce-4a84-a792-adc2449598c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:02.822643 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3521/7087] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:02 2022] DELETE /identity/v3/projects/e855786c15db4583b3d426dedef509b9 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:02.827323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:02.827963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:02.828994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:02.886889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3567/7088] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:02.891388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:02.896150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:02.899871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:02.901075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:02.910573 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3522/7089] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:02.911937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:02.913545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:03.030100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5b03a-b2a1-4e74-9514-6dba9389f932 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.034354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff397d36-0f37-4111-bbed-a31e7da7cfbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.035227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff397d36-0f37-4111-bbed-a31e7da7cfbe service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.035786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff397d36-0f37-4111-bbed-a31e7da7cfbe service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.036910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff397d36-0f37-4111-bbed-a31e7da7cfbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.044153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5f055-3ce2-441a-82e2-0fd0bf27a8a8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.053594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.055037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.055037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.055513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.056566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff397d36-0f37-4111-bbed-a31e7da7cfbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.058138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff397d36-0f37-4111-bbed-a31e7da7cfbe service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.066334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:03.072763 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3523/7090] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:03.074389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7d31-9183-4e72-8641-8f009c5e6cb4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.074935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7d31-9183-4e72-8641-8f009c5e6cb4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.076204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7d31-9183-4e72-8641-8f009c5e6cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.109319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a62f7d31-9183-4e72-8641-8f009c5e6cb4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:03.113561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a62f7d31-9183-4e72-8641-8f009c5e6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:03.115674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.117778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.135451 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3524/7091] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:03.138205 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3568/7092] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:03.139924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7d31-9183-4e72-8641-8f009c5e6cb4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:03.162302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7d31-9183-4e72-8641-8f009c5e6cb4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.173343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.174280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.175089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.175731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.177957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:24:03.178480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.180733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.209177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:03.211323 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3525/7093] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:24:03.227649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696c2a2-5673-480a-9534-9d069cd7749e service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.233889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.234754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.235174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.235623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.236409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.239979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.300681 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3569/7094] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:03 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:24:03.307303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:03.324945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a83e2-d692-4342-a428-1d420484723b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.419270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:03.546935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.547881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.550505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.550954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.553078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.561878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.672609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:03.673371 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3526/7095] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:03 2022] POST /identity/v3/users => generated 287 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:03.687935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ab697-bb45-4d1b-99ea-925dce4885d8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.712869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.713523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.713804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.714083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.714747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.729972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.731582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:03.774408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:03.777262 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3570/7096] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:03.792995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ad09e-3fe0-487e-8f60-79f899289894 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.807800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad583-b63d-4fe8-89b4-91043a43e134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.808404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad583-b63d-4fe8-89b4-91043a43e134 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.808700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad583-b63d-4fe8-89b4-91043a43e134 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.808976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad583-b63d-4fe8-89b4-91043a43e134 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.809632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ad583-b63d-4fe8-89b4-91043a43e134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.811350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ad583-b63d-4fe8-89b4-91043a43e134 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.838896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:03.839924 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3527/7097] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:03.852756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe883a-3af4-4773-b930-e72d48695931 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:03.868349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:03.869453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.869453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.869717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] PATH_INFO: `/v3/projects/bed3b0de0f0e468fb5d2c3c930378f43/users/a6cf4cb74575469a91ae7144012c2e2c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:03.902762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed3b0de0f0e468fb5d2c3c930378f43, user_id=a6cf4cb74575469a91ae7144012c2e2c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:03.905386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:03.937818 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3571/7098] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:03 2022] PUT /identity/v3/projects/bed3b0de0f0e468fb5d2c3c930378f43/users/a6cf4cb74575469a91ae7144012c2e2c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:03.949964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:03.950245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:03.950526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:04.011579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] MFA Rules not processed for user `a6cf4cb74575469a91ae7144012c2e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:04.014549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:04.203861 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3528/7099] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:04.217632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:04.221164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:04.243208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:04.245893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:04.403784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82d4ee-06c3-4cc6-9747-750e62ff9937 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:04.416828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:04.417378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:04.422264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:04.422544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:04.465653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:04.466037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2663ee-37d0-4c13-961c-14e636443569 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:04.469608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:04.472989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:04.474514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:04.474801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:04.475081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] PATH_INFO: `/v3/projects/7c062933ff2d4cceba38c9245a1770ea` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:04.476409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c062933ff2d4cceba38c9245a1770ea)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:04.481326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:04.511392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3572/7100] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:04.537730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:04.557014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73375ef6-09c9-43fb-8ce6-7a48eef5ab9c service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:04.584334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:04.585957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:04.587913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:04.588368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] PATH_INFO: `/v3/users/a6cf4cb74575469a91ae7144012c2e2c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:04.591105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6cf4cb74575469a91ae7144012c2e2c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:04.594177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:04.638257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:04.686762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c062933ff2d4cceba38c9245a1770ea', 'request_id': 'req-b805ec46-d60f-4870-84d6-8921802fccf7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.719261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c062933ff2d4cceba38c9245a1770ea', 'request_id': 'req-b805ec46-d60f-4870-84d6-8921802fccf7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.733718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6cf4cb74575469a91ae7144012c2e2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.737318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6cf4cb74575469a91ae7144012c2e2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.756996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c062933ff2d4cceba38c9245a1770ea', 'request_id': 'req-b805ec46-d60f-4870-84d6-8921802fccf7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.762850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6cf4cb74575469a91ae7144012c2e2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.800833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] The token cache is being invalidate because project 7c062933ff2d4cceba38c9245a1770ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:04.803351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.804438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b805ec46-d60f-4870-84d6-8921802fccf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.808966 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3529/7101] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:04 2022] DELETE /identity/v3/projects/7c062933ff2d4cceba38c9245a1770ea => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:04.809876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6cf4cb74575469a91ae7144012c2e2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.813812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9fd83d72-08cc-4b63-8c1e-220baf5a01cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6cf4cb74575469a91ae7144012c2e2c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:04.816087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4633-61f1-47e2-9fe8-2c879d485937 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:04.816514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4633-61f1-47e2-9fe8-2c879d485937 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:04.818442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4633-61f1-47e2-9fe8-2c879d485937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:04.876141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-079c4633-61f1-47e2-9fe8-2c879d485937 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:04.879894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-079c4633-61f1-47e2-9fe8-2c879d485937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:04.919567 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3573/7102] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:04 2022] DELETE /identity/v3/users/a6cf4cb74575469a91ae7144012c2e2c => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:04.924192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:04.924457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:04.925489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:04.976566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:04.978352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:05.046696 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3530/7103] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:05.048364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b4aeb-fef5-4960-aca5-c972a54a7eaa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.048645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b4aeb-fef5-4960-aca5-c972a54a7eaa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.048940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b4aeb-fef5-4960-aca5-c972a54a7eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.119895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:05.142070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-977b4aeb-fef5-4960-aca5-c972a54a7eaa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:05.145284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-977b4aeb-fef5-4960-aca5-c972a54a7eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:05.180724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3574/7104] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:24:05.219340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:05.225315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:05.322186 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3531/7105] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:24:05.323505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b4aeb-fef5-4960-aca5-c972a54a7eaa None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:05.342769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b4aeb-fef5-4960-aca5-c972a54a7eaa None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:05.366292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:05.366949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.367225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.367494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.368160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:24:05.368462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:05.370115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:05.402011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4449cc4-e8b8-467c-a34a-c6390979c34d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:05.406418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:05.407172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.407416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.407647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.409892 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3532/7106] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:24:05.411141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:05.423877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:05.429798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:05.444478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493de11-ff76-4f2d-8d95-cc50c6b0b8e8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:05.455346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:05.455984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.456260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.456529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.459497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:05.471777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:05.477038 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3575/7107] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:05.524290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3533/7108] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:05 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:05.528796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:05.546124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd258ca-4658-4793-9762-b19e51194f28 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:05.659629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:05.660406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.661061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.661486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.662231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:05.664803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:05.714629 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3576/7109] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:05 2022] POST /identity/v3/users => generated 279 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:05.719175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:05.732292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240ab34-ad6a-4424-9ab8-06a2688261bd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:05.767566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:05.776684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.779144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.779672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.782689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:05.784471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:05.820361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:05.831314 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3534/7110] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:05.838811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72edfe-61e1-4bc4-8f4c-3274013c3af6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:05.857576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cbdd3-ab56-4700-b65f-3c31567c18fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:05.858468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cbdd3-ab56-4700-b65f-3c31567c18fc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.858850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cbdd3-ab56-4700-b65f-3c31567c18fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.859235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cbdd3-ab56-4700-b65f-3c31567c18fc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.860003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cbdd3-ab56-4700-b65f-3c31567c18fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:05.863377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cbdd3-ab56-4700-b65f-3c31567c18fc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:05.898694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3577/7111] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:24:05.903140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:05.919580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd89abe-8fd4-4f5b-93cd-e02e9fe15a8f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:05.934237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:05.934942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:05.935227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:05.935503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] PATH_INFO: `/v3/projects/2694d8288d9a428db2eaaa11ef966905/users/8d0de363404e48f29ffb05eb39db4e03/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:05.977347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2694d8288d9a428db2eaaa11ef966905, user_id=8d0de363404e48f29ffb05eb39db4e03, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:05.980308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:06.008463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:06.018831 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3535/7112] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:05 2022] PUT /identity/v3/projects/2694d8288d9a428db2eaaa11ef966905/users/8d0de363404e48f29ffb05eb39db4e03/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:06.028737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:06.029041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:06.029314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:06.118667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] MFA Rules not processed for user `8d0de363404e48f29ffb05eb39db4e03`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:06.119765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:06.570562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:06.592683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3578/7113] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 565 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:06.707144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:06.849652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cf4dc-d471-4d15-af26-c913fd2c8933 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:06.854461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbb141-26ae-4cc8-91c0-ea8707d021d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:06.855402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbb141-26ae-4cc8-91c0-ea8707d021d0 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:06.855897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbb141-26ae-4cc8-91c0-ea8707d021d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:06.856944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbb141-26ae-4cc8-91c0-ea8707d021d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:06.869640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbb141-26ae-4cc8-91c0-ea8707d021d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:06.871717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbb141-26ae-4cc8-91c0-ea8707d021d0 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:06.886631 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3536/7114] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:07.665006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:07.667682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:07.768332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4c3cd-bbd2-47a0-a0ba-90e779ad3d01 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:07.775015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:07.776758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:07.777279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:07.778073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] PATH_INFO: `/v3/projects/bed3b0de0f0e468fb5d2c3c930378f43` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:07.779924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed3b0de0f0e468fb5d2c3c930378f43)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:07.782972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:07.867415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed3b0de0f0e468fb5d2c3c930378f43', 'request_id': 'req-136dfb5f-bf18-4891-9b3d-47ddd5461e69'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:07.892230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed3b0de0f0e468fb5d2c3c930378f43', 'request_id': 'req-136dfb5f-bf18-4891-9b3d-47ddd5461e69'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:07.905335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed3b0de0f0e468fb5d2c3c930378f43', 'request_id': 'req-136dfb5f-bf18-4891-9b3d-47ddd5461e69'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:07.934105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] The token cache is being invalidate because project bed3b0de0f0e468fb5d2c3c930378f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:07.934406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:07.935704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-136dfb5f-bf18-4891-9b3d-47ddd5461e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:07.939240 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3579/7115] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:07 2022] DELETE /identity/v3/projects/bed3b0de0f0e468fb5d2c3c930378f43 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:07.944198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f7d96-1330-41d9-91e9-635717d3e900 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:07.945070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f7d96-1330-41d9-91e9-635717d3e900 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:07.945489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f7d96-1330-41d9-91e9-635717d3e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:07.992497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-509f7d96-1330-41d9-91e9-635717d3e900 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:07.994255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-509f7d96-1330-41d9-91e9-635717d3e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:08.104443 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3537/7116] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:08.112058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793ee86-11da-402e-bc2d-aeace51ecdb3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.112419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793ee86-11da-402e-bc2d-aeace51ecdb3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.112831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793ee86-11da-402e-bc2d-aeace51ecdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.146587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f793ee86-11da-402e-bc2d-aeace51ecdb3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:08.148435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f793ee86-11da-402e-bc2d-aeace51ecdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:08.158165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3580/7117] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:08.163103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f7d96-1330-41d9-91e9-635717d3e900 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:08.181289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f7d96-1330-41d9-91e9-635717d3e900 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:08.190888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:08.193280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.195032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.195328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.201038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:24:08.201403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:08.203091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:08.231924 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3538/7118] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:24:08.235306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793ee86-11da-402e-bc2d-aeace51ecdb3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:08.252239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793ee86-11da-402e-bc2d-aeace51ecdb3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:08.264323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:08.266953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.267494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.269036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.272393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:08.278675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:08.331354 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3581/7119] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:08 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:08.336205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:08.349930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73c6eb-da6e-4358-a132-4460e11d6772 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:08.460037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:08.460311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.460534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.461428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.462229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:08.464485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:08.498049 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3539/7120] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:08 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:08.502930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:08.527358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029e2a6-8e11-4031-af7e-279383c3bc39 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:08.538933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:08.540483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.541103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.541665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.542884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:08.545364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:08.562194 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3582/7121] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:24:08.566353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:08.582603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4021f73-6e4e-4c6f-a69c-bf97c285404a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:08.588967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bf6f0-993a-4ad1-a718-c99db4f8e63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:08.589603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bf6f0-993a-4ad1-a718-c99db4f8e63a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.589880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bf6f0-993a-4ad1-a718-c99db4f8e63a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.590148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bf6f0-993a-4ad1-a718-c99db4f8e63a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.590847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bf6f0-993a-4ad1-a718-c99db4f8e63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:08.592688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bf6f0-993a-4ad1-a718-c99db4f8e63a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:08.616899 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3540/7122] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:08.617508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:08.649029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d6f64-d73e-4401-a2b1-988a93a66880 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:08.666097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:08.667653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.668114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.669556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] PATH_INFO: `/v3/projects/712c1c4c858b4b788a7703345716504b/users/584c395757a54d9b9a64dd399d8557b1/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.697045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] RBAC: Authorizing `identity:create_grant(project_id=712c1c4c858b4b788a7703345716504b, user_id=584c395757a54d9b9a64dd399d8557b1, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:08.700199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:08.726602 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3583/7123] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:08 2022] PUT /identity/v3/projects/712c1c4c858b4b788a7703345716504b/users/584c395757a54d9b9a64dd399d8557b1/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:08.731562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb8702-fe2b-49c9-9c8a-9ff22888892d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:08.731893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb8702-fe2b-49c9-9c8a-9ff22888892d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:08.732438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb8702-fe2b-49c9-9c8a-9ff22888892d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:08.753419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:08.766297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d2cb8702-fe2b-49c9-9c8a-9ff22888892d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:08.776197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d2cb8702-fe2b-49c9-9c8a-9ff22888892d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:08.900182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3541/7124] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:24:08.947669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:08.950498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:09.009385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:09.068060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76f608-3c25-4861-bce2-bbdfe41e119c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:09.086791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:09.087560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:09.088037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:09.088465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:09.107111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:09.109110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:09.125645 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3584/7125] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:12.589438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaffe71-9549-479e-bc13-55d225d90e27 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:12.590019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaffe71-9549-479e-bc13-55d225d90e27 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:12.590019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaffe71-9549-479e-bc13-55d225d90e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:12.678133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fcaffe71-9549-479e-bc13-55d225d90e27 None None] MFA Rules not processed for user `584c395757a54d9b9a64dd399d8557b1`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:12.680724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fcaffe71-9549-479e-bc13-55d225d90e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:12.842647 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3542/7126] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:12.857057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:12.859652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:12.986672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d49c-88c7-4f0b-a874-935e2d32001e service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:12.992492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:12.993151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:12.993674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:12.994327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:13.004600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:13.006900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:13.024029 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3585/7127] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:14.477730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:14.478687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:14.479523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:14.543612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] MFA Rules not processed for user `220cd4bdb57b402fb15667386d90dc58`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:14.545353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:14.747689 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3543/7128] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:14.758233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:14.759720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:14.880902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc870a3-97cd-481c-8a68-2f3152e1e92b service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:14.900214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:14.901313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:14.901886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:14.902491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:14.904808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:15.102087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:15.104232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:15.134665 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3586/7129] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:16.519503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:16.522811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:16.553670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d143d78-b015-47fa-b8e7-2d40331c2295 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:16.566381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8096c-f932-421a-bc3a-fc2c97e2b432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:16.567671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8096c-f932-421a-bc3a-fc2c97e2b432 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:16.569605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8096c-f932-421a-bc3a-fc2c97e2b432 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:16.570105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8096c-f932-421a-bc3a-fc2c97e2b432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:16.574905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a5c8096c-f932-421a-bc3a-fc2c97e2b432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:16.599490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8096c-f932-421a-bc3a-fc2c97e2b432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:16.619742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8096c-f932-421a-bc3a-fc2c97e2b432 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:16.638595 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3544/7130] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:21.084967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747415-417a-43b3-8922-917ad191e651 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:21.085704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747415-417a-43b3-8922-917ad191e651 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:21.085863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:21.086956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747415-417a-43b3-8922-917ad191e651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:21.087971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:21.109995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3007bb6-8b7f-4fa1-82a1-1360f44ec3ff service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:21.116810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:21.117483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:21.117750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:21.118020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] PATH_INFO: `/v3/users/8d0de363404e48f29ffb05eb39db4e03` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:21.119525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0de363404e48f29ffb05eb39db4e03)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:21.121867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:21.131407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-47747415-417a-43b3-8922-917ad191e651 None None] MFA Rules not processed for user `178956f6c8154a06a0710b295aff3086`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:21.134085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-47747415-417a-43b3-8922-917ad191e651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:21.148301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3545/7131] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:21.153256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:21.202870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0de363404e48f29ffb05eb39db4e03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:21.205305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0de363404e48f29ffb05eb39db4e03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:21.227721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0de363404e48f29ffb05eb39db4e03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:21.238505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0de363404e48f29ffb05eb39db4e03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:21.239125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0de363404e48f29ffb05eb39db4e03'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:21.268764 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3587/7132] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:21 2022] DELETE /identity/v3/users/8d0de363404e48f29ffb05eb39db4e03 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:21.274861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0bf41-5269-48c1-8942-840398f33193 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:21.275461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0bf41-5269-48c1-8942-840398f33193 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:21.275883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0bf41-5269-48c1-8942-840398f33193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:21.298873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:21.319342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-18a0bf41-5269-48c1-8942-840398f33193 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:21.321557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-18a0bf41-5269-48c1-8942-840398f33193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:21.436899 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3546/7133] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:21.459391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:21.461095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:21.585880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc01634-f187-4660-8f42-fafe1a5ef8d7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:21.593861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:21.595786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:21.596304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:21.597091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:21.611796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:21.614191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:21.639320 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3588/7134] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:21.883012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0bf41-5269-48c1-8942-840398f33193 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:21.904819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0bf41-5269-48c1-8942-840398f33193 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:21.911408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea29f81-d1eb-44ce-9a6a-56b826db3ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:21.912572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29f81-d1eb-44ce-9a6a-56b826db3ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:21.913990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29f81-d1eb-44ce-9a6a-56b826db3ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:21.914978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29f81-d1eb-44ce-9a6a-56b826db3ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:21.918073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2ea29f81-d1eb-44ce-9a6a-56b826db3ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:21.936584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea29f81-d1eb-44ce-9a6a-56b826db3ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:21.938526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea29f81-d1eb-44ce-9a6a-56b826db3ab7 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:21.960930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3547/7135] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:22.320957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:22.322921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:22.366079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d9a12-a74c-4750-8b32-54504b95b067 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:22.374305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:22.377918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:22.378409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:22.379086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] PATH_INFO: `/v3/projects/2694d8288d9a428db2eaaa11ef966905` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:22.380666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2694d8288d9a428db2eaaa11ef966905)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:22.383278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:22.568329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2694d8288d9a428db2eaaa11ef966905', 'request_id': 'req-4575abfd-46c0-41b9-9eb9-269e859c8761'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:22.603537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2694d8288d9a428db2eaaa11ef966905', 'request_id': 'req-4575abfd-46c0-41b9-9eb9-269e859c8761'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:22.611712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2694d8288d9a428db2eaaa11ef966905', 'request_id': 'req-4575abfd-46c0-41b9-9eb9-269e859c8761'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:22.635529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] The token cache is being invalidate because project 2694d8288d9a428db2eaaa11ef966905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:22.636924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:22.637939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-4575abfd-46c0-41b9-9eb9-269e859c8761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:22.642925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3589/7136] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:22 2022] DELETE /identity/v3/projects/2694d8288d9a428db2eaaa11ef966905 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:22.647048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859266f7-67f3-439c-855a-3881f0e1e9a1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:22.647570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859266f7-67f3-439c-855a-3881f0e1e9a1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:22.647570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859266f7-67f3-439c-855a-3881f0e1e9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:22.670636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-859266f7-67f3-439c-855a-3881f0e1e9a1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:22.672249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-859266f7-67f3-439c-855a-3881f0e1e9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:22.851423 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3548/7137] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:22.859745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08c76a-6f3a-4829-a8c9-24e69a098f8e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:22.860079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08c76a-6f3a-4829-a8c9-24e69a098f8e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:22.860327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08c76a-6f3a-4829-a8c9-24e69a098f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:22.884963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bc08c76a-6f3a-4829-a8c9-24e69a098f8e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:22.889411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bc08c76a-6f3a-4829-a8c9-24e69a098f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:22.905260 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3590/7138] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:24:22.906830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859266f7-67f3-439c-855a-3881f0e1e9a1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:22.919050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859266f7-67f3-439c-855a-3881f0e1e9a1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:22.926198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:22.927065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:22.927533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:22.927933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:22.928804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:24:22.929139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:22.931283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:22.968518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08c76a-6f3a-4829-a8c9-24e69a098f8e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:22.975781 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3549/7139] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:24:22.983947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08c76a-6f3a-4829-a8c9-24e69a098f8e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:22.989718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:22.990650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:22.991015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:22.991342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:22.992736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:22.994928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:23.039253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:23.075873 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3591/7140] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:22 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:23.081834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:23.094362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4e6ac-617f-4e45-bd7e-df5745254cc1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:23.256250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:23.257054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:23.257406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:23.257725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:23.258449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:23.260706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:23.306126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3550/7141] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:23 2022] POST /identity/v3/users => generated 282 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:23.310880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:23.327790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc063b-9abe-42d1-9b56-8fdc18710580 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:23.365692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:23.366325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:23.366605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:23.367325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:23.367592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:23.369555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:23.389771 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3592/7142] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:23.390977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:23.424029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aaa63-c496-4368-9df1-bdbdc11022cb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:23.430614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e5ad6-888a-4320-9749-b985e05ceedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:23.430614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5ad6-888a-4320-9749-b985e05ceedc admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:23.430614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5ad6-888a-4320-9749-b985e05ceedc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:23.431377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5ad6-888a-4320-9749-b985e05ceedc admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:23.432144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e5ad6-888a-4320-9749-b985e05ceedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:23.434222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e5ad6-888a-4320-9749-b985e05ceedc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:23.435876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:23.465284 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3551/7143] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:24:23.468621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:23.498083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431277b2-d2b2-468b-aace-6a323f72c50c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:23.515580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:23.516906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:23.517535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:23.517977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] PATH_INFO: `/v3/projects/4ea02adf4a3947cfbb8faf8d1ca16fa6/users/eda67f1a9da14894b1f0be2211086e38/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:23.544040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea02adf4a3947cfbb8faf8d1ca16fa6, user_id=eda67f1a9da14894b1f0be2211086e38, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:23.578459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:23.618520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3593/7144] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:23 2022] PUT /identity/v3/projects/4ea02adf4a3947cfbb8faf8d1ca16fa6/users/eda67f1a9da14894b1f0be2211086e38/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:23.633907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4543ec-e577-49c0-8b1a-9923ba982209 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:23.634309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4543ec-e577-49c0-8b1a-9923ba982209 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:23.634524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4543ec-e577-49c0-8b1a-9923ba982209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:23.668891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5a4543ec-e577-49c0-8b1a-9923ba982209 None None] MFA Rules not processed for user `eda67f1a9da14894b1f0be2211086e38`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:23.671681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a4543ec-e577-49c0-8b1a-9923ba982209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:23.806137 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3552/7145] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:23.815249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:23.930351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ed63-599a-4fe2-8ee1-b9a75c16cf98 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:23.936122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:23.937101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:23.937595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:23.938054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:23.938871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:23.941056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:23.985351 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3594/7146] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:23 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:23.989460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4543ec-e577-49c0-8b1a-9923ba982209 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:24.007446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4543ec-e577-49c0-8b1a-9923ba982209 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:24.101433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:24.102133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:24.102599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:24.102935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:24.103731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:24.109456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:24.179649 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3553/7147] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:23 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:24.181071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:24.208414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353f52e-50a7-4b11-a6f4-574bbb4fddd9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:24.228165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:24.228830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:24.229901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:24.230185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:24.231199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:24.234440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:24.251987 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3595/7148] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:24.253139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:24.264423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27068c66-727e-4e6e-b322-9e7e64e1a108 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:24.271999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:24.273089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:24.273089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:24.273384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] PATH_INFO: `/v3/projects/83c844a149364df5a53457931cea6f58/users/43994f0e900645c1831b3144dd3c1e3c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:24.306072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c844a149364df5a53457931cea6f58, user_id=43994f0e900645c1831b3144dd3c1e3c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:24.312114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:24.339157 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3554/7149] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:24 2022] PUT /identity/v3/projects/83c844a149364df5a53457931cea6f58/users/43994f0e900645c1831b3144dd3c1e3c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:24.346032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:24.370216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea395fd-0504-49b3-a52b-fbe3f139bf06 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:24.493551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02da0-1019-45da-b0fc-56d58152414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:24.495021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02da0-1019-45da-b0fc-56d58152414c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:24.495021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02da0-1019-45da-b0fc-56d58152414c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:24.495021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02da0-1019-45da-b0fc-56d58152414c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:24.495561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02da0-1019-45da-b0fc-56d58152414c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:24.497462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02da0-1019-45da-b0fc-56d58152414c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:24.515180 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3596/7150] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:24.518286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:24.536732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875b109-7442-43eb-b69b-c7281c0a707f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:24.551668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:24.555941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:24.555941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:24.556553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] PATH_INFO: `/v3/projects/83c844a149364df5a53457931cea6f58/users/43994f0e900645c1831b3144dd3c1e3c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:24.559449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c844a149364df5a53457931cea6f58, user_id=43994f0e900645c1831b3144dd3c1e3c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:24.562622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:24.581841 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3555/7151] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:24 2022] PUT /identity/v3/projects/83c844a149364df5a53457931cea6f58/users/43994f0e900645c1831b3144dd3c1e3c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:24.597459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def61f6-9879-4e95-b1fd-fcc2ed7dc2ed None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:24.597752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def61f6-9879-4e95-b1fd-fcc2ed7dc2ed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:24.598106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def61f6-9879-4e95-b1fd-fcc2ed7dc2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:24.683628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3def61f6-9879-4e95-b1fd-fcc2ed7dc2ed None None] MFA Rules not processed for user `43994f0e900645c1831b3144dd3c1e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:24.686093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3def61f6-9879-4e95-b1fd-fcc2ed7dc2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:24.718018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:24.885719 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3597/7152] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:24.902140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:24.905662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:25.061430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:25.106702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a33a8-7b6a-495b-a0bc-5582dec9173b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:25.130049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:25.130734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:25.130982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:25.131264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:25.160775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:25.164756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:25.190382 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3556/7153] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:25.354874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def61f6-9879-4e95-b1fd-fcc2ed7dc2ed None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:25.379172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def61f6-9879-4e95-b1fd-fcc2ed7dc2ed None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:25.396345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:25.400310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:25.401197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:25.401479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:25.574494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:25.576102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:25.618958 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3598/7154] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:29.087939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:29.090286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:29.159585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:29.294585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff80d-c031-4a0d-b136-ad428ba29178 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:29.306208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:29.306868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:29.307130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:29.307404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] PATH_INFO: `/v3/users/eda67f1a9da14894b1f0be2211086e38` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:29.317271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda67f1a9da14894b1f0be2211086e38)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:29.319467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:29.342781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0423a-6639-45fe-ab36-918076f66550 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:29.357834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203522d-791e-4bf8-8d13-13a04979b821 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:29.359194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203522d-791e-4bf8-8d13-13a04979b821 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:29.359194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203522d-791e-4bf8-8d13-13a04979b821 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:29.359194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203522d-791e-4bf8-8d13-13a04979b821 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:29.361400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4203522d-791e-4bf8-8d13-13a04979b821 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:29.381860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203522d-791e-4bf8-8d13-13a04979b821 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:29.385517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203522d-791e-4bf8-8d13-13a04979b821 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:29.413035 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3557/7155] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:29.474649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda67f1a9da14894b1f0be2211086e38'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:29.476936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda67f1a9da14894b1f0be2211086e38'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:29.506638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda67f1a9da14894b1f0be2211086e38'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:29.508184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:29.537800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda67f1a9da14894b1f0be2211086e38'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:29.540252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda67f1a9da14894b1f0be2211086e38'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:29.628138 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3599/7156] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:29 2022] DELETE /identity/v3/users/eda67f1a9da14894b1f0be2211086e38 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:29.635046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:29.635328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:29.635949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:29.726502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:29.729606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:29.888426 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3558/7157] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:29.929073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:29.930393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:30.037800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdc216-094d-4356-8c8f-226dbf70a700 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:30.044143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:30.050851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:30.051426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:30.051895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:30.079996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:30.100300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:30.116534 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3600/7158] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:30.755979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:30.765531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:30.795696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b9235-3943-4821-8ec4-226cca734557 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:30.815316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:30.821075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:30.821755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:30.822261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] PATH_INFO: `/v3/projects/4ea02adf4a3947cfbb8faf8d1ca16fa6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:30.824956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea02adf4a3947cfbb8faf8d1ca16fa6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:30.849755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:30.971165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea02adf4a3947cfbb8faf8d1ca16fa6', 'request_id': 'req-f191344a-f003-46a1-ba41-a39e75d35ce6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:30.985539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea02adf4a3947cfbb8faf8d1ca16fa6', 'request_id': 'req-f191344a-f003-46a1-ba41-a39e75d35ce6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:30.987839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:31.002609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea02adf4a3947cfbb8faf8d1ca16fa6', 'request_id': 'req-f191344a-f003-46a1-ba41-a39e75d35ce6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.042418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] The token cache is being invalidate because project 4ea02adf4a3947cfbb8faf8d1ca16fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:31.043078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.044353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.056709 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3559/7159] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:30 2022] DELETE /identity/v3/projects/4ea02adf4a3947cfbb8faf8d1ca16fa6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:31.056962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:31.069473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9cf73-bcc7-4925-a880-bb98e8dcde8e service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:31.205645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:31.205645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:31.205841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:31.206059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] PATH_INFO: `/v3/users/43994f0e900645c1831b3144dd3c1e3c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:31.207372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43994f0e900645c1831b3144dd3c1e3c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:31.210418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:31.322760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43994f0e900645c1831b3144dd3c1e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.323836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43994f0e900645c1831b3144dd3c1e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.357335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43994f0e900645c1831b3144dd3c1e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.374842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43994f0e900645c1831b3144dd3c1e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.375769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-acdaf018-fb82-444c-abfb-bd202cb25860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43994f0e900645c1831b3144dd3c1e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:31.411950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:31.457739 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3601/7160] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:31 2022] DELETE /identity/v3/users/43994f0e900645c1831b3144dd3c1e3c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:32.183221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:32.184904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:32.204617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191344a-f003-46a1-ba41-a39e75d35ce6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:32.329354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:32.330044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:32.330323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:32.330616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] PATH_INFO: `/v3/projects/83c844a149364df5a53457931cea6f58` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:32.335116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c844a149364df5a53457931cea6f58)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:32.337148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:32.407944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c844a149364df5a53457931cea6f58', 'request_id': 'req-52318bef-79f7-4a57-9053-c9f1be38b1c6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:32.435375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c844a149364df5a53457931cea6f58', 'request_id': 'req-52318bef-79f7-4a57-9053-c9f1be38b1c6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:32.460577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c844a149364df5a53457931cea6f58', 'request_id': 'req-52318bef-79f7-4a57-9053-c9f1be38b1c6'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:32.483673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] The token cache is being invalidate because project 83c844a149364df5a53457931cea6f58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:32.484138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:32.486389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-52318bef-79f7-4a57-9053-c9f1be38b1c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:32.492178 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3560/7161] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:32 2022] DELETE /identity/v3/projects/83c844a149364df5a53457931cea6f58 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:32.496681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7856e36-ebbf-44b1-94ec-d51efd01daa6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:32.497238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7856e36-ebbf-44b1-94ec-d51efd01daa6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:32.497482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7856e36-ebbf-44b1-94ec-d51efd01daa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:32.536309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a7856e36-ebbf-44b1-94ec-d51efd01daa6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:32.537770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a7856e36-ebbf-44b1-94ec-d51efd01daa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:32.665522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3602/7162] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:24:32.678072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27488c-afb8-4eb6-826a-7eb60c060ac3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:32.678694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27488c-afb8-4eb6-826a-7eb60c060ac3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:32.679121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27488c-afb8-4eb6-826a-7eb60c060ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:32.724058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0c27488c-afb8-4eb6-826a-7eb60c060ac3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:32.725681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0c27488c-afb8-4eb6-826a-7eb60c060ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:32.757263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3561/7163] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:32.762198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7856e36-ebbf-44b1-94ec-d51efd01daa6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:32.780237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7856e36-ebbf-44b1-94ec-d51efd01daa6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:32.789179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:32.789910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:32.790338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:32.790847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:32.791310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:24:32.791596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:32.800251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:32.828628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3603/7164] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:24:32.832075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27488c-afb8-4eb6-826a-7eb60c060ac3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:32.857950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27488c-afb8-4eb6-826a-7eb60c060ac3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:32.869804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:32.870515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:32.870837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:32.871405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:32.872101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:32.875349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:32.970576 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3562/7165] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:32 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:32.975432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:32.989217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be781d37-c739-4ac2-92d8-214456bf0474 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:33.086931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:33.087702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:33.088134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:33.088315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:33.089460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:33.091811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:33.135021 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3604/7166] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:32 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:33.142366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:33.160724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bacce-a2c4-4f08-b633-2c4f41b19d8e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:33.162682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:33.162682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:33.162948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:33.163401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:33.164696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:33.165866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:33.181399 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3563/7167] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:33.187242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:33.212785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e402-983d-427b-9100-9fc7bd9f3401 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:33.217939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c4259-5357-4634-b057-7e33d90befc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:33.218705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c4259-5357-4634-b057-7e33d90befc6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:33.219169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c4259-5357-4634-b057-7e33d90befc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:33.219281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c4259-5357-4634-b057-7e33d90befc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:33.219987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c4259-5357-4634-b057-7e33d90befc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:33.221882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c4259-5357-4634-b057-7e33d90befc6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:33.238365 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3605/7168] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:33.241695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:33.254402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848020-7ac4-4a71-9688-f60e45c322de admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:33.261890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:33.262579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:33.263136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:33.263556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] PATH_INFO: `/v3/projects/47cb37050e9840688ff10a7c98c3e428/users/f558b8405a9a4364887860f54d4ce225/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:33.287565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=47cb37050e9840688ff10a7c98c3e428, user_id=f558b8405a9a4364887860f54d4ce225, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:33.292484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:33.319321 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3564/7169] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:33 2022] PUT /identity/v3/projects/47cb37050e9840688ff10a7c98c3e428/users/f558b8405a9a4364887860f54d4ce225/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:33.331635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f880-e1a6-4dff-ba0c-453ec66a4ac4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:33.331941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f880-e1a6-4dff-ba0c-453ec66a4ac4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:33.332172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f880-e1a6-4dff-ba0c-453ec66a4ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:33.378553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b9f7f880-e1a6-4dff-ba0c-453ec66a4ac4 None None] MFA Rules not processed for user `f558b8405a9a4364887860f54d4ce225`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:33.379835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b9f7f880-e1a6-4dff-ba0c-453ec66a4ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:33.447472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:33.524400 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3606/7170] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:33.529226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:33.550569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:33.691801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552dcb-40b8-44a9-aa54-d52515cd63fc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:33.717300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:33.718621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:33.718933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:33.719670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:33.721702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:33.725003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:33.794551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7f880-e1a6-4dff-ba0c-453ec66a4ac4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:33.797694 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3565/7171] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:33 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:33.813712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7f880-e1a6-4dff-ba0c-453ec66a4ac4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:33.935965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:33.936767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:33.937141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:33.937820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:33.938557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:33.940880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:33.976998 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3607/7172] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:33 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:33.979085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:33.997265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a81963-d1cb-4d90-967a-e3b0e6fc1587 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:34.005937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:34.006959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:34.007351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:34.007729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:34.008451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:34.011445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:34.028212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3566/7173] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:34.032484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:34.046374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665a53-b161-4987-bde5-910720d161df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:34.054262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107d922-1286-469a-bded-033d4875f873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:34.054899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107d922-1286-469a-bded-033d4875f873 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:34.055209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107d922-1286-469a-bded-033d4875f873 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:34.055479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107d922-1286-469a-bded-033d4875f873 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:34.056205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107d922-1286-469a-bded-033d4875f873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:34.058267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107d922-1286-469a-bded-033d4875f873 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:34.072275 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3608/7174] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:34.076591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:34.092488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe54dfc-db00-41ef-ad60-31a523338c1c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:34.102237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:34.103658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:34.103658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:34.103658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] PATH_INFO: `/v3/projects/7f5d67bbd36444e3ac2c7388742d8cc2/users/86a50fc12f864485a4d54a4f8dba5928/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:34.135325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5d67bbd36444e3ac2c7388742d8cc2, user_id=86a50fc12f864485a4d54a4f8dba5928, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:34.139040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:34.180377 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3567/7175] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:34 2022] PUT /identity/v3/projects/7f5d67bbd36444e3ac2c7388742d8cc2/users/86a50fc12f864485a4d54a4f8dba5928/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:34.197573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cc219-0d73-4997-8441-be14c716419c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:34.197573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cc219-0d73-4997-8441-be14c716419c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:34.197573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cc219-0d73-4997-8441-be14c716419c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:34.265519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c17cc219-0d73-4997-8441-be14c716419c None None] MFA Rules not processed for user `86a50fc12f864485a4d54a4f8dba5928`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:34.266775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c17cc219-0d73-4997-8441-be14c716419c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:34.481342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:34.482043 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3609/7176] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:34.643034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31463fb-9b32-43e3-9374-ddf31d946c38 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:34.659406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:34.660063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:34.660346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:34.660742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:34.661573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:34.663516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:34.723380 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3568/7177] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:34 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:24:34.728185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cc219-0d73-4997-8441-be14c716419c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:34.756713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cc219-0d73-4997-8441-be14c716419c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:34.891540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:34.927971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:34.930393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:34.930393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:34.930393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:34.930393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:34.934493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:34.990405 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3610/7178] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:34 2022] POST /identity/v3/users => generated 300 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:34.996522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:35.015251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c589488-cac8-46d6-b552-bc3a030fb449 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:35.028483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:35.029442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:35.030032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:35.030766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:35.033013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:35.036503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:35.058010 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3569/7179] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:35.071035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:35.088791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbdb15-d906-45f4-a176-bc97c58059f7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:35.096553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:35.096553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:35.096553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:35.096995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] PATH_INFO: `/v3/projects/4a9318dc53ee40cba63026dcb62716f9/users/088ebb5c33374c4d93ac76b00d998c7d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:35.133151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9318dc53ee40cba63026dcb62716f9, user_id=088ebb5c33374c4d93ac76b00d998c7d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:35.137626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:35.172372 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3611/7180] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:35 2022] PUT /identity/v3/projects/4a9318dc53ee40cba63026dcb62716f9/users/088ebb5c33374c4d93ac76b00d998c7d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:35.172631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:35.194914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba6540-bfc8-4d00-a309-c6a395dc04c3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:35.240756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:35.279760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8ee72-7aa4-4af3-b877-55a6364da745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:35.280650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ee72-7aa4-4af3-b877-55a6364da745 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:35.281161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ee72-7aa4-4af3-b877-55a6364da745 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:35.281422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8ee72-7aa4-4af3-b877-55a6364da745 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:35.282076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8ee72-7aa4-4af3-b877-55a6364da745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:35.284631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8ee72-7aa4-4af3-b877-55a6364da745 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:35.297874 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3570/7181] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:35.302003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:35.314454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e4bd0-314b-41f7-87e8-d928c35f1fc5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:35.321057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:35.321812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:35.322095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:35.322377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] PATH_INFO: `/v3/projects/4a9318dc53ee40cba63026dcb62716f9/users/088ebb5c33374c4d93ac76b00d998c7d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:35.325252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9318dc53ee40cba63026dcb62716f9, user_id=088ebb5c33374c4d93ac76b00d998c7d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:35.327460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:35.353502 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3612/7182] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:35 2022] PUT /identity/v3/projects/4a9318dc53ee40cba63026dcb62716f9/users/088ebb5c33374c4d93ac76b00d998c7d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:35.366057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be9311-ab9b-454a-a018-d8efbe084c5a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:35.366356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be9311-ab9b-454a-a018-d8efbe084c5a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:35.366645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be9311-ab9b-454a-a018-d8efbe084c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:35.411196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-27be9311-ab9b-454a-a018-d8efbe084c5a None None] MFA Rules not processed for user `088ebb5c33374c4d93ac76b00d998c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:35.412598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27be9311-ab9b-454a-a018-d8efbe084c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:35.541072 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3571/7183] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:35.551510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:35.560155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:35.688381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0a38e-31d1-44c9-83bc-f5b373d39939 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:35.693587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004f4ed-1495-4691-9b61-b6c6d67b7446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:35.694226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004f4ed-1495-4691-9b61-b6c6d67b7446 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:35.694502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004f4ed-1495-4691-9b61-b6c6d67b7446 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:35.694768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004f4ed-1495-4691-9b61-b6c6d67b7446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:35.707693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004f4ed-1495-4691-9b61-b6c6d67b7446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:35.710384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004f4ed-1495-4691-9b61-b6c6d67b7446 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:35.732907 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3613/7184] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:36.507905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be9311-ab9b-454a-a018-d8efbe084c5a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:36.521860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be9311-ab9b-454a-a018-d8efbe084c5a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:36.526360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:36.527324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:36.527849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:36.528320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:36.644609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:36.648625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:36.675705 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3572/7185] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:38.781942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb61e9-3432-4b07-9493-a59a1ee8a8c6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:38.782508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb61e9-3432-4b07-9493-a59a1ee8a8c6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:38.782508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb61e9-3432-4b07-9493-a59a1ee8a8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:38.808241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-87eb61e9-3432-4b07-9493-a59a1ee8a8c6 None None] MFA Rules not processed for user `584c395757a54d9b9a64dd399d8557b1`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:38.817485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87eb61e9-3432-4b07-9493-a59a1ee8a8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:38.916046 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3614/7186] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:41.516053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:41.517677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:41.536475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510da567-df71-4a59-8625-e1fd2662a719 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:41.541240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb0b7-d444-4f19-8330-62c57b46f4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:41.541872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb0b7-d444-4f19-8330-62c57b46f4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:41.542199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb0b7-d444-4f19-8330-62c57b46f4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:41.542522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb0b7-d444-4f19-8330-62c57b46f4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:41.619749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:41.639961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cb0b7-d444-4f19-8330-62c57b46f4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:41.642229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cb0b7-d444-4f19-8330-62c57b46f4e8 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:41.654971 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3573/7187] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:44.193958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb61e9-3432-4b07-9493-a59a1ee8a8c6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:44.270250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:44.277498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb61e9-3432-4b07-9493-a59a1ee8a8c6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:44.284388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:44.285133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:44.285722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:44.286190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] PATH_INFO: `/v3/users/f558b8405a9a4364887860f54d4ce225` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:44.287562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f558b8405a9a4364887860f54d4ce225)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:44.289805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:44.360933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f558b8405a9a4364887860f54d4ce225'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:44.361853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f558b8405a9a4364887860f54d4ce225'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:44.378811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f558b8405a9a4364887860f54d4ce225'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:44.409126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f558b8405a9a4364887860f54d4ce225'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:44.409969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f558b8405a9a4364887860f54d4ce225'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:44.451850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3615/7188] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:44 2022] DELETE /identity/v3/users/f558b8405a9a4364887860f54d4ce225 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:44.456943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:44.457557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:44.457557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:44.491458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:44.498301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:44.616445 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3574/7189] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:44.663424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:44.665061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:44.776123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6cbd-e2b6-48e7-a6c8-28b8149e74f2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:44.783069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:44.783970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:44.784315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:44.785611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:44.810587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:44.817374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:44.829281 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3616/7190] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:45.841790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:45.844037 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:45.871724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924df513-276f-4397-aa7d-1390052d90d9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:45.881380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:45.881380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:45.881753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:45.881848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] PATH_INFO: `/v3/projects/47cb37050e9840688ff10a7c98c3e428` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:45.883021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47cb37050e9840688ff10a7c98c3e428)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:45.885247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:45.973308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47cb37050e9840688ff10a7c98c3e428', 'request_id': 'req-99aa07e1-a95f-4e4c-8afe-b96f31bff787'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:45.987899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47cb37050e9840688ff10a7c98c3e428', 'request_id': 'req-99aa07e1-a95f-4e4c-8afe-b96f31bff787'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.010345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47cb37050e9840688ff10a7c98c3e428', 'request_id': 'req-99aa07e1-a95f-4e4c-8afe-b96f31bff787'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.059791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] The token cache is being invalidate because project 47cb37050e9840688ff10a7c98c3e428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:46.061119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.063343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.070684 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3575/7191] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:45 2022] DELETE /identity/v3/projects/47cb37050e9840688ff10a7c98c3e428 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:46.070877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:46.087130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154bca9-c120-4e8a-8689-f0c5c16d10b2 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:46.247412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:46.248449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:46.248866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:46.249318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] PATH_INFO: `/v3/users/86a50fc12f864485a4d54a4f8dba5928` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:46.250522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a50fc12f864485a4d54a4f8dba5928)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:46.252580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:46.284752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:46.320848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a50fc12f864485a4d54a4f8dba5928'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.321733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a50fc12f864485a4d54a4f8dba5928'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.332517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a50fc12f864485a4d54a4f8dba5928'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.342823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a50fc12f864485a4d54a4f8dba5928'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.343884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a50fc12f864485a4d54a4f8dba5928'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:46.377032 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3617/7192] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:46 2022] DELETE /identity/v3/users/86a50fc12f864485a4d54a4f8dba5928 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:46.963916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:46.966220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:46.986472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa07e1-a95f-4e4c-8afe-b96f31bff787 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:47.055941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:47.056591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:47.057036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:47.057381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] PATH_INFO: `/v3/projects/7f5d67bbd36444e3ac2c7388742d8cc2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:47.058549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5d67bbd36444e3ac2c7388742d8cc2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:47.060562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:47.078912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:47.117248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5d67bbd36444e3ac2c7388742d8cc2', 'request_id': 'req-2b4abac3-8333-4383-b532-bd9460b4e02b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.125292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5d67bbd36444e3ac2c7388742d8cc2', 'request_id': 'req-2b4abac3-8333-4383-b532-bd9460b4e02b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.136906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5d67bbd36444e3ac2c7388742d8cc2', 'request_id': 'req-2b4abac3-8333-4383-b532-bd9460b4e02b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.160087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] The token cache is being invalidate because project 7f5d67bbd36444e3ac2c7388742d8cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:47.160439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.161265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.162822 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3576/7193] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:46 2022] DELETE /identity/v3/projects/7f5d67bbd36444e3ac2c7388742d8cc2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:47.167112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:47.169192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:47.183349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f2c8a-7be9-48b8-a86f-06bb9bd3fae3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:47.256338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:47.257280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:47.257609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:47.257953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] PATH_INFO: `/v3/users/088ebb5c33374c4d93ac76b00d998c7d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:47.259231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=088ebb5c33374c4d93ac76b00d998c7d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:47.262786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:47.330969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088ebb5c33374c4d93ac76b00d998c7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.331742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088ebb5c33374c4d93ac76b00d998c7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.355046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088ebb5c33374c4d93ac76b00d998c7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.371238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088ebb5c33374c4d93ac76b00d998c7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.372028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e42a972-e036-4e35-9c2e-d45b2287b557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088ebb5c33374c4d93ac76b00d998c7d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:47.405345 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3618/7194] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:47 2022] DELETE /identity/v3/users/088ebb5c33374c4d93ac76b00d998c7d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:24:47.818584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:47.820614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:47.839802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4abac3-8333-4383-b532-bd9460b4e02b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:47.940432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:47.941160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:47.941439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:47.941746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] PATH_INFO: `/v3/projects/4a9318dc53ee40cba63026dcb62716f9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:47.942903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9318dc53ee40cba63026dcb62716f9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:47.944852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:47.996409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9318dc53ee40cba63026dcb62716f9', 'request_id': 'req-8af1db1f-01d0-49f0-b59e-e5d883587c1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:48.003016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9318dc53ee40cba63026dcb62716f9', 'request_id': 'req-8af1db1f-01d0-49f0-b59e-e5d883587c1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:48.011698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9318dc53ee40cba63026dcb62716f9', 'request_id': 'req-8af1db1f-01d0-49f0-b59e-e5d883587c1d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:48.022480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] The token cache is being invalidate because project 4a9318dc53ee40cba63026dcb62716f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:48.022774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:48.023408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8af1db1f-01d0-49f0-b59e-e5d883587c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:48.024972 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3577/7195] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:47 2022] DELETE /identity/v3/projects/4a9318dc53ee40cba63026dcb62716f9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:48.030969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e941ca-e7b9-4989-8106-ff5c6cdc3b28 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.031855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e941ca-e7b9-4989-8106-ff5c6cdc3b28 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.032156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e941ca-e7b9-4989-8106-ff5c6cdc3b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.052314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-56e941ca-e7b9-4989-8106-ff5c6cdc3b28 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:48.053547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-56e941ca-e7b9-4989-8106-ff5c6cdc3b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:48.067409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:48.117043 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3619/7196] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:48.126284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a40a3-8350-4293-ad45-26dc74238fe5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.126721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a40a3-8350-4293-ad45-26dc74238fe5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.126906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a40a3-8350-4293-ad45-26dc74238fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.148687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a79a40a3-8350-4293-ad45-26dc74238fe5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:48.150151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a79a40a3-8350-4293-ad45-26dc74238fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:48.159211 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3578/7197] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:48.163761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e941ca-e7b9-4989-8106-ff5c6cdc3b28 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.173912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e941ca-e7b9-4989-8106-ff5c6cdc3b28 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.194703 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.195591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.195865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.196294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.197002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:24:48.197489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.199280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.213538 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3620/7198] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:24:48.218267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a40a3-8350-4293-ad45-26dc74238fe5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.230390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a40a3-8350-4293-ad45-26dc74238fe5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.239010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.239839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.240271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.240748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.241681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.243672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.278067 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3579/7199] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:48 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:24:48.282602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.295317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad3948-47f7-4f8b-98d6-66e6ca0107dd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.369614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.370526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.370957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.371400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.372276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.374443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.408619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3621/7200] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:48 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:48.414255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.429897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f19d-3d54-4f32-b1b1-3b23b2ca1ceb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.434636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.435827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.435827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.435827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.436617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.438592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.440294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:48.456746 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3580/7201] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:48.457845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.471603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66326d9-2454-41fa-8d73-fe4e0725bf7f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.477471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926eeb8-ac54-4484-8cec-780f2cce5332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.478113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926eeb8-ac54-4484-8cec-780f2cce5332 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.478395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926eeb8-ac54-4484-8cec-780f2cce5332 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.478668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926eeb8-ac54-4484-8cec-780f2cce5332 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.479322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926eeb8-ac54-4484-8cec-780f2cce5332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.480995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926eeb8-ac54-4484-8cec-780f2cce5332 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.495257 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3622/7202] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:24:48.499204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.510515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dd96a-7d52-4b01-aae1-19dbe7476324 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.516172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.516959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.517237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.517509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] PATH_INFO: `/v3/projects/f633659fc9eb47f6bdb3ce49287650b2/users/2a5957e42b6c4d9fb0187fec69c00d56/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.535699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f633659fc9eb47f6bdb3ce49287650b2, user_id=2a5957e42b6c4d9fb0187fec69c00d56, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.539025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.557966 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3581/7203] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:48 2022] PUT /identity/v3/projects/f633659fc9eb47f6bdb3ce49287650b2/users/2a5957e42b6c4d9fb0187fec69c00d56/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:48.570022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971b90b-a206-4139-9574-c52484a6f280 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.575904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971b90b-a206-4139-9574-c52484a6f280 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.576337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971b90b-a206-4139-9574-c52484a6f280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.609288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.617868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:48.629269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8971b90b-a206-4139-9574-c52484a6f280 None None] MFA Rules not processed for user `2a5957e42b6c4d9fb0187fec69c00d56`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:48.630576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8971b90b-a206-4139-9574-c52484a6f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:48.721122 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3623/7204] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:48.725919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971b90b-a206-4139-9574-c52484a6f280 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:48.736921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e7f15-f2c3-4bc4-a470-e1aeee0f2b7e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.743403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.744045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.744474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.744940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.747147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:48.755675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:48.868975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.871308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.883975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971b90b-a206-4139-9574-c52484a6f280 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:48.889930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:48.890559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:48.891304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:48.891803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:48.892526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:48.894492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:48.902196 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3582/7205] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:48.980537 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3624/7206] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:48 2022] POST /identity/v3/projects => generated 377 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:48.981772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:49.002766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880236ae-dc0e-426a-8b46-aeeace26e2ba service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:49.129841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:49.134156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:49.135791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:49.135791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:49.136735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:49.138868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:49.178169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:49.203620 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3583/7207] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:48 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:49.205225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:49.221217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063a5a7-b741-4933-b879-502107335e43 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:49.233061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:49.233977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:49.234411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:49.234892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:49.235735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:49.237592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:49.265406 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3625/7208] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:49.266430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:49.283727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f70b3-31b4-4686-af4a-ef477131ccd0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:49.299493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f8365-8a6a-4dd4-81aa-6139e58cfa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:49.300434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f8365-8a6a-4dd4-81aa-6139e58cfa83 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:49.300860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f8365-8a6a-4dd4-81aa-6139e58cfa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:49.302068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f8365-8a6a-4dd4-81aa-6139e58cfa83 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:49.302822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f8365-8a6a-4dd4-81aa-6139e58cfa83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:49.308871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f8365-8a6a-4dd4-81aa-6139e58cfa83 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:49.341648 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3584/7209] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:24:49.342963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:49.359141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aabb02-5c9f-485f-9542-1317dbfc7d23 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:49.389392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:49.391545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:49.392350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:49.392823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] PATH_INFO: `/v3/projects/ca1d879791044a5980d28a7ddc7d0d5e/users/2a270cb922424cf484f2d1bc84122491/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:49.431111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d879791044a5980d28a7ddc7d0d5e, user_id=2a270cb922424cf484f2d1bc84122491, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:49.434902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:49.470216 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3626/7210] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:49 2022] PUT /identity/v3/projects/ca1d879791044a5980d28a7ddc7d0d5e/users/2a270cb922424cf484f2d1bc84122491/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:49.473542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0627-efd9-481d-b79e-0d496e29aeac None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:49.474112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0627-efd9-481d-b79e-0d496e29aeac None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:49.474487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0627-efd9-481d-b79e-0d496e29aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:49.530533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-897b0627-efd9-481d-b79e-0d496e29aeac None None] MFA Rules not processed for user `2a270cb922424cf484f2d1bc84122491`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:49.531788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-897b0627-efd9-481d-b79e-0d496e29aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:49.686271 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3585/7211] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:49.691027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:49.774567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e2cd2-cf65-4077-928d-a07290854d0b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:49.781914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:49.782835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:49.782835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:49.783117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:49.783721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:49.785950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:49.843733 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3627/7212] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:24:49 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:49.848086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0627-efd9-481d-b79e-0d496e29aeac None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:49.869707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0627-efd9-481d-b79e-0d496e29aeac None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:50.000073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:50.000845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.001199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.001574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.002324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:50.006687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:50.054543 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3586/7213] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:24:49 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:24:50.060577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:50.075674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f54148-7a2c-4a1a-be6a-403df2a8c74d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:50.081858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:50.083516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.084022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.084921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.085685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:50.087416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:50.113455 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3628/7214] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:24:50.117156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:50.131594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994e30-ddc7-4349-ad8a-66ba5abe38ac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:50.144279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:50.146713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.147115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.147531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] PATH_INFO: `/v3/projects/8e91b370da9a4fc3bbd2e7acabdb614a/users/1e884ee5463240fb89c59af368d2dbce/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.177902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e91b370da9a4fc3bbd2e7acabdb614a, user_id=1e884ee5463240fb89c59af368d2dbce, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:50.184554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:50.211084 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3587/7215] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:50 2022] PUT /identity/v3/projects/8e91b370da9a4fc3bbd2e7acabdb614a/users/1e884ee5463240fb89c59af368d2dbce/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:50.211668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:50.226257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdba2b-fe6a-45b9-a174-551a64ac4971 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:50.293175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:50.351462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a5d8c-023a-437f-9599-67742026188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:50.352182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5d8c-023a-437f-9599-67742026188a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.352376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5d8c-023a-437f-9599-67742026188a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.352736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5d8c-023a-437f-9599-67742026188a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.353758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a5d8c-023a-437f-9599-67742026188a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:50.355386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a5d8c-023a-437f-9599-67742026188a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:50.372571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:50.383775 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3629/7216] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:24:50.385845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bbc11-c178-4802-aa6e-4e812516c6d8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:50.391700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:50.392395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.392872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.393159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] PATH_INFO: `/v3/projects/8e91b370da9a4fc3bbd2e7acabdb614a/users/1e884ee5463240fb89c59af368d2dbce/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.397967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e91b370da9a4fc3bbd2e7acabdb614a, user_id=1e884ee5463240fb89c59af368d2dbce, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:50.400838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:50.420775 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3588/7217] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:24:50 2022] PUT /identity/v3/projects/8e91b370da9a4fc3bbd2e7acabdb614a/users/1e884ee5463240fb89c59af368d2dbce/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:50.430680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.430967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.431232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.475222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] MFA Rules not processed for user `1e884ee5463240fb89c59af368d2dbce`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:50.476459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:50.564726 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3630/7218] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:50.579026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:50.581752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:50.633249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:50.665298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c64054-0c1e-49ab-b22c-2dccb360f744 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:50.669466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617e3b8-7a57-423f-9cb2-a33853192969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:50.670200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617e3b8-7a57-423f-9cb2-a33853192969 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.670356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617e3b8-7a57-423f-9cb2-a33853192969 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.670630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617e3b8-7a57-423f-9cb2-a33853192969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.680497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617e3b8-7a57-423f-9cb2-a33853192969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:50.681978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617e3b8-7a57-423f-9cb2-a33853192969 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:50.692687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3589/7219] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:50.736323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:50.738031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:50.752500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc796b-b2e0-486b-ac9d-c80e9b3d6154 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:50.756378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:50.757023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:50.757303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:50.757570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:50.821113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:50.822636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:50.839140 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3631/7220] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:55.255604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53c3b2-6e36-47ad-84dd-4770aacd3ca9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:55.256128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53c3b2-6e36-47ad-84dd-4770aacd3ca9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:55.256332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53c3b2-6e36-47ad-84dd-4770aacd3ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:55.280835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4c53c3b2-6e36-47ad-84dd-4770aacd3ca9 None None] MFA Rules not processed for user `178956f6c8154a06a0710b295aff3086`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:55.282052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4c53c3b2-6e36-47ad-84dd-4770aacd3ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:55.392241 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3590/7221] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:55.410878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:55.560087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:55.569663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353510ea-5e0a-45f4-b7a2-ac8bbeb013b7 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:55.583709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:55.584485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:55.585191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:55.585537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] PATH_INFO: `/v3/users/2a5957e42b6c4d9fb0187fec69c00d56` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:55.587290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5957e42b6c4d9fb0187fec69c00d56)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:55.589687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:55.723996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5957e42b6c4d9fb0187fec69c00d56'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:55.724806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5957e42b6c4d9fb0187fec69c00d56'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:55.746294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5957e42b6c4d9fb0187fec69c00d56'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:55.760699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5957e42b6c4d9fb0187fec69c00d56'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:55.761930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5957e42b6c4d9fb0187fec69c00d56'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:55.794447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3632/7222] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:55 2022] DELETE /identity/v3/users/2a5957e42b6c4d9fb0187fec69c00d56 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:55.807930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:55.808270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:55.808553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:55.855279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:55.857678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:55.952794 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3591/7223] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:55.960286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:55.964337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:56.113498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffff9c8-89d1-4c66-8aff-ff5a62218002 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:56.125196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:56.126209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:56.126873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:56.127408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:56.141323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:56.144213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:56.162327 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3633/7224] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:24:56.770186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:56.771813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:56.790955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb6e0-fb21-42ea-a0f2-c7b596b3b41a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:56.806798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:56.807493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:56.807984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:56.808430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] PATH_INFO: `/v3/projects/f633659fc9eb47f6bdb3ce49287650b2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:56.824572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f633659fc9eb47f6bdb3ce49287650b2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:56.828131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:56.929355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f633659fc9eb47f6bdb3ce49287650b2', 'request_id': 'req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:56.946802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f633659fc9eb47f6bdb3ce49287650b2', 'request_id': 'req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:56.972385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f633659fc9eb47f6bdb3ce49287650b2', 'request_id': 'req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:56.987638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:56.992868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] The token cache is being invalidate because project f633659fc9eb47f6bdb3ce49287650b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:56.993524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:57.007901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a9bbbc83-7658-4e2b-b4c1-11018c5070ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:57.012919 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3592/7225] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:56 2022] DELETE /identity/v3/projects/f633659fc9eb47f6bdb3ce49287650b2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:57.016878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:57.028464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59bdb3-9968-4c50-9e41-bd49e8efff1d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:57.119743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:57.120443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:57.124390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:57.124778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] PATH_INFO: `/v3/users/2a270cb922424cf484f2d1bc84122491` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:57.126062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a270cb922424cf484f2d1bc84122491)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:57.128541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:57.164772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:57.216154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a270cb922424cf484f2d1bc84122491'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:57.217472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a270cb922424cf484f2d1bc84122491'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:57.238748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a270cb922424cf484f2d1bc84122491'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:57.251528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a270cb922424cf484f2d1bc84122491'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:57.254391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a270cb922424cf484f2d1bc84122491'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:57.279846 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3634/7226] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:57 2022] DELETE /identity/v3/users/2a270cb922424cf484f2d1bc84122491 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:58.265820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba239e-a7fa-46a8-ab94-2944c3274779 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:58.266328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba239e-a7fa-46a8-ab94-2944c3274779 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:58.266602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba239e-a7fa-46a8-ab94-2944c3274779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:58.301735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7fba239e-a7fa-46a8-ab94-2944c3274779 None None] MFA Rules not processed for user `97c769dbba794531b0a9db8e764ff097`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:24:58.304400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7fba239e-a7fa-46a8-ab94-2944c3274779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:58.408152 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3593/7227] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:24:58.755076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:58.762913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:24:58.872054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee80c99-3e23-48ce-921a-26fc79a76a6d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:58.877817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:58.878685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:58.879305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:58.879826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] PATH_INFO: `/v3/projects/ca1d879791044a5980d28a7ddc7d0d5e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:58.882093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1d879791044a5980d28a7ddc7d0d5e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:58.884314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:58.984447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1d879791044a5980d28a7ddc7d0d5e', 'request_id': 'req-5f06b737-8135-45ec-9c22-e6e5542f3504'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.013765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1d879791044a5980d28a7ddc7d0d5e', 'request_id': 'req-5f06b737-8135-45ec-9c22-e6e5542f3504'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.028235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1d879791044a5980d28a7ddc7d0d5e', 'request_id': 'req-5f06b737-8135-45ec-9c22-e6e5542f3504'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.068450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] The token cache is being invalidate because project ca1d879791044a5980d28a7ddc7d0d5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:24:59.068772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.069662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.071739 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3635/7228] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:24:58 2022] DELETE /identity/v3/projects/ca1d879791044a5980d28a7ddc7d0d5e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:24:59.077372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba239e-a7fa-46a8-ab94-2944c3274779 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:24:59.112877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba239e-a7fa-46a8-ab94-2944c3274779 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:24:59.238732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:24:59.240614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:24:59.240944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:24:59.241216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] PATH_INFO: `/v3/users/1e884ee5463240fb89c59af368d2dbce` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:24:59.242620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e884ee5463240fb89c59af368d2dbce)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:24:59.244638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:24:59.337860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e884ee5463240fb89c59af368d2dbce'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.340135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e884ee5463240fb89c59af368d2dbce'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.357589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e884ee5463240fb89c59af368d2dbce'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.376915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e884ee5463240fb89c59af368d2dbce'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.399600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c46bf92c-361e-42f7-8f7a-263401bddde7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e884ee5463240fb89c59af368d2dbce'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:24:59.407796 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3594/7229] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:24:59 2022] DELETE /identity/v3/users/1e884ee5463240fb89c59af368d2dbce => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:25:00.473922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:00.476190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:00.500832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06b737-8135-45ec-9c22-e6e5542f3504 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:00.579742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:00.695861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:00.696528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:00.696834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:00.697105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] PATH_INFO: `/v3/projects/8e91b370da9a4fc3bbd2e7acabdb614a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:00.698248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e91b370da9a4fc3bbd2e7acabdb614a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:00.700169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:00.794731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e91b370da9a4fc3bbd2e7acabdb614a', 'request_id': 'req-5ee6dbd4-f765-4c0a-8893-804344557501'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:00.807498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e91b370da9a4fc3bbd2e7acabdb614a', 'request_id': 'req-5ee6dbd4-f765-4c0a-8893-804344557501'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:00.816958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e91b370da9a4fc3bbd2e7acabdb614a', 'request_id': 'req-5ee6dbd4-f765-4c0a-8893-804344557501'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:00.843648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] The token cache is being invalidate because project 8e91b370da9a4fc3bbd2e7acabdb614a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:25:00.846885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:00.847996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5ee6dbd4-f765-4c0a-8893-804344557501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:00.851362 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3636/7230] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:25:00 2022] DELETE /identity/v3/projects/8e91b370da9a4fc3bbd2e7acabdb614a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:25:00.857482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061524f-2f2c-4349-be62-a11ddb8b95c9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:00.857972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061524f-2f2c-4349-be62-a11ddb8b95c9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:00.858064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061524f-2f2c-4349-be62-a11ddb8b95c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:00.918327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8061524f-2f2c-4349-be62-a11ddb8b95c9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:00.923480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8061524f-2f2c-4349-be62-a11ddb8b95c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:00.977161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:01.108463 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3595/7231] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:01.116299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a84e8-fe0b-48d8-9996-01268cf1ed55 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.116580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a84e8-fe0b-48d8-9996-01268cf1ed55 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.116964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a84e8-fe0b-48d8-9996-01268cf1ed55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.158049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e64a84e8-fe0b-48d8-9996-01268cf1ed55 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:01.159160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e64a84e8-fe0b-48d8-9996-01268cf1ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:01.180428 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3637/7232] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:01.185454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061524f-2f2c-4349-be62-a11ddb8b95c9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:01.202384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061524f-2f2c-4349-be62-a11ddb8b95c9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:01.216913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:01.217562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.217847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.218128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.218802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:25:01.219103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:01.220757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:01.237275 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3596/7233] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:25:01.242291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a84e8-fe0b-48d8-9996-01268cf1ed55 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:01.253412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a84e8-fe0b-48d8-9996-01268cf1ed55 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:01.259526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:01.260743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.261218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.261629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.262429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:01.264565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:01.304871 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3638/7234] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:25:01 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:25:01.308945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:01.324652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62d3a-787f-4ac1-a344-04d809539ecd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:01.461578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:01.462698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.463449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.464014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.464903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:01.467600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:01.511036 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3597/7235] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:25:01 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:25:01.513308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:01.526290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945da5d-fc43-491e-9733-a148a6b8cf0e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:01.562357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:01.562984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.563265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.563539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.564201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:01.565911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:01.581912 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3639/7236] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:25:01.595904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:01.637834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a3301-c144-4936-99d2-4003633b9c83 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:01.648275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e4e2d-9e4e-4aee-a1e2-b4dde88f7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:01.649273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4e2d-9e4e-4aee-a1e2-b4dde88f7b9b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.649622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4e2d-9e4e-4aee-a1e2-b4dde88f7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.650136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4e2d-9e4e-4aee-a1e2-b4dde88f7b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.651006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e4e2d-9e4e-4aee-a1e2-b4dde88f7b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:01.652908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e4e2d-9e4e-4aee-a1e2-b4dde88f7b9b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:01.668416 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3598/7237] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:25:01.672922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:01.686251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e40b03-7b46-4e37-a009-76a61efebd88 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:01.695354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:01.696204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.696422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.696735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] PATH_INFO: `/v3/projects/430555edbc87497db7d7f79f2b891c04/users/fbcfed06aa6b4177a4287064884125b2/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.718497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=430555edbc87497db7d7f79f2b891c04, user_id=fbcfed06aa6b4177a4287064884125b2, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:01.722866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:01.744213 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3640/7238] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:25:01 2022] PUT /identity/v3/projects/430555edbc87497db7d7f79f2b891c04/users/fbcfed06aa6b4177a4287064884125b2/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:25:01.757202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593500cb-6c2c-45bf-91ae-9491fa7da325 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:01.758555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593500cb-6c2c-45bf-91ae-9491fa7da325 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:01.758986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593500cb-6c2c-45bf-91ae-9491fa7da325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:01.795427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-593500cb-6c2c-45bf-91ae-9491fa7da325 None None] MFA Rules not processed for user `fbcfed06aa6b4177a4287064884125b2`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:01.797995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-593500cb-6c2c-45bf-91ae-9491fa7da325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:01.933816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3599/7239] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:01.946564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:01.948189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:02.067100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b5933-d745-4285-9558-243907454a0b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:02.081758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b06f2e-42d6-4cb0-a56d-99f9f9ef8d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:02.081758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b06f2e-42d6-4cb0-a56d-99f9f9ef8d05 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:02.082071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b06f2e-42d6-4cb0-a56d-99f9f9ef8d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:02.082290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b06f2e-42d6-4cb0-a56d-99f9f9ef8d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:02.120373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b06f2e-42d6-4cb0-a56d-99f9f9ef8d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:02.124542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b06f2e-42d6-4cb0-a56d-99f9f9ef8d05 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:02.137725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:02.163660 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3641/7240] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:25:04.057629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:04.059553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:04.060007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:04.090984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] MFA Rules not processed for user `fbcfed06aa6b4177a4287064884125b2`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:04.092346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:04.101879 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3600/7241] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:10.794499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76904b-9bb6-449b-9ebb-516d38f87489 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:10.795426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76904b-9bb6-449b-9ebb-516d38f87489 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:10.796256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76904b-9bb6-449b-9ebb-516d38f87489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:10.826776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-df76904b-9bb6-449b-9ebb-516d38f87489 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:10.827614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df76904b-9bb6-449b-9ebb-516d38f87489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:10.951097 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3642/7242] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:25:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:25:11.027908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:11.030551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:11.051832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:11.176878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea3be3-c08d-44d0-8972-c6375c4cc01d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:11.182984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:11.183651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:11.184000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:11.184341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:11.206126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:11.211812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:11.230638 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3601/7243] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:25:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:25:20.919573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=29 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:21.220587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=26 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:23.204804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b7704-d424-47fb-8f03-e08d62f7154b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:23.205352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b7704-d424-47fb-8f03-e08d62f7154b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:23.205352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b7704-d424-47fb-8f03-e08d62f7154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:23.249295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-064b7704-d424-47fb-8f03-e08d62f7154b None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:23.250998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-064b7704-d424-47fb-8f03-e08d62f7154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:23.273040 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3643/7244] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:23.282529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:23.301761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836e6b7-7c26-420a-9dba-d1c9c8a4041d service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:23.311161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:23.312461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:23.312973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:23.313688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:23.337944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:23.340588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:23.378491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3602/7245] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:25:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:25:24.847033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:24.847510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:24.847888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:24.919410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] MFA Rules not processed for user `178956f6c8154a06a0710b295aff3086`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:24.920926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:25.069194 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3644/7246] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:33.372871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:33.874526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:33.876161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:33.975872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e01ef-4c3c-40cf-8d49-3cf7f3ab67c7 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:33.987645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:33.988417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:34.001437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:34.001732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:34.007694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:34.030905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:34.033985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:34.068914 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3603/7247] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:25:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:25:35.050115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=28 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:36.184754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:36.187079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:36.280725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b031a7-c39c-4570-ae6f-567f20e93721 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:36.285906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:36.286836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:36.287111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:36.287606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:36.289320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:36.401702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:36.403790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:36.419634 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3645/7248] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:25:36.512217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:36.513845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:36.633137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4298d5-5d28-4a7b-8354-4b6d14d3da1e service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:36.640387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2cba7-6ec5-41ba-bcc9-e6c8956b020b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:36.641572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2cba7-6ec5-41ba-bcc9-e6c8956b020b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:36.642112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2cba7-6ec5-41ba-bcc9-e6c8956b020b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:36.642687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2cba7-6ec5-41ba-bcc9-e6c8956b020b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:36.650368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a6a2cba7-6ec5-41ba-bcc9-e6c8956b020b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:36.672872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2cba7-6ec5-41ba-bcc9-e6c8956b020b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:36.675909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2cba7-6ec5-41ba-bcc9-e6c8956b020b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:36.695762 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3604/7249] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:25:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:25:36.834508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:36.855128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4141-9ba9-4f2e-bf55-6e8c49080fe2 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:36.865765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:36.866726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:36.867001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:36.867290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:36.869511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:36.891214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:36.892840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:36.908699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3646/7250] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:25:41.477897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:41.478503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:41.478503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:41.521162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:41.522530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:41.533209 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3605/7251] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:25:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:41.552390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:41.579769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf031f-f996-4d28-b3f0-a391743a65da service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:41.593707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7b960-b949-4d45-9e55-9456b037610e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:41.594452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7b960-b949-4d45-9e55-9456b037610e service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:41.595636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7b960-b949-4d45-9e55-9456b037610e service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:41.596121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7b960-b949-4d45-9e55-9456b037610e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:41.677921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7b960-b949-4d45-9e55-9456b037610e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:41.683214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7b960-b949-4d45-9e55-9456b037610e service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:41.752081 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3647/7252] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:25:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:25:50.302989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:50.307948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:50.419776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db606191-8819-4b0d-a5ec-4be909dbe6c2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:50.426378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:50.427238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:50.427708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:50.428016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] PATH_INFO: `/v3/users/604bb3b24e0d404f9daa9ce5855a34bc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:50.429689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=604bb3b24e0d404f9daa9ce5855a34bc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:50.432018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:50.473420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:50.502629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604bb3b24e0d404f9daa9ce5855a34bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:50.504083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604bb3b24e0d404f9daa9ce5855a34bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:50.514949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604bb3b24e0d404f9daa9ce5855a34bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:50.526975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604bb3b24e0d404f9daa9ce5855a34bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:50.544705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604bb3b24e0d404f9daa9ce5855a34bc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:50.552527 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3606/7253] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:25:50 2022] DELETE /identity/v3/users/604bb3b24e0d404f9daa9ce5855a34bc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:25:50.560148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:50.560752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:50.561130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:50.603811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:50.605497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:50.695804 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3648/7254] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:50.708933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:50.712018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:50.877111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959faed1-29a9-496e-a2c1-43f84db34fe6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:50.881601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602d4c0-3efa-49bc-a43e-4fc28429f094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:50.882240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602d4c0-3efa-49bc-a43e-4fc28429f094 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:50.882524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602d4c0-3efa-49bc-a43e-4fc28429f094 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:50.882851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602d4c0-3efa-49bc-a43e-4fc28429f094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:50.913775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602d4c0-3efa-49bc-a43e-4fc28429f094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:50.915267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602d4c0-3efa-49bc-a43e-4fc28429f094 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:50.944904 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3607/7255] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:25:51.390580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:51.392351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:51.413792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:51.421989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802507cc-8c0c-41ff-929b-c31202c2de5c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:51.438937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:51.440272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:51.440563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:51.441310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] PATH_INFO: `/v3/projects/14a715df3201452e83b3cd98bcfcdd85` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:51.442732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a715df3201452e83b3cd98bcfcdd85)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:51.449250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:51.551451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a715df3201452e83b3cd98bcfcdd85', 'request_id': 'req-42d607b3-5bfb-46f2-bb55-5c5168049b48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:51.565973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a715df3201452e83b3cd98bcfcdd85', 'request_id': 'req-42d607b3-5bfb-46f2-bb55-5c5168049b48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:51.576832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a715df3201452e83b3cd98bcfcdd85', 'request_id': 'req-42d607b3-5bfb-46f2-bb55-5c5168049b48'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:51.622737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] The token cache is being invalidate because project 14a715df3201452e83b3cd98bcfcdd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:25:51.623582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:51.624622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-42d607b3-5bfb-46f2-bb55-5c5168049b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:25:51.627099 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3649/7256] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:25:51 2022] DELETE /identity/v3/projects/14a715df3201452e83b3cd98bcfcdd85 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:25:51.634959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589f375-bd62-4c91-99f9-fb0e78a5a543 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:51.635469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589f375-bd62-4c91-99f9-fb0e78a5a543 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:51.635619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589f375-bd62-4c91-99f9-fb0e78a5a543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:51.698604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1589f375-bd62-4c91-99f9-fb0e78a5a543 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:51.700274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1589f375-bd62-4c91-99f9-fb0e78a5a543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:51.852761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3608/7257] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:51.861011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d62dd-1424-446e-8147-33ef6b31c18a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:51.861505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d62dd-1424-446e-8147-33ef6b31c18a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:51.861664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d62dd-1424-446e-8147-33ef6b31c18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:51.887271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ed8d62dd-1424-446e-8147-33ef6b31c18a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:51.889246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ed8d62dd-1424-446e-8147-33ef6b31c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:51.902161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3650/7258] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:51.907309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589f375-bd62-4c91-99f9-fb0e78a5a543 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:51.925149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589f375-bd62-4c91-99f9-fb0e78a5a543 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:51.931745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:51.933178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:51.933664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:51.934387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:51.935423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:25:51.936070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:51.938547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:51.965964 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3609/7259] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:25:51.966613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d62dd-1424-446e-8147-33ef6b31c18a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:51.981403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d62dd-1424-446e-8147-33ef6b31c18a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:51.988587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:51.989634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:51.990062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:51.990513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:51.992467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:51.996485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:52.063542 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3651/7260] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:25:51 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:25:52.068910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:52.081562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a48b2-b3a3-42ca-bf7f-d2177af73962 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:52.133332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:52.177206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:52.178067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:52.178339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:52.178668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:52.179303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:52.181704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:52.233152 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3610/7261] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:25:52 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:25:52.239389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:52.259934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d768e-0c69-46b4-8027-7c9e01780348 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:52.285852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:52.288802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:52.289787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:52.290875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:52.291575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:52.298977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:52.334673 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3652/7262] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:25:52.340267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:52.359829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0eefe-0229-4f97-b73b-41c6d382cea9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:52.373667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363265-15eb-42d0-bac4-fa05abbd388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:52.374458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363265-15eb-42d0-bac4-fa05abbd388b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:52.375102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363265-15eb-42d0-bac4-fa05abbd388b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:52.375566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363265-15eb-42d0-bac4-fa05abbd388b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:52.378326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363265-15eb-42d0-bac4-fa05abbd388b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:52.382975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363265-15eb-42d0-bac4-fa05abbd388b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:52.408778 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3611/7263] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:25:52.413383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:52.451280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7d6e-ea4d-488e-8f37-72a02b61006d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:52.463828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:52.464527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:52.464917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:52.465203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] PATH_INFO: `/v3/projects/fb6c2eb66a3449c6b4b5065ff5ace428/users/b1531c5af9874d12a28dad2f2313dd94/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:52.503713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6c2eb66a3449c6b4b5065ff5ace428, user_id=b1531c5af9874d12a28dad2f2313dd94, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:52.519059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:52.544931 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3653/7264] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:25:52 2022] PUT /identity/v3/projects/fb6c2eb66a3449c6b4b5065ff5ace428/users/b1531c5af9874d12a28dad2f2313dd94/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:25:52.569100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251099e-ffdc-4de3-bad8-b570db9026a0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:52.569100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251099e-ffdc-4de3-bad8-b570db9026a0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:52.569100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251099e-ffdc-4de3-bad8-b570db9026a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:52.600893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5251099e-ffdc-4de3-bad8-b570db9026a0 None None] MFA Rules not processed for user `b1531c5af9874d12a28dad2f2313dd94`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:52.602084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5251099e-ffdc-4de3-bad8-b570db9026a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:52.714610 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3612/7265] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:25:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:52.726581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:52.728508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:52.828651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb6d47-db46-4f6a-abd8-7aa0de4c6279 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:25:52.833998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:25:52.834972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:52.835475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:52.836382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:52.857976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:25:52.860091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:25:52.877315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3654/7266] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:25:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:25:59.665038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:25:59.665669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:25:59.665769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:25:59.703737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:25:59.705330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:59.838016 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3613/7267] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:25:59.874975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:25:59.876840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:25:59.951002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:00.034924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caf556-ecbc-4326-8030-7c8269e5b758 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:00.041065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:00.041846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:00.042726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:00.042726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:00.066224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:00.068505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:00.097786 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3655/7268] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:25:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:01.857799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:01.859901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:01.999450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:02.041093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a86ef2-e8a8-4a09-af2a-c37b70ac63c0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:02.050894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:02.051915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:02.052865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:02.055347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:02.062913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:02.106927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:02.108942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:02.133022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3614/7269] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:10.090487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=20 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:12.123258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:18.163428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:18.166450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:18.278005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9446fa-b720-450f-aa9b-6be09a56bf0a service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:18.283759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498114-f82d-4604-b1fa-c0f8109812e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:18.284391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498114-f82d-4604-b1fa-c0f8109812e2 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:18.284705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498114-f82d-4604-b1fa-c0f8109812e2 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:18.284994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498114-f82d-4604-b1fa-c0f8109812e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:18.287307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c4498114-f82d-4604-b1fa-c0f8109812e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:18.367864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498114-f82d-4604-b1fa-c0f8109812e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:18.369644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498114-f82d-4604-b1fa-c0f8109812e2 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:18.381847 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3656/7270] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:18.667569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:18.670166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:18.779956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cf653-b93c-4959-b44a-b90b223e91ff service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:18.785975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:18.787643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:18.788111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:18.788571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] PATH_INFO: `/v3/users/fbcfed06aa6b4177a4287064884125b2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:18.790032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcfed06aa6b4177a4287064884125b2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:18.792307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:18.866921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcfed06aa6b4177a4287064884125b2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:18.868282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcfed06aa6b4177a4287064884125b2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:18.879011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcfed06aa6b4177a4287064884125b2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:18.891355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcfed06aa6b4177a4287064884125b2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:18.908141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcfed06aa6b4177a4287064884125b2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:18.915985 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3615/7271] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:18 2022] DELETE /identity/v3/users/fbcfed06aa6b4177a4287064884125b2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:18.921086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:18.921369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:18.921644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:18.943635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:18.944898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:19.039590 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3657/7272] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:19.052502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:19.054319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:19.161951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ef7cd-b8dd-4291-a7bd-1339f77479cb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:19.168487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc879a45-4d2f-4460-9c03-4c9cdfb10441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:19.170164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc879a45-4d2f-4460-9c03-4c9cdfb10441 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:19.170833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc879a45-4d2f-4460-9c03-4c9cdfb10441 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:19.171261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc879a45-4d2f-4460-9c03-4c9cdfb10441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:19.184578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc879a45-4d2f-4460-9c03-4c9cdfb10441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:19.186569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc879a45-4d2f-4460-9c03-4c9cdfb10441 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:19.202481 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3616/7273] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:19.671012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:19.673222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:19.695221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e53a1-aa87-4310-b7ea-c1797be3c690 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:19.706894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:19.708792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:19.709178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:19.709529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] PATH_INFO: `/v3/projects/430555edbc87497db7d7f79f2b891c04` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:19.710793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430555edbc87497db7d7f79f2b891c04)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:19.713356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:19.754174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:19.768215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430555edbc87497db7d7f79f2b891c04', 'request_id': 'req-701ffa62-8ffc-4599-89bb-9ce40bc6a830'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:19.778112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430555edbc87497db7d7f79f2b891c04', 'request_id': 'req-701ffa62-8ffc-4599-89bb-9ce40bc6a830'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:19.783838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430555edbc87497db7d7f79f2b891c04', 'request_id': 'req-701ffa62-8ffc-4599-89bb-9ce40bc6a830'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:19.796568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] The token cache is being invalidate because project 430555edbc87497db7d7f79f2b891c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:19.796956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:19.797549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-701ffa62-8ffc-4599-89bb-9ce40bc6a830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:19.799199 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3658/7274] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:19 2022] DELETE /identity/v3/projects/430555edbc87497db7d7f79f2b891c04 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:26:19.804950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f5b60-711a-4ed9-a8f0-3d3cc2e11077 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:19.805450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f5b60-711a-4ed9-a8f0-3d3cc2e11077 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:19.805646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f5b60-711a-4ed9-a8f0-3d3cc2e11077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:19.830769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-481f5b60-711a-4ed9-a8f0-3d3cc2e11077 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:19.842945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-481f5b60-711a-4ed9-a8f0-3d3cc2e11077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:19.868224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:19.926744 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3617/7275] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:19.935532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9ec30-0439-479c-9041-259e429debdd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:19.935806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9ec30-0439-479c-9041-259e429debdd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:19.936071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9ec30-0439-479c-9041-259e429debdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:19.959241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-85c9ec30-0439-479c-9041-259e429debdd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:19.960763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-85c9ec30-0439-479c-9041-259e429debdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:19.977212 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3659/7276] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:19.978543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f5b60-711a-4ed9-a8f0-3d3cc2e11077 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:19.990345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f5b60-711a-4ed9-a8f0-3d3cc2e11077 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:19.995826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:19.996532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:19.996981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:19.997271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:19.998049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:26:19.998424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:20.000120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:20.017230 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3618/7277] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:26:20.022408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9ec30-0439-479c-9041-259e429debdd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:20.044685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9ec30-0439-479c-9041-259e429debdd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:20.044685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:20.044685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:20.044685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:20.045721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:20.045721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:20.045721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:20.076743 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3660/7278] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:20 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:20.080786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:20.092140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738639b-5449-4b15-b787-b02dc3cb78e2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:20.151600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:20.152217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:20.152496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:20.152803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:20.153435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:20.155363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:20.178309 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3619/7279] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:20 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:20.182687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:20.194778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a812b-d797-417d-bfb9-c233787a45ce admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:20.200804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:20.201182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:20.201465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:20.201741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:20.202392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:20.204295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:20.220635 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3661/7280] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:20.225099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:20.236579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e626718-cfc0-464d-87fe-b08b2a1c7849 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:20.241631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d03b10-2d6b-4d49-add3-c383cbda81e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:20.242229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d03b10-2d6b-4d49-add3-c383cbda81e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:20.242527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d03b10-2d6b-4d49-add3-c383cbda81e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:20.242797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d03b10-2d6b-4d49-add3-c383cbda81e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:20.243448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d03b10-2d6b-4d49-add3-c383cbda81e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:20.245175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d03b10-2d6b-4d49-add3-c383cbda81e4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:20.256858 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3620/7281] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:20.261334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:20.272547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e0b3-e594-4693-b580-79c0ba41aeed admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:20.277947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:20.279183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:20.279622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:20.279883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] PATH_INFO: `/v3/projects/08ded570affb4de69a0d6ffbf0ea80aa/users/ff7c05e217c4401eb103a80188392d64/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:20.300813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ded570affb4de69a0d6ffbf0ea80aa, user_id=ff7c05e217c4401eb103a80188392d64, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:20.303842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:20.321766 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3662/7282] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:20 2022] PUT /identity/v3/projects/08ded570affb4de69a0d6ffbf0ea80aa/users/ff7c05e217c4401eb103a80188392d64/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:26:20.333463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:20.335509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:20.336004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:20.378559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] MFA Rules not processed for user `ff7c05e217c4401eb103a80188392d64`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:20.383974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:20.474910 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3621/7283] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:20.486719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:20.488577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:20.556561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba937ab-f840-474d-9a24-bd3351b65401 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:20.561170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb371b-9558-4af4-bf3a-9b26989db8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:20.561805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb371b-9558-4af4-bf3a-9b26989db8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:20.562144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb371b-9558-4af4-bf3a-9b26989db8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:20.562450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb371b-9558-4af4-bf3a-9b26989db8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:20.576497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb371b-9558-4af4-bf3a-9b26989db8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:20.578706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb371b-9558-4af4-bf3a-9b26989db8b5 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:20.589247 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3663/7284] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:23.496604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:23.498497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:23.532592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:23.595292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35bf0f-66a9-4611-a9a6-d9996f9523f2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:23.603573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:23.604497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:23.606256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:23.606547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] PATH_INFO: `/v3/users/b1531c5af9874d12a28dad2f2313dd94` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:23.610568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1531c5af9874d12a28dad2f2313dd94)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:23.613869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:23.714818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1531c5af9874d12a28dad2f2313dd94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:23.715650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1531c5af9874d12a28dad2f2313dd94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:23.727150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1531c5af9874d12a28dad2f2313dd94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:23.834433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1531c5af9874d12a28dad2f2313dd94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:23.853362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1531c5af9874d12a28dad2f2313dd94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:23.860758 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3622/7285] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:23 2022] DELETE /identity/v3/users/b1531c5af9874d12a28dad2f2313dd94 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:23.866144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:23.866699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:23.866927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:23.897043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:23.898326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:23.963761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:23.979713 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3664/7286] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:23.987045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:23.988458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:24.068193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd73cb-24dc-4ce3-869b-7a88cf0df3c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.073233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca300bc7-dec0-46c8-8e1b-754462fe5d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.073979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca300bc7-dec0-46c8-8e1b-754462fe5d1b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.074266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca300bc7-dec0-46c8-8e1b-754462fe5d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.074519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca300bc7-dec0-46c8-8e1b-754462fe5d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.085306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca300bc7-dec0-46c8-8e1b-754462fe5d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.087099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca300bc7-dec0-46c8-8e1b-754462fe5d1b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.097518 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3623/7287] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:24.309976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:24.311947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:24.327955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1d14c-92f2-4fb6-8a59-233238230860 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.335333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.335958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.336747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.337031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] PATH_INFO: `/v3/projects/fb6c2eb66a3449c6b4b5065ff5ace428` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.338547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6c2eb66a3449c6b4b5065ff5ace428)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.340530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.397324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6c2eb66a3449c6b4b5065ff5ace428', 'request_id': 'req-8695708a-75f3-4428-b492-b8526f0e1176'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:24.406293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6c2eb66a3449c6b4b5065ff5ace428', 'request_id': 'req-8695708a-75f3-4428-b492-b8526f0e1176'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:24.412710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6c2eb66a3449c6b4b5065ff5ace428', 'request_id': 'req-8695708a-75f3-4428-b492-b8526f0e1176'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:24.424798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] The token cache is being invalidate because project fb6c2eb66a3449c6b4b5065ff5ace428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:24.425131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:24.425811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8695708a-75f3-4428-b492-b8526f0e1176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:24.427699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3665/7288] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:24 2022] DELETE /identity/v3/projects/fb6c2eb66a3449c6b4b5065ff5ace428 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:24.432832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491f02-58bc-4970-98a8-0d55eda19a94 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.433120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491f02-58bc-4970-98a8-0d55eda19a94 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.433432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491f02-58bc-4970-98a8-0d55eda19a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.454622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b1491f02-58bc-4970-98a8-0d55eda19a94 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:24.455908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b1491f02-58bc-4970-98a8-0d55eda19a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:24.495528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:24.525870 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3624/7289] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:24.534374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6b523-daac-4a4f-85ea-2297e3a94f12 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.534688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6b523-daac-4a4f-85ea-2297e3a94f12 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.534986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6b523-daac-4a4f-85ea-2297e3a94f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.558014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9ac6b523-daac-4a4f-85ea-2297e3a94f12 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:24.559402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9ac6b523-daac-4a4f-85ea-2297e3a94f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:24.572776 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3666/7290] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:24.573673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1491f02-58bc-4970-98a8-0d55eda19a94 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:24.584770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1491f02-58bc-4970-98a8-0d55eda19a94 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.590836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.591410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.591710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.591959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.592622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:26:24.592970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.594983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.612830 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3625/7291] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:26:24.614005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6b523-daac-4a4f-85ea-2297e3a94f12 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:24.623876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6b523-daac-4a4f-85ea-2297e3a94f12 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.629040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.630407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.630781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.631600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.631847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.634504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.665019 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3667/7292] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:24.669810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:24.680846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff358b4-3efd-4477-a43c-8e3db9c18e01 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.741013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.741722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.742000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.742267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.742907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.744875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.776710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3626/7293] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:24 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:26:24.779206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:24.790716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa100e-bff0-4619-ab0a-09e7ab042e5b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.795972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.796577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.796884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.797259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.797803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.799400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.810678 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3668/7294] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:26:24.815093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:24.825955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789e594-e5f3-4eb6-9703-b93fefd6297d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.831650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85469623-924e-4c8d-906c-f99d355140c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.832283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85469623-924e-4c8d-906c-f99d355140c4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.832558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85469623-924e-4c8d-906c-f99d355140c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.832910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85469623-924e-4c8d-906c-f99d355140c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.833632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85469623-924e-4c8d-906c-f99d355140c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.835415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85469623-924e-4c8d-906c-f99d355140c4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.847291 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3627/7295] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:26:24.851841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:24.862516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86be9e-c112-4122-a298-a9be216261c1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:24.868101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:24.868813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.869094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.869368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] PATH_INFO: `/v3/projects/c12393fad7bc444fa79defbebd707e54/users/0da7116ed4f54e2aad90ae144508bba8/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.885679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12393fad7bc444fa79defbebd707e54, user_id=0da7116ed4f54e2aad90ae144508bba8, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:24.888370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:24.905306 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3669/7296] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:24 2022] PUT /identity/v3/projects/c12393fad7bc444fa79defbebd707e54/users/0da7116ed4f54e2aad90ae144508bba8/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:24.915794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c841e-4987-46c9-88bd-66ee0985c1f0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:24.916087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c841e-4987-46c9-88bd-66ee0985c1f0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:24.916377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c841e-4987-46c9-88bd-66ee0985c1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:24.948626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5c7c841e-4987-46c9-88bd-66ee0985c1f0 None None] MFA Rules not processed for user `0da7116ed4f54e2aad90ae144508bba8`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:24.949227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5c7c841e-4987-46c9-88bd-66ee0985c1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:25.022699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3628/7297] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:25.037337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:25.040730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:25.147671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ab729-6477-4027-b8cc-9d27115d512e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:25.147671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dfec3-73d4-41a0-99cf-6bf290b55826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:25.147671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dfec3-73d4-41a0-99cf-6bf290b55826 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:25.147671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dfec3-73d4-41a0-99cf-6bf290b55826 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:25.148168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dfec3-73d4-41a0-99cf-6bf290b55826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:25.157791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dfec3-73d4-41a0-99cf-6bf290b55826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:25.160219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dfec3-73d4-41a0-99cf-6bf290b55826 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:25.172018 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3670/7298] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:25.189193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c841e-4987-46c9-88bd-66ee0985c1f0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:25.257889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:25.296861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c841e-4987-46c9-88bd-66ee0985c1f0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:25.303361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:25.303361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:25.303567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:25.303853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] PATH_INFO: `/v3/users/0da7116ed4f54e2aad90ae144508bba8` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:25.304993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da7116ed4f54e2aad90ae144508bba8)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:25.306966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:25.368550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da7116ed4f54e2aad90ae144508bba8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:25.369203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da7116ed4f54e2aad90ae144508bba8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:25.379951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da7116ed4f54e2aad90ae144508bba8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:25.393261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da7116ed4f54e2aad90ae144508bba8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:25.417949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da7116ed4f54e2aad90ae144508bba8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:25.426750 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3629/7299] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:25 2022] DELETE /identity/v3/users/0da7116ed4f54e2aad90ae144508bba8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:25.432466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:25.432837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:25.433133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:25.456142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:25.457439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:25.477698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:25.536341 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3671/7300] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:25.547820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:25.549336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:25.629041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c76e0-29c9-440a-8fb6-a48bea478be8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:25.634056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807d82c-de06-474c-82d0-f35d99d19cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:25.634848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807d82c-de06-474c-82d0-f35d99d19cae service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:25.635123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807d82c-de06-474c-82d0-f35d99d19cae service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:25.635438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807d82c-de06-474c-82d0-f35d99d19cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:25.646874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807d82c-de06-474c-82d0-f35d99d19cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:25.648031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807d82c-de06-474c-82d0-f35d99d19cae service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:25.664917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3630/7301] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:25.992273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:25.995236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:26.014365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cc33e-25ab-434a-95fd-e9ca5877b923 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.020343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.022030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.023140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.023502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] PATH_INFO: `/v3/projects/c12393fad7bc444fa79defbebd707e54` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.024992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12393fad7bc444fa79defbebd707e54)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.027705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.086821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12393fad7bc444fa79defbebd707e54', 'request_id': 'req-595b016e-39c2-4816-a042-8d6662e6df05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:26.096751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12393fad7bc444fa79defbebd707e54', 'request_id': 'req-595b016e-39c2-4816-a042-8d6662e6df05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:26.103734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12393fad7bc444fa79defbebd707e54', 'request_id': 'req-595b016e-39c2-4816-a042-8d6662e6df05'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:26.116977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] The token cache is being invalidate because project c12393fad7bc444fa79defbebd707e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:26.117303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:26.118214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-595b016e-39c2-4816-a042-8d6662e6df05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:26.119782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3672/7302] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:25 2022] DELETE /identity/v3/projects/c12393fad7bc444fa79defbebd707e54 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:26.125494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e8523-115c-45fc-8535-3e15cfc73c5b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.125785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e8523-115c-45fc-8535-3e15cfc73c5b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.126029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e8523-115c-45fc-8535-3e15cfc73c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.149939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e49e8523-115c-45fc-8535-3e15cfc73c5b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:26.151400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e49e8523-115c-45fc-8535-3e15cfc73c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:26.216853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:26.233863 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3631/7303] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:26.241902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805095a5-035b-4e52-bd47-41dae408bcc3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.242189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805095a5-035b-4e52-bd47-41dae408bcc3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.242460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805095a5-035b-4e52-bd47-41dae408bcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.279940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-805095a5-035b-4e52-bd47-41dae408bcc3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:26.281525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-805095a5-035b-4e52-bd47-41dae408bcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:26.294996 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3673/7304] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:26.299657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e8523-115c-45fc-8535-3e15cfc73c5b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:26.313550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e8523-115c-45fc-8535-3e15cfc73c5b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.319619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.322225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.322225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.335241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.335241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:26:26.335241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.335241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.347003 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3632/7305] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:26:26.351791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805095a5-035b-4e52-bd47-41dae408bcc3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:26.362889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805095a5-035b-4e52-bd47-41dae408bcc3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.368100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.368804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.369082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.369728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.370061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.372323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.410141 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3674/7306] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:26 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:26.411766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:26.423525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54963c95-7132-4963-8020-2d3442206bb0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.494262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.495065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.495350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.495726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.496430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.498379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.528028 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3633/7307] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:26 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:26.532555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:26.546471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cd3aa-3a86-42b9-9e0f-e539a41c0b13 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.552480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.553656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.553962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.554260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.554961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.556679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.569547 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3675/7308] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:26:26.573846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:26.589163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4a19d-2116-433e-8b07-8e0b74898b2e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.594435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b57d34-f8d5-4a75-9d1d-5a4e71794983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.595171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b57d34-f8d5-4a75-9d1d-5a4e71794983 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.595720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b57d34-f8d5-4a75-9d1d-5a4e71794983 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.596001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b57d34-f8d5-4a75-9d1d-5a4e71794983 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.596752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b57d34-f8d5-4a75-9d1d-5a4e71794983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.598527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b57d34-f8d5-4a75-9d1d-5a4e71794983 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.621715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3634/7309] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:26:26.624363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:26.643375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350b172-f0d6-45c9-83d1-984c8df616f3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.649270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.650104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.650363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.650670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] PATH_INFO: `/v3/projects/ade8f869954c48aaac30263fe6d88f5e/users/f825d9606b414d3c8df857693b5b0407/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.671173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade8f869954c48aaac30263fe6d88f5e, user_id=f825d9606b414d3c8df857693b5b0407, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.674098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.691023 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3676/7310] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:26 2022] PUT /identity/v3/projects/ade8f869954c48aaac30263fe6d88f5e/users/f825d9606b414d3c8df857693b5b0407/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:26:26.702800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.703135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.703355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.750607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] MFA Rules not processed for user `f825d9606b414d3c8df857693b5b0407`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:26.751842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:26.835254 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3635/7311] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:26.850208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:26.852478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:26.909242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:26.931230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e134532-b39b-4fd8-be36-a3495787cb3b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:26.936221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8293246-533b-40c8-9340-df607613b6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:26.936917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8293246-533b-40c8-9340-df607613b6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:26.937207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8293246-533b-40c8-9340-df607613b6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:26.937504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8293246-533b-40c8-9340-df607613b6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:26.947628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8293246-533b-40c8-9340-df607613b6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:26.949220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8293246-533b-40c8-9340-df607613b6a6 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:26.959913 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3677/7312] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:31.368179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:31.377931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:31.459549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:31.480439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2f01-1d7d-4b38-9477-f7d75db1b3a0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:31.488858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:31.489926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:31.490400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:31.490710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] PATH_INFO: `/v3/users/ff7c05e217c4401eb103a80188392d64` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:31.492148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7c05e217c4401eb103a80188392d64)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:31.508759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:31.636750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7c05e217c4401eb103a80188392d64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:31.637487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7c05e217c4401eb103a80188392d64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:31.650665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7c05e217c4401eb103a80188392d64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:31.674061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7c05e217c4401eb103a80188392d64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:31.701670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7c05e217c4401eb103a80188392d64'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:31.710184 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3636/7313] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:31 2022] DELETE /identity/v3/users/ff7c05e217c4401eb103a80188392d64 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:31.715677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:31.715677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:31.716360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:31.752954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:31.754475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:31.851055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3678/7314] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:31.897243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:31.899393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:32.057443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f01323-405c-40ea-82d4-d515d648355f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:32.075917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc77ae-a325-4bf2-8f47-929de65998c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:32.076891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc77ae-a325-4bf2-8f47-929de65998c2 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:32.077603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc77ae-a325-4bf2-8f47-929de65998c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:32.085637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc77ae-a325-4bf2-8f47-929de65998c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:32.117510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc77ae-a325-4bf2-8f47-929de65998c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:32.119294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc77ae-a325-4bf2-8f47-929de65998c2 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:32.170777 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3637/7315] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:33.590389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:33.592592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:33.636557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807375c-d597-4318-9d8c-1faf3df13e5d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:33.660710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:33.661806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:33.662405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:33.663033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] PATH_INFO: `/v3/projects/08ded570affb4de69a0d6ffbf0ea80aa` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:33.664426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ded570affb4de69a0d6ffbf0ea80aa)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:33.668924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:33.789203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ded570affb4de69a0d6ffbf0ea80aa', 'request_id': 'req-7a097a75-946d-42b3-8ffd-420efa70fc7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:33.816918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ded570affb4de69a0d6ffbf0ea80aa', 'request_id': 'req-7a097a75-946d-42b3-8ffd-420efa70fc7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:33.832110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ded570affb4de69a0d6ffbf0ea80aa', 'request_id': 'req-7a097a75-946d-42b3-8ffd-420efa70fc7f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:33.867310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] The token cache is being invalidate because project 08ded570affb4de69a0d6ffbf0ea80aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:33.871729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:33.872841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a097a75-946d-42b3-8ffd-420efa70fc7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:33.875198 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3679/7316] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:33 2022] DELETE /identity/v3/projects/08ded570affb4de69a0d6ffbf0ea80aa => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:33.889024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597bd6a-8b03-475d-a68c-c54564b76c00 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:33.890189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597bd6a-8b03-475d-a68c-c54564b76c00 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:33.890758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597bd6a-8b03-475d-a68c-c54564b76c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:33.950651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f597bd6a-8b03-475d-a68c-c54564b76c00 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:33.951963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f597bd6a-8b03-475d-a68c-c54564b76c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:34.102748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3638/7317] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:34.111705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bff17-afff-471e-8567-d1b2a99cb932 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.111993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bff17-afff-471e-8567-d1b2a99cb932 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.112417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bff17-afff-471e-8567-d1b2a99cb932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.153713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f88bff17-afff-471e-8567-d1b2a99cb932 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:34.181239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f88bff17-afff-471e-8567-d1b2a99cb932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:34.213427 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3680/7318] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:34.218726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597bd6a-8b03-475d-a68c-c54564b76c00 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:34.234723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597bd6a-8b03-475d-a68c-c54564b76c00 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:34.244156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:34.245572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.246035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.246462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.249777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:26:34.250276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:34.260850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:34.277576 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3639/7319] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:26:34.283045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bff17-afff-471e-8567-d1b2a99cb932 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:34.295201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bff17-afff-471e-8567-d1b2a99cb932 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:34.304500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:34.305528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.305822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.306215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.308285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:34.310981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:34.388142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3681/7320] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:34 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:34.392847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:34.398882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:34.408711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b546f3-6abb-4121-b04a-7acb891f81f8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:34.520386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:34.521234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.521504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.521783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.522443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:34.528265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:34.564992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3640/7321] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:34 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:34.569412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:34.585042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677253c3-4fdf-4510-9fbc-f1b808137060 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:34.596327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:34.597007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.597282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.597557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.598215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:34.601302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:34.642826 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3682/7322] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:34.647773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:34.666491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1589fd-2caa-442d-9c6e-ff547c872e52 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:34.678316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca51a7f-ff59-44d5-bf5e-bb854c03bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:34.679186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51a7f-ff59-44d5-bf5e-bb854c03bf9a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.679567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51a7f-ff59-44d5-bf5e-bb854c03bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.679946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51a7f-ff59-44d5-bf5e-bb854c03bf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.681286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca51a7f-ff59-44d5-bf5e-bb854c03bf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:34.685425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca51a7f-ff59-44d5-bf5e-bb854c03bf9a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:34.717611 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3641/7323] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:34.719129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:34.734801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dc830-7f6a-4338-9be9-29c102594cfc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:34.766096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:34.768121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.768404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.770414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] PATH_INFO: `/v3/projects/ae2e16cc010540ad8b7ccc66f9e6cbc2/users/b16810199568411495b68afbe47105e8/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.795175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2e16cc010540ad8b7ccc66f9e6cbc2, user_id=b16810199568411495b68afbe47105e8, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:34.798827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:34.828992 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3683/7324] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:34 2022] PUT /identity/v3/projects/ae2e16cc010540ad8b7ccc66f9e6cbc2/users/b16810199568411495b68afbe47105e8/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:34.929968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4e03-335c-4f0b-a6e6-f8f105242370 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:34.930527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4e03-335c-4f0b-a6e6-f8f105242370 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:34.930527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc4e03-335c-4f0b-a6e6-f8f105242370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:34.965192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-55dc4e03-335c-4f0b-a6e6-f8f105242370 None None] MFA Rules not processed for user `b16810199568411495b68afbe47105e8`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:34.965751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55dc4e03-335c-4f0b-a6e6-f8f105242370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:35.033189 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3642/7325] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:35.037828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:35.043832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:35.131717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b82b6b-f2c1-45a7-89c3-765681aa8bfa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:35.141285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:35.142001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.142393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.142735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.143457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:35.145512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:35.188540 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3684/7326] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:35 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:35.192787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc4e03-335c-4f0b-a6e6-f8f105242370 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:35.208813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc4e03-335c-4f0b-a6e6-f8f105242370 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:35.296104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:35.296789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.297066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.297341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.297997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:35.299999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:35.319309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:35.328507 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3643/7327] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:35 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:26:35.332233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:35.357282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c1dfa-77bf-47fb-96df-803000fdc8d7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:35.362395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e47db-2df8-4a68-a18f-e697aa4d4346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:35.363147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e47db-2df8-4a68-a18f-e697aa4d4346 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.363427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e47db-2df8-4a68-a18f-e697aa4d4346 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.363769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e47db-2df8-4a68-a18f-e697aa4d4346 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.364481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e47db-2df8-4a68-a18f-e697aa4d4346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:35.366242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e47db-2df8-4a68-a18f-e697aa4d4346 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:35.380548 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3685/7328] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:26:35.384809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:35.395183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae2dec-75a4-4982-9d8d-bc4d8297e7b8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:35.400782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0046d1-a29f-4b1d-8373-5a81fbcefeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:35.401568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0046d1-a29f-4b1d-8373-5a81fbcefeea admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.401966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0046d1-a29f-4b1d-8373-5a81fbcefeea admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.402311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0046d1-a29f-4b1d-8373-5a81fbcefeea admin admin] PATH_INFO: `/v3/projects/fb14921cdec64953ac219bae38be69f4/users/94c478dc56f84774b6804ffce3fe3011/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.420327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0046d1-a29f-4b1d-8373-5a81fbcefeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb14921cdec64953ac219bae38be69f4, user_id=94c478dc56f84774b6804ffce3fe3011, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:35.438769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0046d1-a29f-4b1d-8373-5a81fbcefeea admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:35.457009 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3644/7329] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:35 2022] PUT /identity/v3/projects/fb14921cdec64953ac219bae38be69f4/users/94c478dc56f84774b6804ffce3fe3011/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:35.470674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.470674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.470674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.501472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] MFA Rules not processed for user `94c478dc56f84774b6804ffce3fe3011`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:35.502748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:35.553437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.553805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.554166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.577434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:35.578980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:35.592919 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3686/7330] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:35 2022] POST /identity/v3/auth/tokens => generated 3271 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:35.629874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:35.632483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:35.675272 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3645/7331] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:26:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:26:35.704636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:35.706206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:35.746485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b2d0d-bc84-48bb-a446-f1828b8a7aa6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:35.753456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:35.754373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.755447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.755975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.772910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:35.776097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:35.782173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:35.811722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff128de2-eeba-43ed-9741-201739e37333 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:35.818922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3687/7332] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 680 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:26:35.825066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:35.826049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:35.826330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:35.826580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:35.837545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:35.838748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:35.861240 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3646/7333] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:36.059527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:36.061440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:36.166387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75e2dd-dc7c-4ae0-b912-9717772471d6 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:36.193611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ac10d-796d-4a54-b13b-abeb3eff64b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:36.196152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ac10d-796d-4a54-b13b-abeb3eff64b4 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:36.196445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ac10d-796d-4a54-b13b-abeb3eff64b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:36.196705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ac10d-796d-4a54-b13b-abeb3eff64b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:36.207860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-968ac10d-796d-4a54-b13b-abeb3eff64b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:36.318012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ac10d-796d-4a54-b13b-abeb3eff64b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:36.319903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ac10d-796d-4a54-b13b-abeb3eff64b4 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:36.338799 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3688/7334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:36.882584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:36.969973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a961bbe-0d86-4ac1-ad33-036071b684d8 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:36.979855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:36.983526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:36.984211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:36.985104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] PATH_INFO: `/v3/users/b16810199568411495b68afbe47105e8` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:36.986667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16810199568411495b68afbe47105e8)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:36.990140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:37.006261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:37.076929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16810199568411495b68afbe47105e8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:37.077616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16810199568411495b68afbe47105e8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:37.086103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16810199568411495b68afbe47105e8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:37.102031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16810199568411495b68afbe47105e8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:37.117836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16810199568411495b68afbe47105e8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:37.128762 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3647/7335] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:36 2022] DELETE /identity/v3/users/b16810199568411495b68afbe47105e8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:37.131628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:37.132596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:37.133693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:37.157970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:37.160469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:37.242294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3689/7336] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:37.261255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:37.264508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:37.359460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ebef-a586-404b-ab4f-aac0085d1aff admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:37.363621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:37.364303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:37.364937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:37.364937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:37.379166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:37.381254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:37.419263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3648/7337] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:38.183089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:38.185337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:38.207740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494643-0b00-422b-a0e2-e3a1f18adf8f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:38.214363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:38.216133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:38.216587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:38.217250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] PATH_INFO: `/v3/projects/ae2e16cc010540ad8b7ccc66f9e6cbc2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:38.218927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2e16cc010540ad8b7ccc66f9e6cbc2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:38.230518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:38.314649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:38.335239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2e16cc010540ad8b7ccc66f9e6cbc2', 'request_id': 'req-143c6697-432c-4131-ad51-2562c2ae9251'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.354611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2e16cc010540ad8b7ccc66f9e6cbc2', 'request_id': 'req-143c6697-432c-4131-ad51-2562c2ae9251'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.371722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2e16cc010540ad8b7ccc66f9e6cbc2', 'request_id': 'req-143c6697-432c-4131-ad51-2562c2ae9251'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.393084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] The token cache is being invalidate because project ae2e16cc010540ad8b7ccc66f9e6cbc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:38.393749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.394483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.397087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3690/7338] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:38 2022] DELETE /identity/v3/projects/ae2e16cc010540ad8b7ccc66f9e6cbc2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:38.401228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:38.416846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c4c88-e629-4e5f-91fd-d34eff47a5b8 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:38.504609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:38.505650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:38.506003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:38.506347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] PATH_INFO: `/v3/users/94c478dc56f84774b6804ffce3fe3011` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:38.507653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c478dc56f84774b6804ffce3fe3011)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:38.509849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:38.581089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:38.594189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c478dc56f84774b6804ffce3fe3011'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.594977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c478dc56f84774b6804ffce3fe3011'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.606431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c478dc56f84774b6804ffce3fe3011'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.623875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c478dc56f84774b6804ffce3fe3011'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.642862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c5d66329-86d3-4d59-8358-59f2d1b9f394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c478dc56f84774b6804ffce3fe3011'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:38.656808 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3649/7339] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:38 2022] DELETE /identity/v3/users/94c478dc56f84774b6804ffce3fe3011 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:39.422584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:39.423999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:39.444345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c6697-432c-4131-ad51-2562c2ae9251 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:39.543553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:39.544545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:39.545026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:39.545475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] PATH_INFO: `/v3/projects/fb14921cdec64953ac219bae38be69f4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:39.546888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb14921cdec64953ac219bae38be69f4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:39.548869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:39.673071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb14921cdec64953ac219bae38be69f4', 'request_id': 'req-d0a7cab7-a600-423f-8141-6b66727a6d82'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:39.686309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb14921cdec64953ac219bae38be69f4', 'request_id': 'req-d0a7cab7-a600-423f-8141-6b66727a6d82'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:39.697329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb14921cdec64953ac219bae38be69f4', 'request_id': 'req-d0a7cab7-a600-423f-8141-6b66727a6d82'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:39.725384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] The token cache is being invalidate because project fb14921cdec64953ac219bae38be69f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:39.726501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:39.727986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d0a7cab7-a600-423f-8141-6b66727a6d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:39.730854 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3691/7340] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:39 2022] DELETE /identity/v3/projects/fb14921cdec64953ac219bae38be69f4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:39.737470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6faca6-2892-4000-84b9-56d414dbd403 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:39.737943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6faca6-2892-4000-84b9-56d414dbd403 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:39.738027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6faca6-2892-4000-84b9-56d414dbd403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:39.763275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4c6faca6-2892-4000-84b9-56d414dbd403 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:39.764460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4c6faca6-2892-4000-84b9-56d414dbd403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:39.875676 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3650/7341] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:39.885591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833300f-afe2-4178-b65a-03b45290a78d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:39.885880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833300f-afe2-4178-b65a-03b45290a78d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:39.886158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833300f-afe2-4178-b65a-03b45290a78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:39.929410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3833300f-afe2-4178-b65a-03b45290a78d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:39.930571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3833300f-afe2-4178-b65a-03b45290a78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:39.944949 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3692/7342] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:39.947783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6faca6-2892-4000-84b9-56d414dbd403 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:39.964815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6faca6-2892-4000-84b9-56d414dbd403 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:39.993665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:39.995603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:39.996378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:39.997132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:39.998073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:26:39.998738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:40.002761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:40.026588 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3651/7343] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:26:40.027063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833300f-afe2-4178-b65a-03b45290a78d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:40.038492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833300f-afe2-4178-b65a-03b45290a78d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:40.045048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:40.045647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:40.045927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:40.046202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:40.047784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:40.050109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:40.087971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:40.090140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:40.117033 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3693/7344] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:40 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:26:40.118086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:40.163660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:40.236558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4ed4d-3d3c-4b00-825c-0bb822bea891 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:40.258400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6167b-a9e8-488a-999d-b6c3cfe4219b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:40.271899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:40.272912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:40.273269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:40.273883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:40.274627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:40.277625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:40.322799 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3694/7345] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:40 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:40.324098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:40.351583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50465f-f9fb-4f95-8a39-a06438c176ca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:40.360245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:40.362024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:40.362569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:40.363114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:40.364734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:40.367818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:40.373400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:40.374094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:40.382939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:40.383779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:40.391557 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3695/7346] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:40.392924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:40.394914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:40.416681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba74f04-98da-4784-b3f6-ba4c1f6d2e1b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:40.426397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:40.427321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:40.427528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:40.428718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:40.429884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:40.432959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:40.440273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:40.463554 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3696/7347] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:26:40.468385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:40.490747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87bb9-aa2d-47fd-a0d1-1d1b087c8944 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:40.500814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422bda9-81f6-4aed-add3-fef940e61193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:40.501912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422bda9-81f6-4aed-add3-fef940e61193 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:40.502681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422bda9-81f6-4aed-add3-fef940e61193 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:40.503157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422bda9-81f6-4aed-add3-fef940e61193 admin admin] PATH_INFO: `/v3/projects/b4c28e76573e40269417e52300814190/users/55b65e5b614143af8fcc5155671e8602/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:40.555356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422bda9-81f6-4aed-add3-fef940e61193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c28e76573e40269417e52300814190, user_id=55b65e5b614143af8fcc5155671e8602, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:40.559176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422bda9-81f6-4aed-add3-fef940e61193 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:40.591377 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3697/7348] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:40 2022] PUT /identity/v3/projects/b4c28e76573e40269417e52300814190/users/55b65e5b614143af8fcc5155671e8602/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:40.609099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ceca8-3319-44cc-aa39-9cb6e84b68fb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:40.609310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ceca8-3319-44cc-aa39-9cb6e84b68fb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:40.609890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ceca8-3319-44cc-aa39-9cb6e84b68fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:40.632576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:40.634876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:40.665355 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3652/7349] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:26:40.677178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-397ceca8-3319-44cc-aa39-9cb6e84b68fb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:40.681906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-397ceca8-3319-44cc-aa39-9cb6e84b68fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:40.831101 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3698/7350] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:40.851029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:40.852921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:40.973681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa9f2-0559-458a-9b9c-9840ac699079 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:41.067217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:41.068408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:41.068408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:41.068777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:41.090531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:41.091958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:41.103118 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3653/7351] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:26:41.720575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:41.721154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:41.721154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:41.768079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] MFA Rules not processed for user `178956f6c8154a06a0710b295aff3086`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:41.776293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:41.960884 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3699/7352] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:43.181078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:43.201977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177077b-3613-4d19-8829-5896cae45f34 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:43.207516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae9820-589b-42e5-bf93-14822cac8e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:43.208246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9820-589b-42e5-bf93-14822cac8e2d service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:43.208858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9820-589b-42e5-bf93-14822cac8e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:43.209270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9820-589b-42e5-bf93-14822cac8e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:43.211591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8ae9820-589b-42e5-bf93-14822cac8e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:43.226576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae9820-589b-42e5-bf93-14822cac8e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:43.228291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae9820-589b-42e5-bf93-14822cac8e2d service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:43.241326 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3654/7353] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:47.561337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:47.562897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:47.577426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d8207-03e1-47e6-8b8f-e7bcbfa3942c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:47.582177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:47.582808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:47.583083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:47.583437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] PATH_INFO: `/v3/users/f825d9606b414d3c8df857693b5b0407` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:47.584630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f825d9606b414d3c8df857693b5b0407)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:47.587033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:47.594634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:47.656484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f825d9606b414d3c8df857693b5b0407'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:47.659831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f825d9606b414d3c8df857693b5b0407'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:47.678365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f825d9606b414d3c8df857693b5b0407'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:47.703188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f825d9606b414d3c8df857693b5b0407'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:47.704868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f825d9606b414d3c8df857693b5b0407'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:47.748314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3700/7354] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:47 2022] DELETE /identity/v3/users/f825d9606b414d3c8df857693b5b0407 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:47.753475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:47.753935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:47.754035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:47.788766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:47.790707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:47.852266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:47.911048 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3655/7355] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:47.927018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:47.929683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:48.049588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e047446-4faa-4cd9-ab66-ecf125635915 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:48.054708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67aff3-6ee7-4f66-b17c-db568655241f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:48.056705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67aff3-6ee7-4f66-b17c-db568655241f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:48.057137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67aff3-6ee7-4f66-b17c-db568655241f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:48.057556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67aff3-6ee7-4f66-b17c-db568655241f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:48.084343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67aff3-6ee7-4f66-b17c-db568655241f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:48.089677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67aff3-6ee7-4f66-b17c-db568655241f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:48.111571 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3701/7356] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:48.558640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:48.562717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:48.582478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422c68b-0b62-4d8d-9692-1235fc297ed2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:48.599170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:48.599985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:48.600271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:48.600548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] PATH_INFO: `/v3/projects/ade8f869954c48aaac30263fe6d88f5e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:48.605669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ade8f869954c48aaac30263fe6d88f5e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:48.610844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:48.725891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ade8f869954c48aaac30263fe6d88f5e', 'request_id': 'req-42721701-7589-4632-9700-eb6329794b58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:48.738837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ade8f869954c48aaac30263fe6d88f5e', 'request_id': 'req-42721701-7589-4632-9700-eb6329794b58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:48.763343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ade8f869954c48aaac30263fe6d88f5e', 'request_id': 'req-42721701-7589-4632-9700-eb6329794b58'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:48.789486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] The token cache is being invalidate because project ade8f869954c48aaac30263fe6d88f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:48.789781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:48.791050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-42721701-7589-4632-9700-eb6329794b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:48.793316 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3656/7357] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:48 2022] DELETE /identity/v3/projects/ade8f869954c48aaac30263fe6d88f5e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:48.800014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c3cf5-406e-43ef-9eba-0c5c1a0db886 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:48.800591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c3cf5-406e-43ef-9eba-0c5c1a0db886 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:48.800591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c3cf5-406e-43ef-9eba-0c5c1a0db886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:48.836183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1e0c3cf5-406e-43ef-9eba-0c5c1a0db886 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:48.838226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e0c3cf5-406e-43ef-9eba-0c5c1a0db886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:48.994576 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3702/7358] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:49.003220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47e32b-8cac-4f87-b1fe-214dfca63b47 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.003793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47e32b-8cac-4f87-b1fe-214dfca63b47 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.004208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47e32b-8cac-4f87-b1fe-214dfca63b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.071661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3c47e32b-8cac-4f87-b1fe-214dfca63b47 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:49.073676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3c47e32b-8cac-4f87-b1fe-214dfca63b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:49.102472 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3657/7359] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:49.106959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c3cf5-406e-43ef-9eba-0c5c1a0db886 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:49.125738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c3cf5-406e-43ef-9eba-0c5c1a0db886 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:49.140204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:49.145042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.146260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.147627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.149322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:26:49.149827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:49.151656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:49.184965 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3703/7360] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:26:49.190651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47e32b-8cac-4f87-b1fe-214dfca63b47 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:49.211581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47e32b-8cac-4f87-b1fe-214dfca63b47 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:49.225683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:49.227539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.229096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.229597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.232077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:49.240149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:49.330817 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3658/7361] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:49 2022] POST /identity/v3/projects => generated 401 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:26:49.334655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:49.352238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5d765-7df8-4b84-b02f-b26d107e6bb0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:49.402116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:49.500012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:49.500955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.501432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.501849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.502934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:49.505616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:49.552867 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3704/7362] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:49 2022] POST /identity/v3/users => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:49.557069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:49.570728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87cdd3-2d63-432e-b39a-09111b400f64 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:49.576572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:49.577716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.578206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.578684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.579571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:49.581786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:49.604340 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3659/7363] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:49.609315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:49.628433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66e7cf-d635-4242-8068-8b6342e09f6b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:49.636541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5d03e-2910-4fc3-b0ed-d7a5c5cf1716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:49.637625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5d03e-2910-4fc3-b0ed-d7a5c5cf1716 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.638057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5d03e-2910-4fc3-b0ed-d7a5c5cf1716 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.638639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5d03e-2910-4fc3-b0ed-d7a5c5cf1716 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.640415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5d03e-2910-4fc3-b0ed-d7a5c5cf1716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:49.642757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5d03e-2910-4fc3-b0ed-d7a5c5cf1716 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:49.669567 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3705/7364] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:49.673431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:49.718822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae501a7-2f13-4ca7-846d-ddbd86bca828 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:49.742753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:49.743761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.744309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.744830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] PATH_INFO: `/v3/projects/3bf83c89117946088316cbd8f1de8fe2/users/e720b6bdc84b488f90165d8971f4198e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.792465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf83c89117946088316cbd8f1de8fe2, user_id=e720b6bdc84b488f90165d8971f4198e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:49.799187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:49.860778 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3660/7365] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:49 2022] PUT /identity/v3/projects/3bf83c89117946088316cbd8f1de8fe2/users/e720b6bdc84b488f90165d8971f4198e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:49.872435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3816ec-e211-44fa-90f3-b947a8b35482 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:49.873021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3816ec-e211-44fa-90f3-b947a8b35482 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:49.873021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3816ec-e211-44fa-90f3-b947a8b35482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:49.975557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2f3816ec-e211-44fa-90f3-b947a8b35482 None None] MFA Rules not processed for user `e720b6bdc84b488f90165d8971f4198e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:49.977370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f3816ec-e211-44fa-90f3-b947a8b35482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:50.157114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:50.157498 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3706/7366] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:50.312439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d2a56-5d34-437c-84e9-6dba8b1a452d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:50.324737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:50.325629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:50.326013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:50.326404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:50.327383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:50.335399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:50.343982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:50.419814 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3661/7367] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:50 2022] POST /identity/v3/projects => generated 399 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:26:50.423786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3816ec-e211-44fa-90f3-b947a8b35482 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:50.468425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3816ec-e211-44fa-90f3-b947a8b35482 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:50.651938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:50.654041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:50.662959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:50.665191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:50.665930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:50.680034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:50.734289 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3707/7368] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:50 2022] POST /identity/v3/users => generated 306 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:50.735221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:50.751478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff0140-1a18-409b-bc7e-956b94e18206 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:50.765416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:50.767883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:50.770197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:50.770696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:50.771804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:50.773668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:50.804487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:50.808621 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3662/7369] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:50.815976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55ecda-3d48-406c-be15-563cf8ee073e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:50.822041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:50.822309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:50.822673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:50.823030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] PATH_INFO: `/v3/projects/a874a695f5e540a19116816f6d66a3dc/users/40c3f8e5fc3040068e7a034851326217/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:50.844933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:50.852118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a874a695f5e540a19116816f6d66a3dc, user_id=40c3f8e5fc3040068e7a034851326217, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:50.859677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:50.882208 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3708/7370] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:50 2022] PUT /identity/v3/projects/a874a695f5e540a19116816f6d66a3dc/users/40c3f8e5fc3040068e7a034851326217/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:26:50.886330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:50.900339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1834c5-9692-42d4-9199-638da3d0c729 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:50.995455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf88d59-24f2-498d-974f-aa3ce0843e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:50.996171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88d59-24f2-498d-974f-aa3ce0843e15 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:50.996354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88d59-24f2-498d-974f-aa3ce0843e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:50.996627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88d59-24f2-498d-974f-aa3ce0843e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:50.997315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf88d59-24f2-498d-974f-aa3ce0843e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:50.999106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf88d59-24f2-498d-974f-aa3ce0843e15 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:51.013032 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3663/7371] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:51.017878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:51.030969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797be2-828b-4ea7-ab2b-2f12b441c8aa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:51.038845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:51.039867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:51.040870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:51.041328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] PATH_INFO: `/v3/projects/a874a695f5e540a19116816f6d66a3dc/users/40c3f8e5fc3040068e7a034851326217/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:51.043789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a874a695f5e540a19116816f6d66a3dc, user_id=40c3f8e5fc3040068e7a034851326217, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:51.046695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:51.071215 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3709/7372] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:51 2022] PUT /identity/v3/projects/a874a695f5e540a19116816f6d66a3dc/users/40c3f8e5fc3040068e7a034851326217/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:51.082398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c792d-823d-4f05-a533-5d87533d5c93 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:51.082936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c792d-823d-4f05-a533-5d87533d5c93 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:51.083392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c792d-823d-4f05-a533-5d87533d5c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:51.148632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b54c792d-823d-4f05-a533-5d87533d5c93 None None] MFA Rules not processed for user `40c3f8e5fc3040068e7a034851326217`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:51.150143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b54c792d-823d-4f05-a533-5d87533d5c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:51.298159 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3664/7373] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:51.311179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:51.313571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:51.510807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212e5ab-0c65-45bb-b86c-f9589ffb3608 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:51.526034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceafb4e-b30d-475c-bf73-cbe31c2d1527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:51.527038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafb4e-b30d-475c-bf73-cbe31c2d1527 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:51.527443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafb4e-b30d-475c-bf73-cbe31c2d1527 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:51.527847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafb4e-b30d-475c-bf73-cbe31c2d1527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:51.559814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceafb4e-b30d-475c-bf73-cbe31c2d1527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:51.562318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceafb4e-b30d-475c-bf73-cbe31c2d1527 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:51.578688 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3710/7374] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:51.639249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd689-bc81-420f-94a8-ec2943f28a75 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:51.639540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd689-bc81-420f-94a8-ec2943f28a75 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:51.639814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd689-bc81-420f-94a8-ec2943f28a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:51.670200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:51.670480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:51.670793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:51.685682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-756cd689-bc81-420f-94a8-ec2943f28a75 None None] MFA Rules not processed for user `584c395757a54d9b9a64dd399d8557b1`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:51.687765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-756cd689-bc81-420f-94a8-ec2943f28a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:51.758607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] MFA Rules not processed for user `55b65e5b614143af8fcc5155671e8602`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:51.760191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:51.806768 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3665/7375] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:51.807751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cd689-bc81-420f-94a8-ec2943f28a75 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:51.821361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:51.832020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cd689-bc81-420f-94a8-ec2943f28a75 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:51.838619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:51.839517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:51.840022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:51.840465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:51.877087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3711/7376] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:26:51.882068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:51.883952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:51.983064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:51.985003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.002937 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3666/7377] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:52.058509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876da4e-4578-4d7e-9b97-3d67fdf5dfcc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:52.064093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:52.065115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:52.065866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:52.066758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:52.067622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:52.074695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.106983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:52.150340 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3712/7378] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:52.155060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:52.173377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b784e-fe1c-4ec9-8f91-c6f78b5c096f service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:52.293860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:52.294589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:52.294959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:52.295408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:52.296227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:52.298799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.338390 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3667/7379] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:52 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:52.340423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:52.356672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f3ac-b872-4345-bcc3-278243338aba admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:52.366808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:52.367958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:52.368540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:52.369172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:52.370604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:52.375485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.420824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:52.421381 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3713/7380] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:52.428839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade921f-34c5-4a51-bab9-844b70ab2600 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:52.443358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:52.444422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:52.444971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:52.446400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] PATH_INFO: `/v3/projects/0a60557528994b829cd5131340a497b7/users/a0e47aca97434f09846b580091b06829/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:52.493003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a60557528994b829cd5131340a497b7, user_id=a0e47aca97434f09846b580091b06829, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:52.496639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.535369 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3668/7381] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:52 2022] PUT /identity/v3/projects/0a60557528994b829cd5131340a497b7/users/a0e47aca97434f09846b580091b06829/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:26:52.540342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:52.555645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ae59b-2016-4688-a40c-2e4edd63f82f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:52.694900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:52.695541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:52.696430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:52.696841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:52.697547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:52.699457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.715313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3714/7382] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:52.719425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:52.730211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:52.732576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:52.741379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70c5f2-81d5-4afb-b8fb-410e4a711ffe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:52.753657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe49cc-a671-494e-bde1-24d24f331e8d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:52.755420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:52.756428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:52.756950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:52.757377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] PATH_INFO: `/v3/projects/0a60557528994b829cd5131340a497b7/users/a0e47aca97434f09846b580091b06829/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:52.760286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a60557528994b829cd5131340a497b7, user_id=a0e47aca97434f09846b580091b06829, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:52.763399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.768903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:52.770100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:52.770670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:52.771126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] PATH_INFO: `/v3/users/74878b715fc94b23bb139cb30d8d20c0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:52.782308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=74878b715fc94b23bb139cb30d8d20c0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:52.784898 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3669/7383] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:52 2022] PUT /identity/v3/projects/0a60557528994b829cd5131340a497b7/users/a0e47aca97434f09846b580091b06829/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:52.803723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:52.890487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74878b715fc94b23bb139cb30d8d20c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:52.900455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74878b715fc94b23bb139cb30d8d20c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:52.916493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74878b715fc94b23bb139cb30d8d20c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:52.930714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74878b715fc94b23bb139cb30d8d20c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:52.931819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-665064f8-6a02-430d-9b25-00a4c03ce84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74878b715fc94b23bb139cb30d8d20c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:52.997414 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3715/7384] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:52 2022] DELETE /identity/v3/users/74878b715fc94b23bb139cb30d8d20c0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:53.387065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:53.389924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:53.526026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:53.558749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c173a05-8a49-49a9-8e73-1b1b8b6d8f88 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:53.695108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:53.695997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:53.699049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:53.699191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] PATH_INFO: `/v3/projects/b590d2de576e41aab42f79e4daef9c63` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:53.717210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b590d2de576e41aab42f79e4daef9c63)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:53.721318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:53.886616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b590d2de576e41aab42f79e4daef9c63', 'request_id': 'req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:53.914386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b590d2de576e41aab42f79e4daef9c63', 'request_id': 'req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:53.941802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b590d2de576e41aab42f79e4daef9c63', 'request_id': 'req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:53.979774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] The token cache is being invalidate because project b590d2de576e41aab42f79e4daef9c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:26:53.980086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:53.981025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c960cf5b-8e17-479e-b2e3-4f9517df6ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:53.988616 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3670/7385] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:53 2022] DELETE /identity/v3/projects/b590d2de576e41aab42f79e4daef9c63 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:53.990670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08b3b-8fab-462a-986c-e3b07645ddb1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:53.991147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08b3b-8fab-462a-986c-e3b07645ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:53.991269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08b3b-8fab-462a-986c-e3b07645ddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.029234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c8b08b3b-8fab-462a-986c-e3b07645ddb1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:54.030496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c8b08b3b-8fab-462a-986c-e3b07645ddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:54.135951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:54.177410 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3716/7386] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:26:54.185509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b94ffa-3c74-4cdd-8c3e-7a8b1873e1e6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.186124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b94ffa-3c74-4cdd-8c3e-7a8b1873e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.186513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b94ffa-3c74-4cdd-8c3e-7a8b1873e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.219464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-95b94ffa-3c74-4cdd-8c3e-7a8b1873e1e6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:54.221608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-95b94ffa-3c74-4cdd-8c3e-7a8b1873e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:54.240732 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3671/7387] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:54.244725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08b3b-8fab-462a-986c-e3b07645ddb1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:54.256013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08b3b-8fab-462a-986c-e3b07645ddb1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:54.262402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:54.263069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.263347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.263869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.265279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:26:54.265622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:54.267998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:54.310336 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3717/7388] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:26:54.315056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b94ffa-3c74-4cdd-8c3e-7a8b1873e1e6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:54.326112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b94ffa-3c74-4cdd-8c3e-7a8b1873e1e6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:54.331412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:54.332057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.332335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.332640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.333623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:54.338869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:54.377131 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3672/7389] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:26:54 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:54.382973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:54.394040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1cfb7-21de-421b-8d47-03c28b1ea90f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:54.473132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:54.473926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.474279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.474614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.475318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:54.477406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:54.506412 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3718/7390] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:26:54 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:26:54.510706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:54.533997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39f787-1047-4977-8d57-6008dce65436 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:54.545734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:54.546727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.547181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.547598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.548885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:54.550649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:54.571001 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3673/7391] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:54.576198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:54.591921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874fa6e-c14c-4dfd-ad7d-e04e3cad4a7b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:54.604421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9dd91-8e5a-43f1-961f-2cca033514b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:54.605827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9dd91-8e5a-43f1-961f-2cca033514b5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.606284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9dd91-8e5a-43f1-961f-2cca033514b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.606705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9dd91-8e5a-43f1-961f-2cca033514b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.607643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9dd91-8e5a-43f1-961f-2cca033514b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:54.611539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9dd91-8e5a-43f1-961f-2cca033514b5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:54.631667 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3719/7392] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:26:54.636031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:54.651729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253b3e3-890a-417e-8cbe-09fc2b15e15d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:54.660870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:54.664002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.665288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.667079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] PATH_INFO: `/v3/projects/90c0fbb4a6d84767b920564584715f9d/users/64090c8fd3394980ad5409adcde4986f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.691889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c0fbb4a6d84767b920564584715f9d, user_id=64090c8fd3394980ad5409adcde4986f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:54.696802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:54.722145 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3674/7393] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:26:54 2022] PUT /identity/v3/projects/90c0fbb4a6d84767b920564584715f9d/users/64090c8fd3394980ad5409adcde4986f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:26:54.729249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:54.730053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:54.730565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:54.778391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:54.781443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:54.878605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3720/7394] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:54.889930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:54.894916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:54.957453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:54.998306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5296ab-2289-476d-97b7-cf159e34bf39 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:55.003167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b8cc4-06aa-4aac-b244-5783805a8cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:55.004291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b8cc4-06aa-4aac-b244-5783805a8cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:55.004794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b8cc4-06aa-4aac-b244-5783805a8cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:55.005437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b8cc4-06aa-4aac-b244-5783805a8cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:55.017602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b8cc4-06aa-4aac-b244-5783805a8cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:55.019694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b8cc4-06aa-4aac-b244-5783805a8cf2 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:55.032632 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3675/7395] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:57.133338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:57.136004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:57.181167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862315d-2829-40cf-821f-43f8ec84b98a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:57.202350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:57.203660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:57.204904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:57.205248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] PATH_INFO: `/v3/users/e720b6bdc84b488f90165d8971f4198e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:57.208197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e720b6bdc84b488f90165d8971f4198e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:57.213486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:57.365322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e720b6bdc84b488f90165d8971f4198e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:57.368009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e720b6bdc84b488f90165d8971f4198e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:57.379154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:57.392846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e720b6bdc84b488f90165d8971f4198e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:57.409784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e720b6bdc84b488f90165d8971f4198e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:57.412142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e720b6bdc84b488f90165d8971f4198e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:26:57.476008 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3721/7396] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:26:57 2022] DELETE /identity/v3/users/e720b6bdc84b488f90165d8971f4198e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:26:57.477792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:57.478264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:57.478358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:57.515716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:26:57.517069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:57.719332 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3676/7397] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:26:57.779842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:57.783130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:57.933412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5199c-1924-4253-8784-f3202dcb0401 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:57.940206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:57.941271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:57.941624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:57.941966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:57.954853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:57.974241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:57.992551 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3722/7398] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:26:59.727116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:26:59.728682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:26:59.762289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901716-74c7-4311-8d4a-e06a234f9040 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:26:59.787504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:26:59.789555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:26:59.789938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:26:59.820477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] PATH_INFO: `/v3/projects/3bf83c89117946088316cbd8f1de8fe2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:26:59.823461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf83c89117946088316cbd8f1de8fe2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:26:59.840612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:26:59.989010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf83c89117946088316cbd8f1de8fe2', 'request_id': 'req-02d11583-6182-4940-a8a8-bbec4061fe75'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.003205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf83c89117946088316cbd8f1de8fe2', 'request_id': 'req-02d11583-6182-4940-a8a8-bbec4061fe75'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.046500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf83c89117946088316cbd8f1de8fe2', 'request_id': 'req-02d11583-6182-4940-a8a8-bbec4061fe75'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.080442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] The token cache is being invalidate because project 3bf83c89117946088316cbd8f1de8fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:00.081424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.082082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.085695 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3677/7399] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:26:59 2022] DELETE /identity/v3/projects/3bf83c89117946088316cbd8f1de8fe2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:00.090025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:00.104685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cadc86-eb31-468e-ba1c-d3cdd3047367 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:00.264430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:00.264707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:00.264840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:00.265071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] PATH_INFO: `/v3/users/40c3f8e5fc3040068e7a034851326217` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:00.266236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c3f8e5fc3040068e7a034851326217)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:00.268182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:00.377768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c3f8e5fc3040068e7a034851326217'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.378507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c3f8e5fc3040068e7a034851326217'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.394166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c3f8e5fc3040068e7a034851326217'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.407325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c3f8e5fc3040068e7a034851326217'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.409639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1dce3ddc-251f-44a7-aad5-fe99dfac6901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c3f8e5fc3040068e7a034851326217'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:00.430969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:00.453633 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3723/7400] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:00 2022] DELETE /identity/v3/users/40c3f8e5fc3040068e7a034851326217 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:02.494070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:02.495547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:02.514516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d11583-6182-4940-a8a8-bbec4061fe75 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:02.664813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:02.671751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:02.673458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:02.674166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:02.674308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2c40-cc91-4cb8-afa3-f8c7d9b64aae None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:02.674777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2c40-cc91-4cb8-afa3-f8c7d9b64aae None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:02.674879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2c40-cc91-4cb8-afa3-f8c7d9b64aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:02.675416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] PATH_INFO: `/v3/projects/a874a695f5e540a19116816f6d66a3dc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:02.677492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a874a695f5e540a19116816f6d66a3dc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:02.680194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:02.714179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6e9a2c40-cc91-4cb8-afa3-f8c7d9b64aae None None] MFA Rules not processed for user `97c769dbba794531b0a9db8e764ff097`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:02.715452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e9a2c40-cc91-4cb8-afa3-f8c7d9b64aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:02.756432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a874a695f5e540a19116816f6d66a3dc', 'request_id': 'req-b88a8392-1d19-4554-bb49-20b2177113b7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:02.765862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a874a695f5e540a19116816f6d66a3dc', 'request_id': 'req-b88a8392-1d19-4554-bb49-20b2177113b7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:02.778491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a874a695f5e540a19116816f6d66a3dc', 'request_id': 'req-b88a8392-1d19-4554-bb49-20b2177113b7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:02.797866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] The token cache is being invalidate because project a874a695f5e540a19116816f6d66a3dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:02.798557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:02.799599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b88a8392-1d19-4554-bb49-20b2177113b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:02.804519 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3678/7401] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:02 2022] DELETE /identity/v3/projects/a874a695f5e540a19116816f6d66a3dc => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:02.809545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27727b7d-81b1-4cee-a5d0-cc9767cda783 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:02.810217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27727b7d-81b1-4cee-a5d0-cc9767cda783 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:02.811101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27727b7d-81b1-4cee-a5d0-cc9767cda783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:02.832301 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3724/7402] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:27:02.849222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-27727b7d-81b1-4cee-a5d0-cc9767cda783 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:02.850763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27727b7d-81b1-4cee-a5d0-cc9767cda783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:02.986930 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3679/7403] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:02.993437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565640f2-9536-4ef9-89bc-a5787d4eae3d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:02.994038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565640f2-9536-4ef9-89bc-a5787d4eae3d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:02.994573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565640f2-9536-4ef9-89bc-a5787d4eae3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.035626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-565640f2-9536-4ef9-89bc-a5787d4eae3d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:03.039352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-565640f2-9536-4ef9-89bc-a5787d4eae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:03.063087 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3725/7404] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:27:03.068168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27727b7d-81b1-4cee-a5d0-cc9767cda783 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:03.097290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27727b7d-81b1-4cee-a5d0-cc9767cda783 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:03.120369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:03.122341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.122729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.124167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.125716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:27:03.127898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:03.134911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:03.186606 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3680/7405] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:27:03.191300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565640f2-9536-4ef9-89bc-a5787d4eae3d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:03.207997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565640f2-9536-4ef9-89bc-a5787d4eae3d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:03.224515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:03.225495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.228227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.228978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.229842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:03.237165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:03.316706 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3726/7406] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:03 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:27:03.320207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:03.334936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af019-0600-47c7-99c0-bcf5e74e5880 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:03.448548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:03.454901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.456952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.456952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.459147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:03.471148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:03.518174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:03.526283 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3681/7407] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:03 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:03.530873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:03.557535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281626fd-c5db-4e5e-a04f-af94cb2899d5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:03.570331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:03.571520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.573767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.575646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.577036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:03.584606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:03.618311 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3727/7408] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:03.622107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:03.640104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1dff5-87ed-4d4b-9eaf-e0955c0c6bfc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:03.649381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38abe5-3e8e-464a-b693-7ee8153b831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:03.650124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38abe5-3e8e-464a-b693-7ee8153b831b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.650409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38abe5-3e8e-464a-b693-7ee8153b831b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.650732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38abe5-3e8e-464a-b693-7ee8153b831b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.651770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38abe5-3e8e-464a-b693-7ee8153b831b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:03.655003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38abe5-3e8e-464a-b693-7ee8153b831b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:03.674509 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3682/7409] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:03.679077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:03.695527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62cabe-7113-4201-a892-080b97a545b8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:03.702915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfa390-d9d2-40e0-a093-3145b800fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:03.704410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfa390-d9d2-40e0-a093-3145b800fe35 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.705051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfa390-d9d2-40e0-a093-3145b800fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.705666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfa390-d9d2-40e0-a093-3145b800fe35 admin admin] PATH_INFO: `/v3/projects/99e1389595b74807b615e1c6a289ca10/users/5ac35bd9df384cfcadd179543142ba80/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.729880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfa390-d9d2-40e0-a093-3145b800fe35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e1389595b74807b615e1c6a289ca10, user_id=5ac35bd9df384cfcadd179543142ba80, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:03.734006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfa390-d9d2-40e0-a093-3145b800fe35 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:03.757530 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3728/7410] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:03 2022] PUT /identity/v3/projects/99e1389595b74807b615e1c6a289ca10/users/5ac35bd9df384cfcadd179543142ba80/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:03.771373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1812686-ac4e-49d4-9f1f-4d06a92a4955 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.771561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1812686-ac4e-49d4-9f1f-4d06a92a4955 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.771811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1812686-ac4e-49d4-9f1f-4d06a92a4955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.783405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:03.784263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:03.784569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:03.885245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d1812686-ac4e-49d4-9f1f-4d06a92a4955 None None] MFA Rules not processed for user `5ac35bd9df384cfcadd179543142ba80`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:03.887221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d1812686-ac4e-49d4-9f1f-4d06a92a4955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:03.898497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] MFA Rules not processed for user `a0e47aca97434f09846b580091b06829`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:03.899772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:04.100943 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3683/7411] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:04.108347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1812686-ac4e-49d4-9f1f-4d06a92a4955 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:04.136997 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3729/7412] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:04.151105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:04.153209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:04.210237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:04.307524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1812686-ac4e-49d4-9f1f-4d06a92a4955 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:04.324012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:04.324841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:04.326391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:04.326683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:04.328855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:04.335815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:04.383553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f79d95-8c2b-4b4a-a201-5b75bf78276f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:04.409300 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3684/7413] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:04 2022] POST /identity/v3/projects => generated 385 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:04.410203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59120f-e060-4fbf-85bc-7995dd95d150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:04.410656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59120f-e060-4fbf-85bc-7995dd95d150 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:04.410936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59120f-e060-4fbf-85bc-7995dd95d150 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:04.414065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:04.414361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59120f-e060-4fbf-85bc-7995dd95d150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:04.444608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59120f-e060-4fbf-85bc-7995dd95d150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:04.449401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59120f-e060-4fbf-85bc-7995dd95d150 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:04.475296 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3730/7414] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:04.607352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a17bc-d979-4968-aa30-3c89b2121cdf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:04.607458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a17bc-d979-4968-aa30-3c89b2121cdf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:04.607724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a17bc-d979-4968-aa30-3c89b2121cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:04.622287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe37f8-13eb-4974-97c2-b57855457ac1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:04.642032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:04.643700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:04.644180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:04.645860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:04.648461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:04.655916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:04.672313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-de7a17bc-d979-4968-aa30-3c89b2121cdf None None] MFA Rules not processed for user `55b65e5b614143af8fcc5155671e8602`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:04.677556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-de7a17bc-d979-4968-aa30-3c89b2121cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:04.683245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:04.722056 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3685/7415] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:04 2022] POST /identity/v3/users => generated 299 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:27:04.723158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:04.744421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277887-fc52-4835-ba1f-ed33e8a50257 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:04.772892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:04.776106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:04.778312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:04.778608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:04.779868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:04.787890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:04.825871 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3686/7416] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:04.827051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:04.863732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5aae-72f1-4fcc-a5a7-287daf1ba440 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:04.877048 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3731/7417] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:04.878230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:04.880582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:04.883020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:04.884716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] PATH_INFO: `/v3/projects/9c8517e1994049f099d505507358ee6c/users/e78a5831ff154e0d8cd76029b705d941/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:04.949212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8517e1994049f099d505507358ee6c, user_id=e78a5831ff154e0d8cd76029b705d941, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:04.953493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:04.985951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3687/7418] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:04 2022] PUT /identity/v3/projects/9c8517e1994049f099d505507358ee6c/users/e78a5831ff154e0d8cd76029b705d941/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:04.991179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a17bc-d979-4968-aa30-3c89b2121cdf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:05.010279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a17bc-d979-4968-aa30-3c89b2121cdf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:05.283167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0dc41-9699-4929-b5c0-2aa4aba1425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:05.289621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0dc41-9699-4929-b5c0-2aa4aba1425a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:05.290316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0dc41-9699-4929-b5c0-2aa4aba1425a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:05.290746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0dc41-9699-4929-b5c0-2aa4aba1425a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:05.291693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0dc41-9699-4929-b5c0-2aa4aba1425a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:05.303412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0dc41-9699-4929-b5c0-2aa4aba1425a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:05.322736 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3732/7419] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:05.327903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:05.348276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52445a6c-b55c-4f94-ba56-fab582912915 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:05.355692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:05.357320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:05.357881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:05.358338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] PATH_INFO: `/v3/projects/9c8517e1994049f099d505507358ee6c/users/e78a5831ff154e0d8cd76029b705d941/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:05.360778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8517e1994049f099d505507358ee6c, user_id=e78a5831ff154e0d8cd76029b705d941, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:05.364012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:05.386121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3688/7420] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:05 2022] PUT /identity/v3/projects/9c8517e1994049f099d505507358ee6c/users/e78a5831ff154e0d8cd76029b705d941/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:05.404009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:05.404734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:05.405536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:05.477514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] MFA Rules not processed for user `e78a5831ff154e0d8cd76029b705d941`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:05.478910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:05.593556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:05.707384 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3733/7421] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:05.714040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:05.715793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:05.920777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0490-cf29-421f-8c0b-c663024dc2f7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:05.936708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2816ba5-cfc4-463f-ad2a-8fdb7bb8cf00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:05.938050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816ba5-cfc4-463f-ad2a-8fdb7bb8cf00 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:05.938547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816ba5-cfc4-463f-ad2a-8fdb7bb8cf00 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:05.938997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2816ba5-cfc4-463f-ad2a-8fdb7bb8cf00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:06.081508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2816ba5-cfc4-463f-ad2a-8fdb7bb8cf00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:06.085317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2816ba5-cfc4-463f-ad2a-8fdb7bb8cf00 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:06.106848 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3689/7422] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:06.793740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:06.796235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:06.857250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328b6b-923e-4f9a-9fc2-2bd2c19edb32 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:06.857726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328b6b-923e-4f9a-9fc2-2bd2c19edb32 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:06.857827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328b6b-923e-4f9a-9fc2-2bd2c19edb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:06.932774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bf328b6b-923e-4f9a-9fc2-2bd2c19edb32 None None] MFA Rules not processed for user `64090c8fd3394980ad5409adcde4986f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:06.938317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bf328b6b-923e-4f9a-9fc2-2bd2c19edb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:07.097848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36518e-6fdc-4f9d-a41e-c5c04912c2ce None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:07.110436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:07.112381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:07.113394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:07.113829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:07.114582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:07.118101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:07.165227 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3690/7423] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:07.182587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328b6b-923e-4f9a-9fc2-2bd2c19edb32 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:07.352405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:07.367780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:07.401321 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3734/7424] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:07.422485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328b6b-923e-4f9a-9fc2-2bd2c19edb32 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:07.440798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec95b72-a344-4026-b147-c31096a7ff25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:07.441455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec95b72-a344-4026-b147-c31096a7ff25 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:07.441732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec95b72-a344-4026-b147-c31096a7ff25 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:07.442004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec95b72-a344-4026-b147-c31096a7ff25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:07.460899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec95b72-a344-4026-b147-c31096a7ff25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:07.463102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec95b72-a344-4026-b147-c31096a7ff25 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:07.498939 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3691/7425] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:27:11.974118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:11.996372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0ebe7-ae36-4671-b2f3-8516f373dabd service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:12.011310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:12.012802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:12.013286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:12.014465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:12.031672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:12.033784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:12.066052 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3735/7426] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:13.823167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fb546-b631-4d8e-a019-05912f676e23 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:13.823781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fb546-b631-4d8e-a019-05912f676e23 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:13.823781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fb546-b631-4d8e-a019-05912f676e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:13.848192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e21fb546-b631-4d8e-a019-05912f676e23 None None] MFA Rules not processed for user `64090c8fd3394980ad5409adcde4986f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:13.850351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e21fb546-b631-4d8e-a019-05912f676e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:13.862315 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3692/7427] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:17.778740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:17.845413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212182ee-5e93-44b2-9608-71aef175e8a4 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:17.851264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:17.853379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:17.853939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:17.854382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] PATH_INFO: `/v3/users/5ac35bd9df384cfcadd179543142ba80` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:17.855570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac35bd9df384cfcadd179543142ba80)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:17.857638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:17.864523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:17.920292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac35bd9df384cfcadd179543142ba80'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:17.921688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac35bd9df384cfcadd179543142ba80'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:17.935461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac35bd9df384cfcadd179543142ba80'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:17.946352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac35bd9df384cfcadd179543142ba80'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:17.947259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac35bd9df384cfcadd179543142ba80'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:17.997599 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3736/7428] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:17 2022] DELETE /identity/v3/users/5ac35bd9df384cfcadd179543142ba80 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:18.002558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:18.003041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:18.003124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:18.029893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:18.031114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:18.123732 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3693/7429] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:18.131571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:18.134045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:18.274196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416607d-7567-41a5-af6b-9bab5c4cf29a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:18.280493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:18.281211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:18.281756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:18.282236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:18.294978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:18.296822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:18.326195 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3737/7430] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:18.694606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:18.696412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:18.716896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49cd54-dbca-4e69-8336-5d2d058e92b4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:18.723304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:18.724015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:18.724857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:18.725296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] PATH_INFO: `/v3/projects/99e1389595b74807b615e1c6a289ca10` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:18.727150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e1389595b74807b615e1c6a289ca10)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:18.729457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:18.805451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e1389595b74807b615e1c6a289ca10', 'request_id': 'req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:18.819217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e1389595b74807b615e1c6a289ca10', 'request_id': 'req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:18.834540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e1389595b74807b615e1c6a289ca10', 'request_id': 'req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:18.838119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:18.852972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] The token cache is being invalidate because project 99e1389595b74807b615e1c6a289ca10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:18.853405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:18.854303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:18.859311 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3694/7431] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:18 2022] DELETE /identity/v3/projects/99e1389595b74807b615e1c6a289ca10 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:18.863535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:18.877440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb33791-312e-4939-8b3e-9241c9bd95b3 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:18.980758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:18.981277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:18.981555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:18.981825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] PATH_INFO: `/v3/users/e78a5831ff154e0d8cd76029b705d941` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:18.983054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78a5831ff154e0d8cd76029b705d941)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:18.985039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:19.025682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:19.045094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78a5831ff154e0d8cd76029b705d941'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.046203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78a5831ff154e0d8cd76029b705d941'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.058568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78a5831ff154e0d8cd76029b705d941'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.069586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78a5831ff154e0d8cd76029b705d941'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.070297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-99afabe2-2970-4132-bd0d-48dd0225863e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78a5831ff154e0d8cd76029b705d941'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.091291 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3738/7432] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:18 2022] DELETE /identity/v3/users/e78a5831ff154e0d8cd76029b705d941 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:19.406758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:19.408239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:19.422034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af2d0-04fe-4c26-b412-d9d1efc9d7ab admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:19.489754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:19.490595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.490834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.491075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] PATH_INFO: `/v3/projects/9c8517e1994049f099d505507358ee6c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.492289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c8517e1994049f099d505507358ee6c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:19.494371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:19.545410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c8517e1994049f099d505507358ee6c', 'request_id': 'req-b27dcceb-4529-4dfa-a072-76d8023bf987'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.551713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c8517e1994049f099d505507358ee6c', 'request_id': 'req-b27dcceb-4529-4dfa-a072-76d8023bf987'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.562798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c8517e1994049f099d505507358ee6c', 'request_id': 'req-b27dcceb-4529-4dfa-a072-76d8023bf987'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.573737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] The token cache is being invalidate because project 9c8517e1994049f099d505507358ee6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:19.574149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.574917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b27dcceb-4529-4dfa-a072-76d8023bf987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:19.576438 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3695/7433] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:19 2022] DELETE /identity/v3/projects/9c8517e1994049f099d505507358ee6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:19.582480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0223-1ff1-4fed-8206-8e0ab9b36974 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.582790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0223-1ff1-4fed-8206-8e0ab9b36974 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.583065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0223-1ff1-4fed-8206-8e0ab9b36974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.603393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7f7e0223-1ff1-4fed-8206-8e0ab9b36974 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:19.604636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7f7e0223-1ff1-4fed-8206-8e0ab9b36974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:19.669955 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3739/7434] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:19.678338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57946fd-8537-4644-99cb-3c7e6d9a3d2b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.678625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57946fd-8537-4644-99cb-3c7e6d9a3d2b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.678899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57946fd-8537-4644-99cb-3c7e6d9a3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.699600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f57946fd-8537-4644-99cb-3c7e6d9a3d2b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:19.700914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f57946fd-8537-4644-99cb-3c7e6d9a3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:19.709917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3696/7435] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:27:19.714604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0223-1ff1-4fed-8206-8e0ab9b36974 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:19.725416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0223-1ff1-4fed-8206-8e0ab9b36974 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:19.730593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:19.731214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.731491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.731757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.732468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:27:19.732810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:19.734476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:19.751251 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3740/7436] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:27:19.752762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57946fd-8537-4644-99cb-3c7e6d9a3d2b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:19.763796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57946fd-8537-4644-99cb-3c7e6d9a3d2b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:19.771863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:19.772223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.772571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.773004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.775123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:19.777282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:19.810854 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3697/7437] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:19 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:19.815123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:19.828597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7e5fa-d03b-43f3-aaa8-8d3ac5e26d8a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:19.899510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:19.900215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.900401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.900706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.901411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:19.903581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:19.930849 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3741/7438] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:19 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:19.932263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:19.945245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6010e-685d-4e44-a08d-ae20c317d78e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:19.953006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:19.953648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.954740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.955028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.955694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:19.958066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:19.972191 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3698/7439] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:19.977402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:19.988311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dd80a-de2c-4b71-aeb4-386e48f2cf04 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:19.994812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67163785-d2b8-4162-b774-e85d35b02d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:19.996280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67163785-d2b8-4162-b774-e85d35b02d65 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:19.996557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67163785-d2b8-4162-b774-e85d35b02d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:19.996879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67163785-d2b8-4162-b774-e85d35b02d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:19.998028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67163785-d2b8-4162-b774-e85d35b02d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:20.000142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67163785-d2b8-4162-b774-e85d35b02d65 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:20.012062 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3742/7440] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:20.016190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:20.028335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaa0f1-874d-456a-b0f3-18ac22727f4a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:20.033415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:20.035599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:20.036488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:20.036800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] PATH_INFO: `/v3/projects/f3b0c51c56bb4f62892ad8ab972fd550/users/920de6c5d8564485968c5bc6a8d742bd/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:20.052341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b0c51c56bb4f62892ad8ab972fd550, user_id=920de6c5d8564485968c5bc6a8d742bd, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:20.055059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:20.071925 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3699/7441] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:20 2022] PUT /identity/v3/projects/f3b0c51c56bb4f62892ad8ab972fd550/users/920de6c5d8564485968c5bc6a8d742bd/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:20.084302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129803e6-aca9-4f95-93e4-ad036375f94f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:20.084589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129803e6-aca9-4f95-93e4-ad036375f94f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:20.084985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129803e6-aca9-4f95-93e4-ad036375f94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:20.112300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:20.115775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-129803e6-aca9-4f95-93e4-ad036375f94f None None] MFA Rules not processed for user `920de6c5d8564485968c5bc6a8d742bd`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:20.117920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-129803e6-aca9-4f95-93e4-ad036375f94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:20.185326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3743/7442] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:20.196778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:20.200699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:20.224143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:20.291303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341ef6-045b-40d3-96ab-ed6324e55980 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:20.295272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35a24a-91dd-4945-b802-03f1016892e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:20.295891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35a24a-91dd-4945-b802-03f1016892e2 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:20.296172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35a24a-91dd-4945-b802-03f1016892e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:20.296443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35a24a-91dd-4945-b802-03f1016892e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:20.305773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35a24a-91dd-4945-b802-03f1016892e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:20.307215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35a24a-91dd-4945-b802-03f1016892e2 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:20.316838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3700/7443] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:22.717288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:22.717958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:22.717958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:22.739134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] MFA Rules not processed for user `97c769dbba794531b0a9db8e764ff097`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:22.740534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:22.802551 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3744/7444] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:23.327120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ae89-c8a0-41be-aba4-e4a223e93a19 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:23.327664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ae89-c8a0-41be-aba4-e4a223e93a19 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:23.327664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ae89-c8a0-41be-aba4-e4a223e93a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:23.350358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6579ae89-c8a0-41be-aba4-e4a223e93a19 None None] MFA Rules not processed for user `55b65e5b614143af8fcc5155671e8602`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:23.351570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6579ae89-c8a0-41be-aba4-e4a223e93a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:23.440295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3701/7445] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:23.508084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:23.510178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:23.654978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a8fc0-9218-4ba2-b553-86dfb3663fbc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:23.668760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:23.669784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:23.670218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:23.670876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:23.673758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:23.696352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:23.698246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:23.714552 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3745/7446] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:26.442832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579ae89-c8a0-41be-aba4-e4a223e93a19 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:26.568305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:26.570520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:26.605682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579ae89-c8a0-41be-aba4-e4a223e93a19 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:26.625522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:26.627154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:26.627624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:26.627903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] PATH_INFO: `/v3/users/920de6c5d8564485968c5bc6a8d742bd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:26.629944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=920de6c5d8564485968c5bc6a8d742bd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:26.633264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:26.652204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:26.682085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98413709-c665-4c4f-8785-3724a4b772b9 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:26.701947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848ba96-de38-4342-af1d-f2be8c7e335f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:26.702934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ba96-de38-4342-af1d-f2be8c7e335f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:26.703384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ba96-de38-4342-af1d-f2be8c7e335f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:26.703812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ba96-de38-4342-af1d-f2be8c7e335f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:26.708921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d848ba96-de38-4342-af1d-f2be8c7e335f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:26.730585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848ba96-de38-4342-af1d-f2be8c7e335f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:26.732705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848ba96-de38-4342-af1d-f2be8c7e335f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:26.742776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920de6c5d8564485968c5bc6a8d742bd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:26.755164 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3746/7447] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:26.757644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920de6c5d8564485968c5bc6a8d742bd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:26.775650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920de6c5d8564485968c5bc6a8d742bd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:26.778345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:26.794871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920de6c5d8564485968c5bc6a8d742bd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:26.834610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920de6c5d8564485968c5bc6a8d742bd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:26.848878 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3702/7448] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:26 2022] DELETE /identity/v3/users/920de6c5d8564485968c5bc6a8d742bd => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:26.851606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:26.852076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:26.852360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:26.906206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:26.908332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:27.028782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3747/7449] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:27.039434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:27.041146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:27.144438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fa66a-6c92-484d-a8fa-c86f8ff5b70c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:27.150401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ec822-ed3c-4cba-975e-73bbc77a1b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:27.151050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec822-ed3c-4cba-975e-73bbc77a1b9c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:27.151325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec822-ed3c-4cba-975e-73bbc77a1b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:27.152192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec822-ed3c-4cba-975e-73bbc77a1b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:27.165035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ec822-ed3c-4cba-975e-73bbc77a1b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:27.166541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ec822-ed3c-4cba-975e-73bbc77a1b9c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:27.180267 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3703/7450] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:27.936367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:27.939992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:27.958983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfbd4d-8014-4e33-8c8c-adf9ea2d920a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:27.965446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:27.966222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:27.966746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:27.967213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] PATH_INFO: `/v3/projects/f3b0c51c56bb4f62892ad8ab972fd550` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:27.968714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3b0c51c56bb4f62892ad8ab972fd550)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:27.971148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.050500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3b0c51c56bb4f62892ad8ab972fd550', 'request_id': 'req-97a129d8-a4af-4eb8-abf6-14c585a41321'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:28.063587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3b0c51c56bb4f62892ad8ab972fd550', 'request_id': 'req-97a129d8-a4af-4eb8-abf6-14c585a41321'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:28.072918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3b0c51c56bb4f62892ad8ab972fd550', 'request_id': 'req-97a129d8-a4af-4eb8-abf6-14c585a41321'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:28.086489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] The token cache is being invalidate because project f3b0c51c56bb4f62892ad8ab972fd550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:28.086771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:28.087873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-97a129d8-a4af-4eb8-abf6-14c585a41321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:28.089715 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3748/7451] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:27 2022] DELETE /identity/v3/projects/f3b0c51c56bb4f62892ad8ab972fd550 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:28.095427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7ca11-565c-4f2e-b23b-3ff221372f44 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.096162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7ca11-565c-4f2e-b23b-3ff221372f44 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.096523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7ca11-565c-4f2e-b23b-3ff221372f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.122902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-efa7ca11-565c-4f2e-b23b-3ff221372f44 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:28.125261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-efa7ca11-565c-4f2e-b23b-3ff221372f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:28.207555 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3704/7452] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:28.217396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3a9c2-b4b4-444a-8918-092ea6609abf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.218006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3a9c2-b4b4-444a-8918-092ea6609abf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.218387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3a9c2-b4b4-444a-8918-092ea6609abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.241919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1cc3a9c2-b4b4-444a-8918-092ea6609abf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:28.243171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1cc3a9c2-b4b4-444a-8918-092ea6609abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:28.252154 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3749/7453] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:28.257027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7ca11-565c-4f2e-b23b-3ff221372f44 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:28.268188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7ca11-565c-4f2e-b23b-3ff221372f44 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:28.273465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:28.274513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.274513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.275403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.276074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:27:28.276378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:28.278204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.292437 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3705/7454] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:27:28.296504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3a9c2-b4b4-444a-8918-092ea6609abf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:28.308404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3a9c2-b4b4-444a-8918-092ea6609abf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:28.312698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:28.313392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.313735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.314067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.314778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:28.316943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.352998 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3750/7455] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:28 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:28.357336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:28.372366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab949be-9bdb-4cd7-88bd-26c973703aa7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:28.454696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:28.455332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.455638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.455908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.456582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:28.459051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.470788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:28.486942 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3706/7456] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:28 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:28.491763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:28.506502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8bfe-7767-4089-9b5d-de32dfbb56d4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:28.512103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:28.512897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.513243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.513639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.514576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:28.516381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.528520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3751/7457] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:27:28.532442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:28.543293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91765f99-bc11-45b6-b953-09d0b080f83a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:28.548443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0a4e1-03c7-4a34-a654-eaf01b802576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:28.549175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0a4e1-03c7-4a34-a654-eaf01b802576 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.549458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0a4e1-03c7-4a34-a654-eaf01b802576 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.549730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0a4e1-03c7-4a34-a654-eaf01b802576 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.550380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0a4e1-03c7-4a34-a654-eaf01b802576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:28.552130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0a4e1-03c7-4a34-a654-eaf01b802576 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.564167 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3707/7458] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:27:28.569165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:28.580810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2ce5-5ab3-40e7-b243-507a3d9743dc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:28.586542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:28.587065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.587873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.588198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] PATH_INFO: `/v3/projects/fade501d78244c13bd31fddc884c8981/users/0b207e8521b74e7bb783b9dfbc8f98ad/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.606973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fade501d78244c13bd31fddc884c8981, user_id=0b207e8521b74e7bb783b9dfbc8f98ad, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:28.610110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.623114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:28.629889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3752/7459] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:28 2022] PUT /identity/v3/projects/fade501d78244c13bd31fddc884c8981/users/0b207e8521b74e7bb783b9dfbc8f98ad/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:28.641491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.641882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.642264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.672307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] MFA Rules not processed for user `0b207e8521b74e7bb783b9dfbc8f98ad`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:28.673765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:28.760744 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3708/7460] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:28.776049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:28.778041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:28.865479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77ceb-b7d2-40ea-b5a4-f529fb8e3b4d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:28.869265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:28.869884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:28.870149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:28.870414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:28.881098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:28.882675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:28.892848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3753/7461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:31.523076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:31.524633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:31.653142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684c74d-206d-4c1b-b7f7-cb5d96fa50fc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:31.664196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e4bbe-0ba3-4833-9343-b5d70bde4dab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:31.665459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4bbe-0ba3-4833-9343-b5d70bde4dab service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:31.665459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4bbe-0ba3-4833-9343-b5d70bde4dab service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:31.665726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4bbe-0ba3-4833-9343-b5d70bde4dab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:31.668308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-295e4bbe-0ba3-4833-9343-b5d70bde4dab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:31.687684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e4bbe-0ba3-4833-9343-b5d70bde4dab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:31.689475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e4bbe-0ba3-4833-9343-b5d70bde4dab service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:31.716704 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3709/7462] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:32.204858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:32.301432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85fb09-9d0f-4629-9bc9-0d1ad4c0790d service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:32.309092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:32.310016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:32.310770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:32.311116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] PATH_INFO: `/v3/users/0b207e8521b74e7bb783b9dfbc8f98ad` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:32.312971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b207e8521b74e7bb783b9dfbc8f98ad)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:32.315825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:32.386268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b207e8521b74e7bb783b9dfbc8f98ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:32.387012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b207e8521b74e7bb783b9dfbc8f98ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:32.403470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b207e8521b74e7bb783b9dfbc8f98ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:32.415193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b207e8521b74e7bb783b9dfbc8f98ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:32.416410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b207e8521b74e7bb783b9dfbc8f98ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:32.441235 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3754/7463] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:32 2022] DELETE /identity/v3/users/0b207e8521b74e7bb783b9dfbc8f98ad => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:32.446470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:32.446756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:32.447091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:32.471556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:32.472870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:32.544413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:32.565538 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3710/7464] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:32.577506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:32.578952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:32.623191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:32.683245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6768-fa68-4ea2-b6cc-58d62f5039bc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:32.688155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992d8e-b340-418c-9425-c5817d7c989b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:32.689565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992d8e-b340-418c-9425-c5817d7c989b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:32.689886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992d8e-b340-418c-9425-c5817d7c989b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:32.690165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992d8e-b340-418c-9425-c5817d7c989b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:32.705042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1992d8e-b340-418c-9425-c5817d7c989b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:32.707105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1992d8e-b340-418c-9425-c5817d7c989b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:32.721513 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3755/7465] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:33.052163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.053982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:33.091430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff1f08-46a2-48f4-bdae-97d3abf28d0e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.101713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.101713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.101713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.102107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] PATH_INFO: `/v3/projects/fade501d78244c13bd31fddc884c8981` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.104505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fade501d78244c13bd31fddc884c8981)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.107824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.176271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fade501d78244c13bd31fddc884c8981', 'request_id': 'req-c512f712-a50d-4870-b501-9d13ed448706'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:33.184552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fade501d78244c13bd31fddc884c8981', 'request_id': 'req-c512f712-a50d-4870-b501-9d13ed448706'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:33.195963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fade501d78244c13bd31fddc884c8981', 'request_id': 'req-c512f712-a50d-4870-b501-9d13ed448706'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:33.212160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] The token cache is being invalidate because project fade501d78244c13bd31fddc884c8981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:33.212507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:33.213719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c512f712-a50d-4870-b501-9d13ed448706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:33.215669 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3711/7466] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:33 2022] DELETE /identity/v3/projects/fade501d78244c13bd31fddc884c8981 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:27:33.222184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a54d-49f3-41ff-90ec-136da9fe2b15 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.222727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a54d-49f3-41ff-90ec-136da9fe2b15 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.223076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a54d-49f3-41ff-90ec-136da9fe2b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.247148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d135a54d-49f3-41ff-90ec-136da9fe2b15 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:33.248757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d135a54d-49f3-41ff-90ec-136da9fe2b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:33.322304 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3756/7467] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:33.330692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca19fe-8a92-414a-b031-799e25aff278 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.331146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca19fe-8a92-414a-b031-799e25aff278 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.331461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca19fe-8a92-414a-b031-799e25aff278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.353206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-71ca19fe-8a92-414a-b031-799e25aff278 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:33.354487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71ca19fe-8a92-414a-b031-799e25aff278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:33.363712 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3712/7468] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:33.368367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135a54d-49f3-41ff-90ec-136da9fe2b15 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.386753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135a54d-49f3-41ff-90ec-136da9fe2b15 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.392866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.393914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.394094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.394506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.395377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:27:33.396294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.398167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.412626 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3757/7469] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:27:33.418020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca19fe-8a92-414a-b031-799e25aff278 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.435704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca19fe-8a92-414a-b031-799e25aff278 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.441431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.442059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.442334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.442631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.443280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.445381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.484887 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3713/7470] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:33 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:33.486194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.498103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdd1ab-3d20-47f9-ad14-375991e1e425 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.592308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.593186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.593523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.593770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.594502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.596928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.627772 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3758/7471] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:33 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:33.632827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.645175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebb0bb-b7ba-4d2c-bb74-81a67f7dc41b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.650912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.651539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.651820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.652093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.652789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.654523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.667995 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3714/7472] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:27:33.672359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.683764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286366d-b08c-4bad-a4b5-b0138b0359e2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.690668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.691400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.691806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.692108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.693132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.694781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.734223 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3759/7473] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:33.738599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.756249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495fd25-f0ac-4243-b2b7-a2bbfb747bc6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.762740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea56dff-6197-48db-af31-b25c43a299c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.763721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea56dff-6197-48db-af31-b25c43a299c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.764013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea56dff-6197-48db-af31-b25c43a299c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.764585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea56dff-6197-48db-af31-b25c43a299c4 admin admin] PATH_INFO: `/v3/projects/7735186883cb4234bbb276780101cecb/users/e2a8653304ce4425971752f838a0ea01/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.790121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.792133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:33.805967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea56dff-6197-48db-af31-b25c43a299c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7735186883cb4234bbb276780101cecb, user_id=e2a8653304ce4425971752f838a0ea01, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.810742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea56dff-6197-48db-af31-b25c43a299c4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.839304 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3715/7474] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:33 2022] PUT /identity/v3/projects/7735186883cb4234bbb276780101cecb/users/e2a8653304ce4425971752f838a0ea01/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:33.841375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad05f2-821e-4268-b613-47fe7b3d7c4b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.841809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad05f2-821e-4268-b613-47fe7b3d7c4b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.842031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad05f2-821e-4268-b613-47fe7b3d7c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.847788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:33.892173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049259-c01f-4404-80dc-a88a3bc0ef29 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:33.892436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6fad05f2-821e-4268-b613-47fe7b3d7c4b None None] MFA Rules not processed for user `e2a8653304ce4425971752f838a0ea01`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:33.893764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6fad05f2-821e-4268-b613-47fe7b3d7c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:33.961525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:33.963343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:33.963904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:33.964313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:33.966674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:33.980070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3716/7475] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:27:33.980919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:33.982510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:33.984490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad05f2-821e-4268-b613-47fe7b3d7c4b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:33.994614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:33.995758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3760/7476] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:27:34.068838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad05f2-821e-4268-b613-47fe7b3d7c4b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.078479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.079297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.079784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.080024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.080849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.082858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.124979 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3717/7477] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:33 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:27:34.129331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.143000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbad20-6a22-4402-8775-68ad278517ef service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.211750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.213082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.213366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.213636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.214441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.216442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.246798 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3761/7478] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:34 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:34.251530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.275866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568e2e5-47cf-4076-90dd-75e1c5b5135e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.282160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.283534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.284456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.284843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.285616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.287459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.301528 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3718/7479] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:34.305682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.317713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81b3d9-313a-488e-8050-41214475fec5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.322608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55bea9-3719-4797-bb80-5825bf8fca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.323301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55bea9-3719-4797-bb80-5825bf8fca1c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.323614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55bea9-3719-4797-bb80-5825bf8fca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.323902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55bea9-3719-4797-bb80-5825bf8fca1c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.324539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55bea9-3719-4797-bb80-5825bf8fca1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.326171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55bea9-3719-4797-bb80-5825bf8fca1c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.336652 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3762/7480] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:34.341047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.353365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736540b7-4ec1-4481-bdda-fb0852037a9b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.358092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.358764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.359068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.359345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] PATH_INFO: `/v3/projects/671e101ddddd4ec1bbc287c24d1b52da/users/14750d3883984bb59dbafbf4e40510ea/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.374921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671e101ddddd4ec1bbc287c24d1b52da, user_id=14750d3883984bb59dbafbf4e40510ea, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.377702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.394222 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3719/7481] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:34 2022] PUT /identity/v3/projects/671e101ddddd4ec1bbc287c24d1b52da/users/14750d3883984bb59dbafbf4e40510ea/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:34.405868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503500-2569-4e3e-80fe-888c87ac9073 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.406156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503500-2569-4e3e-80fe-888c87ac9073 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.406434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503500-2569-4e3e-80fe-888c87ac9073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.435324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-56503500-2569-4e3e-80fe-888c87ac9073 None None] MFA Rules not processed for user `14750d3883984bb59dbafbf4e40510ea`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:34.436614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-56503500-2569-4e3e-80fe-888c87ac9073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:34.498252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:34.504855 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3763/7482] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:34.509999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.591228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a117a53-68b8-4631-9ca1-5e80b886de56 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.597208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.597830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.598117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.598388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.599166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.601142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.631650 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3720/7483] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:34 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:34.635994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56503500-2569-4e3e-80fe-888c87ac9073 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.647880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56503500-2569-4e3e-80fe-888c87ac9073 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.712286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.713298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.713574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.713844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.714482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.716419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.744029 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3764/7484] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:34 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:34.746988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.783335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21a792-9aa6-43a5-b6cd-f67a1b7d33c3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.790388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.790723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.791001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.791318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.792108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.793849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.811578 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3721/7485] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:34.814755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.826695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56101954-105c-44bb-afd2-882cceb388c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.831708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.832374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.832670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.832980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] PATH_INFO: `/v3/projects/02c3e39929364a369a731387f2137ca4/users/57756bf7b4b343fabc78c93b7687d837/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.850020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c3e39929364a369a731387f2137ca4, user_id=57756bf7b4b343fabc78c93b7687d837, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.852992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.868246 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3765/7486] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:34 2022] PUT /identity/v3/projects/02c3e39929364a369a731387f2137ca4/users/57756bf7b4b343fabc78c93b7687d837/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:34.872566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.883448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb9668-e7e0-4f0f-a046-e4fbd27758c5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.913943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:34.948805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d6317-1f47-452f-9bfb-cf191a865d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.949451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d6317-1f47-452f-9bfb-cf191a865d4c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.949731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d6317-1f47-452f-9bfb-cf191a865d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.950004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d6317-1f47-452f-9bfb-cf191a865d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.950650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d6317-1f47-452f-9bfb-cf191a865d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.952298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d6317-1f47-452f-9bfb-cf191a865d4c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:34.964461 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3722/7487] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:34.969585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:34.981898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce640999-2b61-4292-9165-7f7b11bfb4df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:34.988084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:34.988953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:34.989412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:34.989826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] PATH_INFO: `/v3/projects/02c3e39929364a369a731387f2137ca4/users/57756bf7b4b343fabc78c93b7687d837/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:34.992485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c3e39929364a369a731387f2137ca4, user_id=57756bf7b4b343fabc78c93b7687d837, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:34.996227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:35.011054 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3766/7488] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:34 2022] PUT /identity/v3/projects/02c3e39929364a369a731387f2137ca4/users/57756bf7b4b343fabc78c93b7687d837/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:35.022569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:35.023081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:35.023363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:35.054472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] MFA Rules not processed for user `57756bf7b4b343fabc78c93b7687d837`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:35.055763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:35.139218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3723/7489] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:35.150903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:35.152646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:35.225597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb34ee-c0cd-4ed5-80ba-c7c62d2c636b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:35.229626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:35.230305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:35.230650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:35.230956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:35.310678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:35.312955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:35.324601 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3767/7490] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:36.861285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:36.862747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:36.877926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691495-272f-415a-93d9-a02cf7cad4b0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:36.882275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c2c32-bee7-4cd1-8406-b69668bb33ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:36.882893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c2c32-bee7-4cd1-8406-b69668bb33ba service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:36.883175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c2c32-bee7-4cd1-8406-b69668bb33ba service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:36.883639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c2c32-bee7-4cd1-8406-b69668bb33ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:36.956019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c2c32-bee7-4cd1-8406-b69668bb33ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:36.957477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c2c32-bee7-4cd1-8406-b69668bb33ba service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:36.970949 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3724/7491] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:37.854142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:37.860153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:37.866382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658fa91-014d-4eb7-95d2-7662d7b2a056 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:37.876544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:37.877625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:37.878171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:37.878358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:37.890929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:37.892561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:37.915836 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3768/7492] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:39.216235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:39.216783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:39.216909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:39.240375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:39.242065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:39.298327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:39.311843 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3725/7493] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:27:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:39.379270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:39.382476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:39.537376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98396c55-e11a-47b3-a157-15ec17c931e4 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:39.547169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:39.548389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:39.548389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:39.548647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:39.565570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:39.567638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:39.578527 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3769/7494] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:27:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:27:40.357424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:40.359054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:40.572856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67669bb8-a68f-4ba6-b6b4-53b44ae2c6ea None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:40.579396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3387f9-5ed6-4fdf-b8cc-9e4f30b10392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:40.580198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3387f9-5ed6-4fdf-b8cc-9e4f30b10392 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:40.580478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3387f9-5ed6-4fdf-b8cc-9e4f30b10392 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:40.580921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3387f9-5ed6-4fdf-b8cc-9e4f30b10392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:40.583723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7e3387f9-5ed6-4fdf-b8cc-9e4f30b10392 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:40.673136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3387f9-5ed6-4fdf-b8cc-9e4f30b10392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:40.676126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3387f9-5ed6-4fdf-b8cc-9e4f30b10392 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:40.721021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3726/7495] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:42.282196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:42.397144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337e0b5-c449-44cf-97e8-562dbd883bec service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:42.407348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:42.408029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:42.408305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:42.408575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] PATH_INFO: `/v3/users/e2a8653304ce4425971752f838a0ea01` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:42.409909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a8653304ce4425971752f838a0ea01)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:42.411882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:42.489446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a8653304ce4425971752f838a0ea01'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:42.490478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a8653304ce4425971752f838a0ea01'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:42.506676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a8653304ce4425971752f838a0ea01'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:42.521251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a8653304ce4425971752f838a0ea01'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:42.522727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a8653304ce4425971752f838a0ea01'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:42.549705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:42.563098 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3770/7496] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:42 2022] DELETE /identity/v3/users/e2a8653304ce4425971752f838a0ea01 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:42.569399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:42.570343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:42.570343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:42.629367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:42.630754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:42.754997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3727/7497] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:42.766141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:42.767605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:42.855594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d740e-3f3d-430d-ba26-a4a25d6e2eca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:42.865129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:42.865977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:42.866270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:42.866595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:42.882443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:42.885884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:42.898943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3771/7498] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:43.288941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:43.290788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:43.306028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb555a32-6695-4269-b557-356dfc1dc02d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:43.329019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:43.332569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:43.332569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:43.332569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] PATH_INFO: `/v3/projects/7735186883cb4234bbb276780101cecb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:43.333353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7735186883cb4234bbb276780101cecb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:43.337066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:43.410752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7735186883cb4234bbb276780101cecb', 'request_id': 'req-bc74b3a4-13e2-44c6-8504-573de3c125d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.429445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7735186883cb4234bbb276780101cecb', 'request_id': 'req-bc74b3a4-13e2-44c6-8504-573de3c125d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.446031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7735186883cb4234bbb276780101cecb', 'request_id': 'req-bc74b3a4-13e2-44c6-8504-573de3c125d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.460429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] The token cache is being invalidate because project 7735186883cb4234bbb276780101cecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:43.460918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.461834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.463775 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3728/7499] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:43 2022] DELETE /identity/v3/projects/7735186883cb4234bbb276780101cecb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:43.468169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:43.482091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c092e-b518-424c-a747-26801446375a service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:43.569919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:43.570848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:43.571011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:43.571399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] PATH_INFO: `/v3/users/14750d3883984bb59dbafbf4e40510ea` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:43.573487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14750d3883984bb59dbafbf4e40510ea)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:43.577727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:43.657583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14750d3883984bb59dbafbf4e40510ea'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.661428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14750d3883984bb59dbafbf4e40510ea'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.674499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14750d3883984bb59dbafbf4e40510ea'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.685806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14750d3883984bb59dbafbf4e40510ea'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.686598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14750d3883984bb59dbafbf4e40510ea'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:43.714080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3772/7500] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:43 2022] DELETE /identity/v3/users/14750d3883984bb59dbafbf4e40510ea => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:44.063661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:44.065420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:44.078075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:44.083741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74b3a4-13e2-44c6-8504-573de3c125d2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:44.182833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:44.185487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:44.185487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:44.185487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] PATH_INFO: `/v3/projects/671e101ddddd4ec1bbc287c24d1b52da` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:44.185857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] RBAC: Authorizing `identity:delete_project(project_id=671e101ddddd4ec1bbc287c24d1b52da)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:44.188185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:44.287248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '671e101ddddd4ec1bbc287c24d1b52da', 'request_id': 'req-6455148a-cfd8-4bce-b4be-db5b694d957f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.299735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '671e101ddddd4ec1bbc287c24d1b52da', 'request_id': 'req-6455148a-cfd8-4bce-b4be-db5b694d957f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.320476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '671e101ddddd4ec1bbc287c24d1b52da', 'request_id': 'req-6455148a-cfd8-4bce-b4be-db5b694d957f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.363551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] The token cache is being invalidate because project 671e101ddddd4ec1bbc287c24d1b52da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:44.365234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.367494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.371275 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3729/7501] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:44 2022] DELETE /identity/v3/projects/671e101ddddd4ec1bbc287c24d1b52da => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:44.376298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:44.379155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:44.395163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70d238-fad1-4576-a45e-23902c9a41d9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:44.427872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:44.491892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:44.493534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:44.494117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:44.494669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] PATH_INFO: `/v3/users/57756bf7b4b343fabc78c93b7687d837` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:44.496288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] RBAC: Authorizing `identity:delete_user(user_id=57756bf7b4b343fabc78c93b7687d837)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:44.498473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:44.594312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57756bf7b4b343fabc78c93b7687d837'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.595656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57756bf7b4b343fabc78c93b7687d837'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.613482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57756bf7b4b343fabc78c93b7687d837'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.641067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57756bf7b4b343fabc78c93b7687d837'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.642098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-50339c1f-a094-493c-a384-44c661e6cbda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57756bf7b4b343fabc78c93b7687d837'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:44.672061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3773/7502] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:44 2022] DELETE /identity/v3/users/57756bf7b4b343fabc78c93b7687d837 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:45.090073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:45.093711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:45.120561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455148a-cfd8-4bce-b4be-db5b694d957f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:45.258827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:45.259486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:45.259765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:45.260108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] PATH_INFO: `/v3/projects/02c3e39929364a369a731387f2137ca4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:45.261261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c3e39929364a369a731387f2137ca4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:45.263340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:45.375435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:45.393186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c3e39929364a369a731387f2137ca4', 'request_id': 'req-68806e32-f70c-413d-adc3-fced0cae88b4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:45.410786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c3e39929364a369a731387f2137ca4', 'request_id': 'req-68806e32-f70c-413d-adc3-fced0cae88b4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:45.430081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c3e39929364a369a731387f2137ca4', 'request_id': 'req-68806e32-f70c-413d-adc3-fced0cae88b4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:45.452518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] The token cache is being invalidate because project 02c3e39929364a369a731387f2137ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:45.452518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:45.453756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68806e32-f70c-413d-adc3-fced0cae88b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:45.456210 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3730/7503] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:45 2022] DELETE /identity/v3/projects/02c3e39929364a369a731387f2137ca4 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:45.461558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9234-306d-4d32-be7d-a5acfd371dca None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:45.461806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9234-306d-4d32-be7d-a5acfd371dca None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:45.462524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9234-306d-4d32-be7d-a5acfd371dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:45.505272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cf8c9234-306d-4d32-be7d-a5acfd371dca None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:45.506875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf8c9234-306d-4d32-be7d-a5acfd371dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:45.634103 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3774/7504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:45.643707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d8d8-e7bd-4bf5-8f54-01e2c12da0a4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:45.643992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d8d8-e7bd-4bf5-8f54-01e2c12da0a4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:45.644278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d8d8-e7bd-4bf5-8f54-01e2c12da0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:45.668784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e2e3d8d8-e7bd-4bf5-8f54-01e2c12da0a4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:45.670245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e2e3d8d8-e7bd-4bf5-8f54-01e2c12da0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:45.682429 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3731/7505] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:45.687083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c9234-306d-4d32-be7d-a5acfd371dca None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:45.707084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c9234-306d-4d32-be7d-a5acfd371dca None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:45.714859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:45.716046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:45.716797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:45.717413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:45.718692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:27:45.718972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:45.721210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:45.736387 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3775/7506] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:27:45.741544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3d8d8-e7bd-4bf5-8f54-01e2c12da0a4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:45.754538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3d8d8-e7bd-4bf5-8f54-01e2c12da0a4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:45.761464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:45.762209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:45.762402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:45.762692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:45.763367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:45.765567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:45.813489 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3732/7507] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:27:45 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:45.819826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:45.833715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372592-2075-46e6-aa51-30a01c25c2a6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:45.870861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:45.931995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:45.938362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:45.938662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:45.938997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:45.940038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:45.942753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:45.992324 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3776/7508] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:27:45 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:27:45.993524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:46.011930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acefdab-b442-4d49-b308-46ca5055004b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:46.017551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:46.018506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:46.018972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:46.019249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:46.020012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:46.023808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:46.045116 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3733/7509] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:46.046779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:46.064453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642894ad-c6ba-4db5-9bcb-9b05ec495ae1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:46.076111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a358b8-b4b6-4390-aef7-e7e75c981fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:46.077640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a358b8-b4b6-4390-aef7-e7e75c981fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:46.078078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a358b8-b4b6-4390-aef7-e7e75c981fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:46.085596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a358b8-b4b6-4390-aef7-e7e75c981fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:46.086795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a358b8-b4b6-4390-aef7-e7e75c981fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:46.089152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a358b8-b4b6-4390-aef7-e7e75c981fb2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:46.101196 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3777/7510] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:27:46.106749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:46.130316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2ffee-2ac0-4fd5-908c-2d76cd51bf6b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:46.138378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:46.143544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:46.144286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:46.144721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] PATH_INFO: `/v3/projects/b1b9805b22374f82b3e46883f786d963/users/da3aef8b80bb411eb325010a4744f350/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:46.165393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b9805b22374f82b3e46883f786d963, user_id=da3aef8b80bb411eb325010a4744f350, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:46.168354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:46.196582 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3734/7511] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:27:46 2022] PUT /identity/v3/projects/b1b9805b22374f82b3e46883f786d963/users/da3aef8b80bb411eb325010a4744f350/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:46.200648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:46.200648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:46.200648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:46.236688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] MFA Rules not processed for user `da3aef8b80bb411eb325010a4744f350`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:27:46.238020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:46.329969 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3778/7512] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:27:46.340431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:46.342490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:46.433758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298de2f-09bc-4dee-aef6-178415238db6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:46.437650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:46.438267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:46.438588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:46.438932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:46.451277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:46.452401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:46.464306 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3735/7513] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:27:51.832107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:51.849607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:52.028196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48ed08-7528-495a-97b8-6852ed419974 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:52.045603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:52.046463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:52.046728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:52.047064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:52.049152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:52.101741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:52.144744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:52.197575 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3779/7514] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:27:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:27:56.024393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:56.025993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:56.083900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:56.098606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1ff35-c437-4dea-9e4d-741ac2430996 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:56.104286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:56.104910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:56.105193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:56.105470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] PATH_INFO: `/v3/users/55b65e5b614143af8fcc5155671e8602` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:56.106614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b65e5b614143af8fcc5155671e8602)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:56.108552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:56.158303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b65e5b614143af8fcc5155671e8602'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.158869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b65e5b614143af8fcc5155671e8602'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.166178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b65e5b614143af8fcc5155671e8602'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.174522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b65e5b614143af8fcc5155671e8602'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.187001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b65e5b614143af8fcc5155671e8602'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.192994 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3736/7515] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:56 2022] DELETE /identity/v3/users/55b65e5b614143af8fcc5155671e8602 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:56.378871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:56.390078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d85c4-f53d-451c-9c65-c9f23e7a7b5d service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:56.428043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:56.491311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:56.491311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:56.491311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:56.491311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] PATH_INFO: `/v3/projects/b4c28e76573e40269417e52300814190` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:56.491702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c28e76573e40269417e52300814190)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:56.491702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:56.552402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c28e76573e40269417e52300814190', 'request_id': 'req-598f1086-2c05-4407-905d-296807c5912c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.571592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c28e76573e40269417e52300814190', 'request_id': 'req-598f1086-2c05-4407-905d-296807c5912c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.578938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c28e76573e40269417e52300814190', 'request_id': 'req-598f1086-2c05-4407-905d-296807c5912c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.593256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] The token cache is being invalidate because project b4c28e76573e40269417e52300814190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:56.593539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.594217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.596036 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3780/7516] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:56 2022] DELETE /identity/v3/projects/b4c28e76573e40269417e52300814190 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:56.600465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:56.602447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:56.620426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7efc76-cf2c-4b97-b8bc-e3a80c58dce3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:56.699995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:56.700630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:56.701164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:56.701436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] PATH_INFO: `/v3/users/a0e47aca97434f09846b580091b06829` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:56.702824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e47aca97434f09846b580091b06829)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:56.715533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:56.783448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e47aca97434f09846b580091b06829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.783819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e47aca97434f09846b580091b06829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.792527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e47aca97434f09846b580091b06829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.802920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e47aca97434f09846b580091b06829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.814651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:56.825769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e47aca97434f09846b580091b06829'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:56.839699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3737/7517] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:27:56 2022] DELETE /identity/v3/users/a0e47aca97434f09846b580091b06829 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:27:57.055245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:27:57.057296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:27:57.075478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f1086-2c05-4407-905d-296807c5912c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:27:57.151146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:27:57.151795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:27:57.152076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:27:57.152354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] PATH_INFO: `/v3/projects/0a60557528994b829cd5131340a497b7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:27:57.153521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a60557528994b829cd5131340a497b7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:27:57.155447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:27:57.205198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a60557528994b829cd5131340a497b7', 'request_id': 'req-03ee1af1-e999-43c7-befa-909bf5cc3bdf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:57.214773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a60557528994b829cd5131340a497b7', 'request_id': 'req-03ee1af1-e999-43c7-befa-909bf5cc3bdf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:57.217037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:57.222086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a60557528994b829cd5131340a497b7', 'request_id': 'req-03ee1af1-e999-43c7-befa-909bf5cc3bdf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:57.235170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] The token cache is being invalidate because project 0a60557528994b829cd5131340a497b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:27:57.235454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:57.236102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-03ee1af1-e999-43c7-befa-909bf5cc3bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:27:57.237647 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3781/7518] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:27:57 2022] DELETE /identity/v3/projects/0a60557528994b829cd5131340a497b7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:02.064914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:02.066544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:02.082551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b240d2-eb56-4aa3-906b-c051ebd51fc3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:02.154507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:02.154782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:02.155101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:02.155379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] PATH_INFO: `/v3/users/da3aef8b80bb411eb325010a4744f350` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:02.156556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3aef8b80bb411eb325010a4744f350)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:02.158624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:02.214027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3aef8b80bb411eb325010a4744f350'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:02.214837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3aef8b80bb411eb325010a4744f350'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:02.223096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3aef8b80bb411eb325010a4744f350'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:02.231512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3aef8b80bb411eb325010a4744f350'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:02.245448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3aef8b80bb411eb325010a4744f350'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:02.252491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3738/7519] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:28:02 2022] DELETE /identity/v3/users/da3aef8b80bb411eb325010a4744f350 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:28:02.257506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:02.258008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:02.258264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:02.281507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:02.282892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:02.348668 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3782/7520] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:02.360130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:02.361583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:02.430294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7af43a-0986-4804-b44b-9a2fb0b70a97 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:02.434584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bb0a5-0a26-4e00-b4d3-044c4bd61494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:02.435548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bb0a5-0a26-4e00-b4d3-044c4bd61494 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:02.436345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bb0a5-0a26-4e00-b4d3-044c4bd61494 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:02.436783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bb0a5-0a26-4e00-b4d3-044c4bd61494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:02.447844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bb0a5-0a26-4e00-b4d3-044c4bd61494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:02.449568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bb0a5-0a26-4e00-b4d3-044c4bd61494 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:02.462222 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3739/7521] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:02.890000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:02.891602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:02.904263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230cc0-84fb-4515-9199-034941f8bc86 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:02.909775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:02.910944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:02.911232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:02.911648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] PATH_INFO: `/v3/projects/b1b9805b22374f82b3e46883f786d963` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:02.912923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b9805b22374f82b3e46883f786d963)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:02.915010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:02.972966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b9805b22374f82b3e46883f786d963', 'request_id': 'req-c76ef564-7f05-4677-8552-998880ac2d06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:02.982146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b9805b22374f82b3e46883f786d963', 'request_id': 'req-c76ef564-7f05-4677-8552-998880ac2d06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:02.988467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b9805b22374f82b3e46883f786d963', 'request_id': 'req-c76ef564-7f05-4677-8552-998880ac2d06'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:03.001866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] The token cache is being invalidate because project b1b9805b22374f82b3e46883f786d963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:28:03.002212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:03.002901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c76ef564-7f05-4677-8552-998880ac2d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:03.004562 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3783/7522] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:28:02 2022] DELETE /identity/v3/projects/b1b9805b22374f82b3e46883f786d963 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:03.011305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619399-b14e-4b07-b2b4-db7afb2b53c6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.011597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619399-b14e-4b07-b2b4-db7afb2b53c6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.011890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619399-b14e-4b07-b2b4-db7afb2b53c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.035466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0d619399-b14e-4b07-b2b4-db7afb2b53c6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:03.036719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d619399-b14e-4b07-b2b4-db7afb2b53c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:03.046005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:03.104125 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3740/7523] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:03.113248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436531-2ef9-42dc-b503-d3f93009ca16 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.113557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436531-2ef9-42dc-b503-d3f93009ca16 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.114008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436531-2ef9-42dc-b503-d3f93009ca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.138746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b1436531-2ef9-42dc-b503-d3f93009ca16 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:03.139997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b1436531-2ef9-42dc-b503-d3f93009ca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:03.150075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3784/7524] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:03.154838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619399-b14e-4b07-b2b4-db7afb2b53c6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.165060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619399-b14e-4b07-b2b4-db7afb2b53c6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.170155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.170776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.171054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.171328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.172017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:28:03.172322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.173977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.190120 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3741/7525] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:28:03.191226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1436531-2ef9-42dc-b503-d3f93009ca16 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.201137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1436531-2ef9-42dc-b503-d3f93009ca16 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.206197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.206839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.207085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.207356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.208002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.209993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.241937 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3785/7526] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:28:03 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:28:03.246367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.258213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ee7c0-bdb2-48a2-bd6f-10e17b33b5a0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.322245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.322902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.323180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.323459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.324177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.326129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.350856 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3742/7527] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:28:03 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:28:03.356118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.369378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a555c3b-cbd5-49a8-b460-b7f4070b4a8c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.374718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.375362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.375784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.376034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.376733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.378401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.389519 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3786/7528] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:28:03.394018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.405804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8f84-d581-4f37-b051-c426414e0570 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.411602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b50d8-9b57-437e-8e51-ca56c417fc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.412288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b50d8-9b57-437e-8e51-ca56c417fc8c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.412578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b50d8-9b57-437e-8e51-ca56c417fc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.412858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b50d8-9b57-437e-8e51-ca56c417fc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.413510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b50d8-9b57-437e-8e51-ca56c417fc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.415195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b50d8-9b57-437e-8e51-ca56c417fc8c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.427402 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3743/7529] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:28:03.432239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.443971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59422154-2131-459b-ae6c-4c23fc1b29fd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.449071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.449815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.450103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.450412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] PATH_INFO: `/v3/projects/86e71212d96f46a78680afaacd8709ca/users/0bd8a19d0ab44011b5430f26757a0efe/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.471887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e71212d96f46a78680afaacd8709ca, user_id=0bd8a19d0ab44011b5430f26757a0efe, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.474590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.491720 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3787/7530] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:28:03 2022] PUT /identity/v3/projects/86e71212d96f46a78680afaacd8709ca/users/0bd8a19d0ab44011b5430f26757a0efe/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:03.503645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e6a7b-1e66-40b8-886a-8a345c6fa1fc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.503835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e6a7b-1e66-40b8-886a-8a345c6fa1fc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.504187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e6a7b-1e66-40b8-886a-8a345c6fa1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.538050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b03e6a7b-1e66-40b8-886a-8a345c6fa1fc None None] MFA Rules not processed for user `0bd8a19d0ab44011b5430f26757a0efe`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:03.539501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b03e6a7b-1e66-40b8-886a-8a345c6fa1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:03.616819 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3744/7531] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:03.621497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.631297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:03.694535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3daa6-a132-4faa-9476-57c0e70957fe admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.700552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.701306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.701652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.701947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.702712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.704733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.737207 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3788/7532] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:28:03 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:28:03.742075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e6a7b-1e66-40b8-886a-8a345c6fa1fc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.757960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e6a7b-1e66-40b8-886a-8a345c6fa1fc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.765687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:03.830429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.831183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.831471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.831829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.832540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.834902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.862552 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3745/7533] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:28:03 2022] POST /identity/v3/users => generated 303 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:28:03.866885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.878605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6a22-ebd1-4d81-aba4-94cc79891b4e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.884323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.884994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.885292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.885549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.886196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.887814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.902854 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3789/7534] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:28:03.905221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.916582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fb232-504e-484f-b052-36cc35560e4f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:03.923465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:03.924239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:03.924519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:03.925349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] PATH_INFO: `/v3/projects/d4cd350726034f9985727755c395c181/users/5436a87ddec64a47bde28e63c0a3a368/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:03.941994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cd350726034f9985727755c395c181, user_id=5436a87ddec64a47bde28e63c0a3a368, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:03.944852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:03.961529 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3746/7535] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:28:03 2022] PUT /identity/v3/projects/d4cd350726034f9985727755c395c181/users/5436a87ddec64a47bde28e63c0a3a368/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:03.966113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:03.977903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2b72b-88c4-432b-9a16-a2a95a9960f1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:04.036584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1624af-add8-4c3e-928e-c7e1df262896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:04.037229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1624af-add8-4c3e-928e-c7e1df262896 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:04.037531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1624af-add8-4c3e-928e-c7e1df262896 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:04.037799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1624af-add8-4c3e-928e-c7e1df262896 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:04.038461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1624af-add8-4c3e-928e-c7e1df262896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:04.040050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1624af-add8-4c3e-928e-c7e1df262896 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:04.053742 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3790/7536] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:28:04.055199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:04.068992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b12c0-855e-4ffc-9ee1-6b07b4a14a48 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:04.070678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:04.072088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:04.072443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:04.072763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] PATH_INFO: `/v3/projects/d4cd350726034f9985727755c395c181/users/5436a87ddec64a47bde28e63c0a3a368/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:04.074888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cd350726034f9985727755c395c181, user_id=5436a87ddec64a47bde28e63c0a3a368, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:04.077888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:04.089126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3747/7537] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:28:04 2022] PUT /identity/v3/projects/d4cd350726034f9985727755c395c181/users/5436a87ddec64a47bde28e63c0a3a368/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:28:04.100049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:04.100309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:04.100651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:04.130053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] MFA Rules not processed for user `5436a87ddec64a47bde28e63c0a3a368`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:04.131344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:04.212082 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3791/7538] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:04.227073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:04.228907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:04.308216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a345d-86a7-4869-9067-3f71918af439 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:04.314655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d38dd-471f-4c99-803e-c622b6bdff2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:04.315294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d38dd-471f-4c99-803e-c622b6bdff2d service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:04.315534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d38dd-471f-4c99-803e-c622b6bdff2d service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:04.315800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d38dd-471f-4c99-803e-c622b6bdff2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:04.382556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d38dd-471f-4c99-803e-c622b6bdff2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:04.385399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d38dd-471f-4c99-803e-c622b6bdff2d service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:04.390856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:04.398485 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3748/7539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:07.720652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:07.723298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:07.739432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:07.800945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47a1e6-c59f-49ba-9e72-d9d98c4c9475 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:07.805055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bd072-2ebb-48a6-b51f-729b71c97307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:07.805278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bd072-2ebb-48a6-b51f-729b71c97307 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:07.805551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bd072-2ebb-48a6-b51f-729b71c97307 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:07.805838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bd072-2ebb-48a6-b51f-729b71c97307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:07.815319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bd072-2ebb-48a6-b51f-729b71c97307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:07.816797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bd072-2ebb-48a6-b51f-729b71c97307 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:07.828023 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3792/7540] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:11.401630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:11.402516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:11.402929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:11.426071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] MFA Rules not processed for user `0bd8a19d0ab44011b5430f26757a0efe`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:11.427903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:11.435510 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3749/7541] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:16.677799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:16.678283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:16.678393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:16.701131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:16.702551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:16.762952 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3793/7542] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:28:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:16.776525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:16.778600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:16.852788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0ea01-f9ce-4b96-ac6f-757ee644ab7d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:16.859621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:16.860274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:16.860550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:16.860850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:16.870367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:16.872085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:16.883216 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3750/7543] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:17.835143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:17.836675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:17.930281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424da61-ed91-4a5f-a049-e12f4cc93f16 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:17.935531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:17.936428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:17.936636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:17.936913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:17.938843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:17.953938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:17.955543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:17.968547 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3794/7544] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:28:22.089828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:22.091443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:22.110802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605103c-58b9-4662-91d3-dabf62bd381e service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:22.117835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:22.118460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:22.118749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:22.119025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:22.121057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:22.196696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:22.198295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:22.207911 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3751/7545] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:24.996684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:25.019109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc9095-2527-4d5e-85fd-592cd4af0302 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:25.027870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:25.028965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:25.029268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:25.029730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:25.032694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:25.052875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:25.054827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:25.077541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3795/7546] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:29.234317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:29.236027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:29.257450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6449-477d-443a-92f4-ada4f5fc3795 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:29.262541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:29.263440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:29.263797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:29.264162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:29.266438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:29.343937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:29.345960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:29.361707 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3752/7547] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:35.071554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:36.457625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:36.470652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa2a1c-bd42-4552-91a3-c92721e6a53f service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:36.476324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:36.477466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:36.477945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:36.478243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:36.480174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:36.498002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:36.499769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:36.511400 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3796/7548] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:36.934170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:36.935670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:36.946919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:36.958689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9af2-1803-4c0f-82da-64f5e1949556 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:36.969601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:36.970378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:36.970654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:36.970929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] PATH_INFO: `/v3/users/dc394c36e4bb436fabbecada1990b41e` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:36.972099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc394c36e4bb436fabbecada1990b41e)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:36.974229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:37.046880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc394c36e4bb436fabbecada1990b41e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.047568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc394c36e4bb436fabbecada1990b41e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.060307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc394c36e4bb436fabbecada1990b41e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.069643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc394c36e4bb436fabbecada1990b41e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.082598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc394c36e4bb436fabbecada1990b41e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.088575 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3753/7549] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:28:36 2022] DELETE /identity/v3/users/dc394c36e4bb436fabbecada1990b41e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:37.272818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:37.284580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ff826-6a23-420f-8285-ef46d7425a80 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:37.349428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:37.350113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:37.350540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:37.350823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] PATH_INFO: `/v3/projects/4195812e6f274bbba026e71b7fc4e231` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:37.352008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4195812e6f274bbba026e71b7fc4e231)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:37.354066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:37.404469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4195812e6f274bbba026e71b7fc4e231', 'request_id': 'req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.414740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4195812e6f274bbba026e71b7fc4e231', 'request_id': 'req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.421336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4195812e6f274bbba026e71b7fc4e231', 'request_id': 'req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.435407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] The token cache is being invalidate because project 4195812e6f274bbba026e71b7fc4e231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:28:37.435695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.436351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.438179 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3797/7550] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:28:37 2022] DELETE /identity/v3/projects/4195812e6f274bbba026e71b7fc4e231 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:37.442785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:37.444281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:37.458565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ff5b8-fe3f-49f6-84fe-91d384bbec82 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:37.521272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:37.521955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:37.522234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:37.522504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] PATH_INFO: `/v3/users/37715b55759e4187b56197b342a84947` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:37.523655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] RBAC: Authorizing `identity:delete_user(user_id=37715b55759e4187b56197b342a84947)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:37.525764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:37.594929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37715b55759e4187b56197b342a84947'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.596361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37715b55759e4187b56197b342a84947'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.620536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37715b55759e4187b56197b342a84947'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.632198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37715b55759e4187b56197b342a84947'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.646559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:37.649552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37715b55759e4187b56197b342a84947'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:37.657661 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3754/7551] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:28:37 2022] DELETE /identity/v3/users/37715b55759e4187b56197b342a84947 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:37.849734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:37.851472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:37.872736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d135e0-6c5f-41d6-b00f-a90162b23c8c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:37.942001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:37.942649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:37.942927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:37.943200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] PATH_INFO: `/v3/projects/29517ab16de8485a9044b547b7cf260c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:37.944324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29517ab16de8485a9044b547b7cf260c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:37.946915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:37.969520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:38.002025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29517ab16de8485a9044b547b7cf260c', 'request_id': 'req-a7c0b70d-2bbb-4789-ac6d-176ea8269309'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:38.011945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29517ab16de8485a9044b547b7cf260c', 'request_id': 'req-a7c0b70d-2bbb-4789-ac6d-176ea8269309'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:38.023229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29517ab16de8485a9044b547b7cf260c', 'request_id': 'req-a7c0b70d-2bbb-4789-ac6d-176ea8269309'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:38.036708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] The token cache is being invalidate because project 29517ab16de8485a9044b547b7cf260c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:28:38.036965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:38.037531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:38.039273 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3798/7552] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:28:37 2022] DELETE /identity/v3/projects/29517ab16de8485a9044b547b7cf260c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:28:40.312174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:40.313836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:40.328164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2251-eaa2-4fa4-83f8-473b5ce0569a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:40.391138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:40.391816 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:40.392099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:40.392375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] PATH_INFO: `/v3/users/97c769dbba794531b0a9db8e764ff097` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:40.393591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c769dbba794531b0a9db8e764ff097)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:40.395581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:40.444957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c769dbba794531b0a9db8e764ff097'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.445637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c769dbba794531b0a9db8e764ff097'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.452559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c769dbba794531b0a9db8e764ff097'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.463445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c769dbba794531b0a9db8e764ff097'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.476859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-147486ff-ce53-4799-a102-ccfb5b48cd93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c769dbba794531b0a9db8e764ff097'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.483305 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3755/7553] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:28:40 2022] DELETE /identity/v3/users/97c769dbba794531b0a9db8e764ff097 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:40.675810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:40.677422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:40.694224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b70d-2bbb-4789-ac6d-176ea8269309 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:40.755845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:40.756521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:40.756833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:40.757110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] PATH_INFO: `/v3/projects/10375bd836f84383b39f2df73f53800e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:40.758351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10375bd836f84383b39f2df73f53800e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:40.760291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:40.811519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10375bd836f84383b39f2df73f53800e', 'request_id': 'req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.821726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10375bd836f84383b39f2df73f53800e', 'request_id': 'req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.827437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10375bd836f84383b39f2df73f53800e', 'request_id': 'req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.839579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] The token cache is being invalidate because project 10375bd836f84383b39f2df73f53800e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:28:40.839865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.840518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ae25cee7-6abc-4128-9ce8-ca3dfdfab151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:28:40.842094 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3799/7554] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:28:40 2022] DELETE /identity/v3/projects/10375bd836f84383b39f2df73f53800e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:40.853546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acc05d-5174-46cf-8987-aadbcb1501db None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:40.853851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acc05d-5174-46cf-8987-aadbcb1501db None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:40.854123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acc05d-5174-46cf-8987-aadbcb1501db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:40.878384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-05acc05d-5174-46cf-8987-aadbcb1501db None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:40.879142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-05acc05d-5174-46cf-8987-aadbcb1501db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:40.943737 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3756/7555] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:40.952115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e536581-8557-4dcb-ab51-d96463e6b98f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:40.953012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e536581-8557-4dcb-ab51-d96463e6b98f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:40.953301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e536581-8557-4dcb-ab51-d96463e6b98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:40.975085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1e536581-8557-4dcb-ab51-d96463e6b98f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:40.976174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e536581-8557-4dcb-ab51-d96463e6b98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:40.984231 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3800/7556] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:40.988902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acc05d-5174-46cf-8987-aadbcb1501db None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:40.999028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acc05d-5174-46cf-8987-aadbcb1501db None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:41.004767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:41.005385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.005661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.005931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.006582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:28:41.006882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:41.008478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:41.017990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:41.021413 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3757/7557] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:28:41.025948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e536581-8557-4dcb-ab51-d96463e6b98f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:41.035561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e536581-8557-4dcb-ab51-d96463e6b98f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:41.040413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:41.041044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.041316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.041578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.042203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:41.044099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:41.074205 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3801/7558] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:28:41 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:28:41.078371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:41.089564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8f22-d880-477c-b03f-089ab5fb16c6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:41.150627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:41.151241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.151522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.151800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.152444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:41.154491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:41.177960 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3758/7559] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:28:41 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:28:41.182240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:41.193917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8535183-9bbb-4933-8aaf-1ce76fe6af4f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:41.199396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:41.200009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.200281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.200548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.201209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:41.202829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:41.205194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:41.215110 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3802/7560] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:28:41.219595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:41.230697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa536a4-f1e4-467a-93c0-300d15bbc6f9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:41.236164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6bac6-074a-4f09-9588-be265a6939d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:41.236813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6bac6-074a-4f09-9588-be265a6939d9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.237097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6bac6-074a-4f09-9588-be265a6939d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.237373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6bac6-074a-4f09-9588-be265a6939d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.238025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6bac6-074a-4f09-9588-be265a6939d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:41.239643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6bac6-074a-4f09-9588-be265a6939d9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:41.251477 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3759/7561] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:28:41.255919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:41.266788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfcf55-e618-41c2-a1d7-89d634511449 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:41.272788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:41.273464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.273821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.274345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] PATH_INFO: `/v3/projects/2c5bb6bd7e434b25992308dca2593154/users/91b11c3c34f8402390e4ee61b6c1bc8f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.293647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5bb6bd7e434b25992308dca2593154, user_id=91b11c3c34f8402390e4ee61b6c1bc8f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:41.296779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:41.312629 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3803/7562] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:28:41 2022] PUT /identity/v3/projects/2c5bb6bd7e434b25992308dca2593154/users/91b11c3c34f8402390e4ee61b6c1bc8f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:28:41.318123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4200b5c-5c72-4bf4-90c7-0009fbdff0fa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.318625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4200b5c-5c72-4bf4-90c7-0009fbdff0fa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.318778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4200b5c-5c72-4bf4-90c7-0009fbdff0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.338634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f4200b5c-5c72-4bf4-90c7-0009fbdff0fa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:41.340196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f4200b5c-5c72-4bf4-90c7-0009fbdff0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:41.403404 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3760/7563] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:41.413663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:41.415413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:41.477796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec142e8-0e4a-42b0-8992-5c4b5501b604 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:41.482260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:41.482885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:41.483164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:41.483462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:41.493885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:41.495330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:41.505897 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3804/7564] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:45.471815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd012a17-86f8-44d7-8746-33ca2b6ea36c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:45.472366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd012a17-86f8-44d7-8746-33ca2b6ea36c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:45.472581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd012a17-86f8-44d7-8746-33ca2b6ea36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:45.505562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bd012a17-86f8-44d7-8746-33ca2b6ea36c None None] MFA Rules not processed for user `91b11c3c34f8402390e4ee61b6c1bc8f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:45.507001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bd012a17-86f8-44d7-8746-33ca2b6ea36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:45.577535 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3761/7565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:45.590299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:28:45.591784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:45.679182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60478cc-4be4-42bc-8aa0-b46f46a0c0f1 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:28:45.685102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f818-931a-4c8c-b759-8fd8853b2936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:28:45.685713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f818-931a-4c8c-b759-8fd8853b2936 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:45.685997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f818-931a-4c8c-b759-8fd8853b2936 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:45.686377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f818-931a-4c8c-b759-8fd8853b2936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:45.696851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067f818-931a-4c8c-b759-8fd8853b2936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:28:45.698698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067f818-931a-4c8c-b759-8fd8853b2936 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:28:45.712966 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3805/7566] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:28:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:28:46.556569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c8e05-d19d-4638-9d06-219071eac408 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:46.557087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c8e05-d19d-4638-9d06-219071eac408 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:46.557196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c8e05-d19d-4638-9d06-219071eac408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:46.578243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a40c8e05-d19d-4638-9d06-219071eac408 None None] MFA Rules not processed for user `91b11c3c34f8402390e4ee61b6c1bc8f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:46.579602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a40c8e05-d19d-4638-9d06-219071eac408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:46.589540 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3762/7567] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:53.609097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc7bcc-3814-49c1-a9e9-a17f59bafa10 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:28:53.610052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc7bcc-3814-49c1-a9e9-a17f59bafa10 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:28:53.610514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc7bcc-3814-49c1-a9e9-a17f59bafa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:28:53.632874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-01cc7bcc-3814-49c1-a9e9-a17f59bafa10 None None] MFA Rules not processed for user `64090c8fd3394980ad5409adcde4986f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:28:53.634424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-01cc7bcc-3814-49c1-a9e9-a17f59bafa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:28:53.700924 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3806/7568] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:28:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:28:56.573176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:03.693672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:06.791440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c8e05-d19d-4638-9d06-219071eac408 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:06.809278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c8e05-d19d-4638-9d06-219071eac408 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:06.829600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:06.830608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:06.830965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:06.831241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] PATH_INFO: `/v3/users/91b11c3c34f8402390e4ee61b6c1bc8f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:06.833102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b11c3c34f8402390e4ee61b6c1bc8f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:06.835528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:06.923554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b11c3c34f8402390e4ee61b6c1bc8f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:06.925071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b11c3c34f8402390e4ee61b6c1bc8f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:06.934623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b11c3c34f8402390e4ee61b6c1bc8f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:06.948260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b11c3c34f8402390e4ee61b6c1bc8f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:06.967780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-655e1920-c50d-4dec-a94e-54f8f191a57b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b11c3c34f8402390e4ee61b6c1bc8f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:06.978364 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3763/7569] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:29:06 2022] DELETE /identity/v3/users/91b11c3c34f8402390e4ee61b6c1bc8f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:29:07.228069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc7bcc-3814-49c1-a9e9-a17f59bafa10 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:07.241226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc7bcc-3814-49c1-a9e9-a17f59bafa10 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:07.326575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:07.327566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.327857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.328137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] PATH_INFO: `/v3/projects/2c5bb6bd7e434b25992308dca2593154` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.329394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5bb6bd7e434b25992308dca2593154)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:07.331799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:07.399314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5bb6bd7e434b25992308dca2593154', 'request_id': 'req-05846e6c-bcd9-43c3-bc68-f8397aed983c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:07.411633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5bb6bd7e434b25992308dca2593154', 'request_id': 'req-05846e6c-bcd9-43c3-bc68-f8397aed983c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:07.421460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5bb6bd7e434b25992308dca2593154', 'request_id': 'req-05846e6c-bcd9-43c3-bc68-f8397aed983c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:07.440165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] The token cache is being invalidate because project 2c5bb6bd7e434b25992308dca2593154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:29:07.440464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:07.441244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-05846e6c-bcd9-43c3-bc68-f8397aed983c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:07.447561 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3807/7570] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:29:07 2022] DELETE /identity/v3/projects/2c5bb6bd7e434b25992308dca2593154 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:29:07.449412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263b24c-be76-480a-8b9c-f89d9ad2c038 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.449694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263b24c-be76-480a-8b9c-f89d9ad2c038 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.450382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263b24c-be76-480a-8b9c-f89d9ad2c038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.487331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7263b24c-be76-480a-8b9c-f89d9ad2c038 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:29:07.488618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7263b24c-be76-480a-8b9c-f89d9ad2c038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:07.566223 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3764/7571] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:29:07.576636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205ba6-665c-421e-a1a1-36c990b9ce01 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.577309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205ba6-665c-421e-a1a1-36c990b9ce01 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.577593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205ba6-665c-421e-a1a1-36c990b9ce01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.603911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-78205ba6-665c-421e-a1a1-36c990b9ce01 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:29:07.605825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78205ba6-665c-421e-a1a1-36c990b9ce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:07.615419 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3808/7572] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:29:07.620435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263b24c-be76-480a-8b9c-f89d9ad2c038 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:07.632212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263b24c-be76-480a-8b9c-f89d9ad2c038 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:07.639154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:07.639839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.640156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.640428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.641377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:29:07.641680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:07.643551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:07.663461 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3765/7573] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:29:07.665128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78205ba6-665c-421e-a1a1-36c990b9ce01 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:07.677219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78205ba6-665c-421e-a1a1-36c990b9ce01 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:07.682374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:07.683046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.683333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.683555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.684273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:07.686436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:07.718076 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3809/7574] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:29:07 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:29:07.722745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:07.733857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85d7f6-b055-4d83-9971-176b3a88fad8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:07.798488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:07.799171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.799661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.799661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.800334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:07.802342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:07.811062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:07.830110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3766/7575] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:29:07 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:29:07.834584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:07.847227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141f042-576c-40d7-95d5-74e9335c11df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:07.856174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:07.856848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.857179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.857448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.858092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:07.859731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:07.874079 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3810/7576] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:29:07.878013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:07.894240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa25a-6730-429c-a77d-3099621dee5e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:07.902976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4d39e-15b0-4c79-a1c9-3f17e672668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:07.903899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4d39e-15b0-4c79-a1c9-3f17e672668f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.904870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4d39e-15b0-4c79-a1c9-3f17e672668f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.905216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4d39e-15b0-4c79-a1c9-3f17e672668f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.906167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4d39e-15b0-4c79-a1c9-3f17e672668f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:07.909055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4d39e-15b0-4c79-a1c9-3f17e672668f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:07.928551 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3767/7577] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:29:07.933194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:07.945466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a038a-05a1-43f7-aa4a-9367bbd29fa7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:07.950158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:07.950836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:07.951105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:07.951389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] PATH_INFO: `/v3/projects/40247edd5a444fd8b1c937c13d39393a/users/6771df45b4ee4426b2f76a09eab2998a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:07.971091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40247edd5a444fd8b1c937c13d39393a, user_id=6771df45b4ee4426b2f76a09eab2998a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:07.973767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:07.999392 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3811/7578] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:29:07 2022] PUT /identity/v3/projects/40247edd5a444fd8b1c937c13d39393a/users/6771df45b4ee4426b2f76a09eab2998a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:29:08.000906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:08.001131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:08.002679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:08.033619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:29:08.034975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:08.116148 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3768/7579] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:29:08.127203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:08.129280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:08.163348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:08.198080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5388a2-a149-478a-8f9d-f93ff20d60b7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:08.202597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b77091-2510-4d45-9c2a-f38a43f307e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:08.204726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b77091-2510-4d45-9c2a-f38a43f307e6 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:08.204726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b77091-2510-4d45-9c2a-f38a43f307e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:08.204726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b77091-2510-4d45-9c2a-f38a43f307e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:08.216965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b77091-2510-4d45-9c2a-f38a43f307e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:08.218659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b77091-2510-4d45-9c2a-f38a43f307e6 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:08.230143 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3812/7580] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:29:12.697674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:12.700803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:12.826634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428d63b-4758-44e7-a63e-fb9cdb9fb8bf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:12.830577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a381e2-ebfe-48ac-a273-b03a7d88358d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:12.831273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a381e2-ebfe-48ac-a273-b03a7d88358d service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:12.831583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a381e2-ebfe-48ac-a273-b03a7d88358d service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:12.831907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a381e2-ebfe-48ac-a273-b03a7d88358d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:12.833726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d8a381e2-ebfe-48ac-a273-b03a7d88358d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:12.979232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a381e2-ebfe-48ac-a273-b03a7d88358d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:12.981084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a381e2-ebfe-48ac-a273-b03a7d88358d service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:13.002209 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3769/7581] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:13.012792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:13.013301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:13.013301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:13.075813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] MFA Rules not processed for user `6771df45b4ee4426b2f76a09eab2998a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:29:13.077650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:13.160622 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3813/7582] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:29:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:29:13.175370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbdb52-aaab-4e24-972d-e59a6cfd6e6c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:13.175970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbdb52-aaab-4e24-972d-e59a6cfd6e6c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:13.177485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbdb52-aaab-4e24-972d-e59a6cfd6e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:13.207331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-75bbdb52-aaab-4e24-972d-e59a6cfd6e6c None None] MFA Rules not processed for user `6771df45b4ee4426b2f76a09eab2998a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:29:13.208749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-75bbdb52-aaab-4e24-972d-e59a6cfd6e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:13.229016 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3770/7583] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:29:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:29:13.237493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:13.245646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:13.343860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57110a63-b591-490c-b483-d5afa99ce763 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:13.350084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:13.350713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:13.350980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:13.351271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:13.370632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:13.372515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:13.408013 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3814/7584] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:14.374669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:14.375535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:14.375813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:14.397597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] MFA Rules not processed for user `6771df45b4ee4426b2f76a09eab2998a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:29:14.398837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:14.410565 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3771/7585] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:29:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:29:19.046292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:19.048011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:19.067331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3f3a-ecc5-4ce3-8a53-a72fc0f8fc39 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:19.071215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:19.071822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:19.072084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:19.072531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:19.074628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:19.088786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:19.090498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:19.101327 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3815/7586] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:20.129810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:20.131328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:20.151893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca5de4-3e54-4b2b-84b0-9bfda21730da None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:20.156683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:20.157363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:20.157669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:20.158362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] PATH_INFO: `/v3/users/584c395757a54d9b9a64dd399d8557b1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:20.159618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=584c395757a54d9b9a64dd399d8557b1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:20.161638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:20.176270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:20.217522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '584c395757a54d9b9a64dd399d8557b1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.218196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '584c395757a54d9b9a64dd399d8557b1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.225345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '584c395757a54d9b9a64dd399d8557b1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.234332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '584c395757a54d9b9a64dd399d8557b1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.249103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '584c395757a54d9b9a64dd399d8557b1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.254804 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3772/7587] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:29:20 2022] DELETE /identity/v3/users/584c395757a54d9b9a64dd399d8557b1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:29:20.451039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:20.463424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ed0be-1d7d-4ae8-8892-1a777afed44e service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:20.500248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:20.534448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:20.535087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:20.535357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:20.535625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] PATH_INFO: `/v3/projects/712c1c4c858b4b788a7703345716504b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:20.536895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] RBAC: Authorizing `identity:delete_project(project_id=712c1c4c858b4b788a7703345716504b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:20.538802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:20.592023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712c1c4c858b4b788a7703345716504b', 'request_id': 'req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.600178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712c1c4c858b4b788a7703345716504b', 'request_id': 'req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.612478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712c1c4c858b4b788a7703345716504b', 'request_id': 'req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.625815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] The token cache is being invalidate because project 712c1c4c858b4b788a7703345716504b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:29:20.626126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.628146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6d4545bf-b8dc-4c58-8ef7-8f0590d2057c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:29:20.629867 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3816/7588] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:29:20 2022] DELETE /identity/v3/projects/712c1c4c858b4b788a7703345716504b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:29:25.970226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:25.971758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:26.055362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd6c78-b4e9-4156-abd0-a57e49ea0ac5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:26.122714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:26.123329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:26.123613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:26.123888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:26.125709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:26.153346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:26.189603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:26.191433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:26.201006 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3773/7589] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:30.622522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:31.112580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:31.113411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:31.113691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:31.134261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] MFA Rules not processed for user `178956f6c8154a06a0710b295aff3086`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:29:31.135469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:31.143613 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3817/7590] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:29:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:29:31.488423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:31.490289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:31.552690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748716-6d1e-4006-9eff-ffaa037d0fec service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:31.556786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:31.557035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:31.557312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:31.557583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:31.559360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:31.570973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:31.572448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:31.582243 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3774/7591] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:35.121571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:35.123231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:35.199630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7a098-9aef-4271-8c0e-7ff795330bfd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:35.204932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:35.205549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:35.205829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:35.206103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:35.208298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:35.223792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:35.225605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:35.239510 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3818/7592] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:41.577043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=41 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:42.154606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:42.156107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:42.169153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbfe50-1073-4ab5-9883-32d417e8db5f service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:42.173748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:42.174356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:42.174638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:42.174910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:42.176869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:42.188601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:42.190097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:42.200488 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3775/7593] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:42.236625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:42.250096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ac49b-658c-4d04-8bfc-1aa866c47583 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:42.256762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ecdcb-bfa9-4e87-81c1-85b05149729b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:42.257368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ecdcb-bfa9-4e87-81c1-85b05149729b service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:42.257644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ecdcb-bfa9-4e87-81c1-85b05149729b service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:42.257910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ecdcb-bfa9-4e87-81c1-85b05149729b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:42.260105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-899ecdcb-bfa9-4e87-81c1-85b05149729b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:42.275225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ecdcb-bfa9-4e87-81c1-85b05149729b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:42.276864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ecdcb-bfa9-4e87-81c1-85b05149729b service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:42.290149 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3819/7594] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:42.528960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:29:42.530415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:42.547354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1b284-46c3-423c-9619-372266574bca service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:29:42.552622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:29:42.553274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:29:42.553574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:29:42.553852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:29:42.555642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:29:42.570331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:29:42.571958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:29:42.584834 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3776/7595] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:29:52.282662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=41 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:52.578178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:29.993566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd58c35-c1cd-489a-80d8-a4bec44f0de3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:30:29.994182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd58c35-c1cd-489a-80d8-a4bec44f0de3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:30:29.994182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd58c35-c1cd-489a-80d8-a4bec44f0de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:30:30.017514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6fd58c35-c1cd-489a-80d8-a4bec44f0de3 None None] MFA Rules not processed for user `6771df45b4ee4426b2f76a09eab2998a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:30:30.018782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6fd58c35-c1cd-489a-80d8-a4bec44f0de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:30:30.082047 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3820/7596] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:30:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:30:34.359043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:30:34.377682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a5a1d-d0a0-42e0-afe5-92ec0aa46881 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:30:34.398198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df20a5-a030-49b9-acdb-368894b4af2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:30:34.399123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df20a5-a030-49b9-acdb-368894b4af2e service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:30:34.399593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df20a5-a030-49b9-acdb-368894b4af2e service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:30:34.400056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df20a5-a030-49b9-acdb-368894b4af2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:30:34.406120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-09df20a5-a030-49b9-acdb-368894b4af2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:30:34.433719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df20a5-a030-49b9-acdb-368894b4af2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:30:34.436157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df20a5-a030-49b9-acdb-368894b4af2e service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:30:34.452273 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3777/7597] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:30:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:30:39.056496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59560a-0c22-4d98-8e5f-c8a7f59e4f37 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:30:39.057086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59560a-0c22-4d98-8e5f-c8a7f59e4f37 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:30:39.057210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59560a-0c22-4d98-8e5f-c8a7f59e4f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:30:39.079757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5a59560a-0c22-4d98-8e5f-c8a7f59e4f37 None None] MFA Rules not processed for user `64090c8fd3394980ad5409adcde4986f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:30:39.082007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a59560a-0c22-4d98-8e5f-c8a7f59e4f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:30:39.154795 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3821/7598] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:30:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:30:44.443594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:49.147301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=50 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:52.208609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04aeb9-6f2c-401c-be41-79edb5e53431 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:30:52.209200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04aeb9-6f2c-401c-be41-79edb5e53431 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:30:52.209200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04aeb9-6f2c-401c-be41-79edb5e53431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:30:52.245400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3c04aeb9-6f2c-401c-be41-79edb5e53431 None None] MFA Rules not processed for user `0bd8a19d0ab44011b5430f26757a0efe`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:30:52.246616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3c04aeb9-6f2c-401c-be41-79edb5e53431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:30:52.356865 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3778/7599] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:30:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:31:02.354792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=37 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:30.261354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583e2cb-bdee-447e-a1e1-5c0937a654d9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:31:30.261958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583e2cb-bdee-447e-a1e1-5c0937a654d9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:31:30.261958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583e2cb-bdee-447e-a1e1-5c0937a654d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:31:30.282338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0583e2cb-bdee-447e-a1e1-5c0937a654d9 None None] MFA Rules not processed for user `178956f6c8154a06a0710b295aff3086`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:31:30.283571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0583e2cb-bdee-447e-a1e1-5c0937a654d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:31:30.292194 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3822/7600] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:31:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:31:37.445712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04aeb9-6f2c-401c-be41-79edb5e53431 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:31:37.485068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04aeb9-6f2c-401c-be41-79edb5e53431 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:31:37.491467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b77b-5bd2-483d-8755-58a89e3bf9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:31:37.492373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b77b-5bd2-483d-8755-58a89e3bf9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:31:37.492722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b77b-5bd2-483d-8755-58a89e3bf9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:31:37.493128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b77b-5bd2-483d-8755-58a89e3bf9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:31:37.495849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e98b77b-5bd2-483d-8755-58a89e3bf9f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:31:37.520112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b77b-5bd2-483d-8755-58a89e3bf9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:31:37.521580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b77b-5bd2-483d-8755-58a89e3bf9f4 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:31:37.546194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3779/7601] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:31:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:31:40.275749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:43.127492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583e2cb-bdee-447e-a1e1-5c0937a654d9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:31:43.138201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583e2cb-bdee-447e-a1e1-5c0937a654d9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:31:43.141838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:31:43.142446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:31:43.142722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:31:43.142991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:31:43.144814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:31:43.161466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:31:43.163347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:31:43.177546 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3823/7602] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:31:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:31:43.271094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0776d-d7dc-430e-a87f-5ac9cbff0cb3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:31:43.271659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0776d-d7dc-430e-a87f-5ac9cbff0cb3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:31:43.271659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0776d-d7dc-430e-a87f-5ac9cbff0cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:31:43.293582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c6f0776d-d7dc-430e-a87f-5ac9cbff0cb3 None None] MFA Rules not processed for user `64090c8fd3394980ad5409adcde4986f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:31:43.294806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c6f0776d-d7dc-430e-a87f-5ac9cbff0cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:31:43.302176 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3780/7603] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:31:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:31:53.170651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:53.287843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=26 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:24.015358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:32:24.026485 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa140f1c-6946-4c2e-936a-5efafd7ddffe service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:32:24.031422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:32:24.032047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:24.032328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:24.032605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:24.034823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:24.062017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:32:24.063571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:32:24.074245 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3824/7604] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:32:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:32:24.369099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0776d-d7dc-430e-a87f-5ac9cbff0cb3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:32:24.382034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0776d-d7dc-430e-a87f-5ac9cbff0cb3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:32:24.386143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:32:24.386863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:24.387146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:24.387426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:24.389433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:24.405717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:32:24.407188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:32:24.420900 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3781/7605] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:32:24.480111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:32:24.505232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849f672-2616-4f69-a0a1-45ed478dffd9 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:32:24.510420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00192036-2f2b-4b56-bc83-bdec13643073 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:32:24.511028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00192036-2f2b-4b56-bc83-bdec13643073 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:24.511319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00192036-2f2b-4b56-bc83-bdec13643073 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:24.511570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00192036-2f2b-4b56-bc83-bdec13643073 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:24.513334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-00192036-2f2b-4b56-bc83-bdec13643073 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:24.527457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00192036-2f2b-4b56-bc83-bdec13643073 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:32:24.529386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00192036-2f2b-4b56-bc83-bdec13643073 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:32:24.541802 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3825/7606] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:32:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:32:26.484319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:32:26.495752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760d08d-6b71-4593-8437-e41c1b1e18de service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:32:26.501204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:32:26.501822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:26.502104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:26.502380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:26.504401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:26.517793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:32:26.519266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:32:26.530162 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3782/7607] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:32:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:32:28.188990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:28.189527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:28.189527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:28.209334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] MFA Rules not processed for user `6771df45b4ee4426b2f76a09eab2998a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:32:28.210564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:28.219165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3826/7608] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:32:36.524245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:38.203652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=24 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:38.494472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:32:38.505786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdd19b-1c9e-4912-b89f-10257ece1d12 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:32:38.510323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:32:38.510967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:38.511253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:38.511524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:38.513645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:38.526385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:32:38.527866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:32:38.538473 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3783/7609] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:32:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:32:48.532621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:51.595401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:32:51.597300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:51.625334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e341dd-39aa-47f8-aa91-122eb111ec71 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:32:51.631823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:32:51.632440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:51.632946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:51.633903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:51.636021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:51.653988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:32:51.656352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:32:51.674657 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3827/7610] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:32:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:32:52.479348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:32:52.480986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:52.499934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5f698-fa3a-4493-852d-0db9df23b6f7 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:32:52.506048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:32:52.506687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:32:52.506966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:32:52.507235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:32:52.509258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:32:52.524777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:32:52.526666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:32:52.539777 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3784/7611] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:32:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:01.664146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:02.533299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:13.531885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:13.547234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7dc1a-2c7e-4283-9225-301bd0d86c05 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:13.555003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:13.555686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:13.555955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:13.556218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:13.558458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:13.569982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:13.571607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:13.582155 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3828/7612] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:23.576979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:27.964510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:27.979338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7395cac-f963-4121-925f-927ad9a2475d service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:27.986854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:27.987517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:27.987814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:27.988077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:27.990061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:28.006359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:28.008014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:28.022022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3785/7613] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:29.705685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:29.721264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1416d3e-5ebc-4e6c-a846-8e69aad8e919 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:29.726459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:29.727408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:29.727781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:29.728366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:29.730622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:29.869626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:29.871319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:29.894327 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3829/7614] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:33.337243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:33.352926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2081b11-29c9-46f2-8a5b-26da86a63910 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:33.358296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:33.359008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:33.359381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:33.359756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:33.361600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:33.375582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:33.377197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:33.390153 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3786/7615] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:33.403292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:33.404869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:33.417533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1076-8a36-40d4-95b6-53b2406023f9 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:33.425905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:33.426570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:33.426855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:33.427136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] PATH_INFO: `/v3/users/64090c8fd3394980ad5409adcde4986f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:33.428817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64090c8fd3394980ad5409adcde4986f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:33.430913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:33.482781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64090c8fd3394980ad5409adcde4986f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.483448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64090c8fd3394980ad5409adcde4986f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.492784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64090c8fd3394980ad5409adcde4986f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.500277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64090c8fd3394980ad5409adcde4986f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.505555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-492d4dda-952c-436a-82c5-46450867aa35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64090c8fd3394980ad5409adcde4986f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.523031 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3830/7616] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:33 2022] DELETE /identity/v3/users/64090c8fd3394980ad5409adcde4986f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:33.732479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:33.747387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83feb5-35d8-46a8-8fa1-b4014df83917 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:33.820362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:33.821401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:33.821831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:33.822287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] PATH_INFO: `/v3/projects/90c0fbb4a6d84767b920564584715f9d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:33.823619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c0fbb4a6d84767b920564584715f9d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:33.826014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:33.885245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c0fbb4a6d84767b920564584715f9d', 'request_id': 'req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.892899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c0fbb4a6d84767b920564584715f9d', 'request_id': 'req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.902569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c0fbb4a6d84767b920564584715f9d', 'request_id': 'req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.913876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] The token cache is being invalidate because project 90c0fbb4a6d84767b920564584715f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:33.914222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.914874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fcb47469-7c67-45ea-9fbb-a1abd5bfe6b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:33.916430 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3787/7617] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:33 2022] DELETE /identity/v3/projects/90c0fbb4a6d84767b920564584715f9d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:33.921939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d77ffd-579a-437a-9042-2c646afb3916 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:33.922251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d77ffd-579a-437a-9042-2c646afb3916 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:33.922524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d77ffd-579a-437a-9042-2c646afb3916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:33.985511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:33.987299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-09d77ffd-579a-437a-9042-2c646afb3916 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:33.988982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-09d77ffd-579a-437a-9042-2c646afb3916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:34.056264 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3831/7618] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:34.064774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.065146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.065432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.086316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:34.087529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:34.095939 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3788/7619] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:34.100329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d77ffd-579a-437a-9042-2c646afb3916 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.110861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d77ffd-579a-437a-9042-2c646afb3916 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.119372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.120035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.120337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.120638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.121375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:34.121673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.123602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.125789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.127718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:34.140100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3832/7620] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:34.141188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.151640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73ad23-0a2d-4d75-a37b-e540ef4aace1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.156513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.157162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.157433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.157737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.158394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.160513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.198200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:34.204362 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3833/7621] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:34 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:34.205559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.226909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bb497-042a-442e-9eaa-db114d68164d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.284350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428de4cb-4c9b-4dde-9a7a-10a8b811815f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.291022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.292570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.292901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.293173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.293937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.297164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.308871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.309997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.311116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.311116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.312959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:34.324386 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3834/7622] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:34 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:34.328883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.341030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb822a-8b67-447d-be86-153cd483ec71 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.346764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.347525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.348174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.348526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.350958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.353599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.369791 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3835/7623] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:34.371608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.382889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58338019-9a97-4b07-8319-c06aac812c6e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.386357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.388076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.389904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865938bb-2e69-49d6-b6b5-cebd678086eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.390608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865938bb-2e69-49d6-b6b5-cebd678086eb admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.390946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865938bb-2e69-49d6-b6b5-cebd678086eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.391284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865938bb-2e69-49d6-b6b5-cebd678086eb admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.391994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865938bb-2e69-49d6-b6b5-cebd678086eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.394325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865938bb-2e69-49d6-b6b5-cebd678086eb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.399574 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3789/7624] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:34.406800 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3836/7625] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:34.411321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.424783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a0704-e337-401b-b520-f4891dbbc10f service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.429678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.430448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.430800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.431312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] PATH_INFO: `/v3/projects/78c18800a5fc4f40994fade5d422694d/users/f827322737bf4518b0b262ae61797687/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.449530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c18800a5fc4f40994fade5d422694d, user_id=f827322737bf4518b0b262ae61797687, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.452748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.469840 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3790/7626] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:34 2022] PUT /identity/v3/projects/78c18800a5fc4f40994fade5d422694d/users/f827322737bf4518b0b262ae61797687/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:34.481661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59935-9cc7-4219-943f-e54b4109e0c7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.482106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59935-9cc7-4219-943f-e54b4109e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.482408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59935-9cc7-4219-943f-e54b4109e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.513783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f7e59935-9cc7-4219-943f-e54b4109e0c7 None None] MFA Rules not processed for user `f827322737bf4518b0b262ae61797687`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:34.515027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f7e59935-9cc7-4219-943f-e54b4109e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:34.534692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:34.583253 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3837/7627] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:34.597802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:34.611563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:34.700849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc2a0a-25fc-404b-aa74-986f01e7d233 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:34.706956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134df9b-81d6-4fe6-9461-866f42b41ae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:34.707699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134df9b-81d6-4fe6-9461-866f42b41ae6 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:34.708028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134df9b-81d6-4fe6-9461-866f42b41ae6 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:34.708360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134df9b-81d6-4fe6-9461-866f42b41ae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:34.714014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:34.721093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134df9b-81d6-4fe6-9461-866f42b41ae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:34.722737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134df9b-81d6-4fe6-9461-866f42b41ae6 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:34.733982 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3791/7628] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:41.196929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e59935-9cc7-4219-943f-e54b4109e0c7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:41.268526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e59935-9cc7-4219-943f-e54b4109e0c7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:41.273086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:41.273718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:41.274048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:41.274266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] PATH_INFO: `/v3/users/f827322737bf4518b0b262ae61797687` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:41.275419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f827322737bf4518b0b262ae61797687)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:41.277994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:41.327728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f827322737bf4518b0b262ae61797687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:41.328393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f827322737bf4518b0b262ae61797687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:41.337648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f827322737bf4518b0b262ae61797687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:41.348577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f827322737bf4518b0b262ae61797687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:41.349339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f827322737bf4518b0b262ae61797687'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:41.368169 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3838/7629] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:41 2022] DELETE /identity/v3/users/f827322737bf4518b0b262ae61797687 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:41.373164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:41.373693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:41.373942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:41.394181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:41.395398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:41.460020 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3792/7630] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:41.470989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:41.472452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:41.517429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:41.537097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca6245-1d30-4c39-a5c9-c70b2c48c999 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:41.541093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3674fbf-e8e4-47df-a831-45ab208f6789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:41.541823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674fbf-e8e4-47df-a831-45ab208f6789 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:41.542106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674fbf-e8e4-47df-a831-45ab208f6789 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:41.542373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674fbf-e8e4-47df-a831-45ab208f6789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:41.552103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3674fbf-e8e4-47df-a831-45ab208f6789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:41.553679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3674fbf-e8e4-47df-a831-45ab208f6789 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:41.564037 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3839/7631] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:41.863204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:41.866165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:41.902885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92a10b-9735-4e00-b0be-d4973d1a288a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:41.908679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:41.909784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:41.910125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:41.913214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] PATH_INFO: `/v3/projects/78c18800a5fc4f40994fade5d422694d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:41.914810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78c18800a5fc4f40994fade5d422694d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:41.927770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.000090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78c18800a5fc4f40994fade5d422694d', 'request_id': 'req-747c7822-808a-4afa-bf17-f02d8c979152'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:42.006980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78c18800a5fc4f40994fade5d422694d', 'request_id': 'req-747c7822-808a-4afa-bf17-f02d8c979152'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:42.016606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78c18800a5fc4f40994fade5d422694d', 'request_id': 'req-747c7822-808a-4afa-bf17-f02d8c979152'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:42.027575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] The token cache is being invalidate because project 78c18800a5fc4f40994fade5d422694d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:42.027869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:42.028507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-747c7822-808a-4afa-bf17-f02d8c979152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:42.030070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3793/7632] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:41 2022] DELETE /identity/v3/projects/78c18800a5fc4f40994fade5d422694d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:42.035006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9987-4ffe-48ca-9469-166842880633 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.035295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9987-4ffe-48ca-9469-166842880633 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.035572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9987-4ffe-48ca-9469-166842880633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.058736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-959e9987-4ffe-48ca-9469-166842880633 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:42.060154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-959e9987-4ffe-48ca-9469-166842880633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:42.123861 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3840/7633] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:42.132532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291a113-8c69-4b51-b51f-1a19d1a525ba None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.132854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291a113-8c69-4b51-b51f-1a19d1a525ba None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.133138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291a113-8c69-4b51-b51f-1a19d1a525ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.156250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e291a113-8c69-4b51-b51f-1a19d1a525ba None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:42.157676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e291a113-8c69-4b51-b51f-1a19d1a525ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:42.166293 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3794/7634] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:42.171087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9987-4ffe-48ca-9469-166842880633 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.181570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9987-4ffe-48ca-9469-166842880633 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.186643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.187270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.187734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.188011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.188712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:42.189029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.190746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.204127 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3841/7635] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:42.208756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291a113-8c69-4b51-b51f-1a19d1a525ba None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.220145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291a113-8c69-4b51-b51f-1a19d1a525ba None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.225958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.226736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.227129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.227513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.228229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.230781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.264411 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3795/7636] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:42 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:42.265757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.276229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9392-7fe0-432f-93a8-c5cbb68b5d38 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.335303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.335919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.336195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.336464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.337139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.340012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.362368 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3842/7637] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:42 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:42.366611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.377996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ebf31-816a-49ab-a4b5-dd22898d4393 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.383830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.384510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.384982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.385263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.386025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.387772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.398805 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3796/7638] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:42.403714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.413985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f5fa6-3eaa-448f-b88b-8de7bc117cb6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.419297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c82b7d-6faa-47ce-b877-18a6886d44c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.419908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c82b7d-6faa-47ce-b877-18a6886d44c6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.420202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c82b7d-6faa-47ce-b877-18a6886d44c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.420471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c82b7d-6faa-47ce-b877-18a6886d44c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.421221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c82b7d-6faa-47ce-b877-18a6886d44c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.422811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c82b7d-6faa-47ce-b877-18a6886d44c6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.435884 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3843/7639] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:42.441044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.452565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad28396-59d2-4ddd-85a9-8ed8374c92a1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.457602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.458271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.458556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.458862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] PATH_INFO: `/v3/projects/8fcab8074e364277b02ba683cd28ec85/users/1712e1d632ac4a6bb0fefe6493cd022b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.474807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcab8074e364277b02ba683cd28ec85, user_id=1712e1d632ac4a6bb0fefe6493cd022b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.477540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.492680 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3797/7640] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:42 2022] PUT /identity/v3/projects/8fcab8074e364277b02ba683cd28ec85/users/1712e1d632ac4a6bb0fefe6493cd022b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:42.503577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838352c5-6143-4591-9765-8515af98516a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.503861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838352c5-6143-4591-9765-8515af98516a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.504156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838352c5-6143-4591-9765-8515af98516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.536997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-838352c5-6143-4591-9765-8515af98516a None None] MFA Rules not processed for user `1712e1d632ac4a6bb0fefe6493cd022b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:42.538207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-838352c5-6143-4591-9765-8515af98516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:42.578236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:42.608041 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3844/7641] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:42.612689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.628082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:42.680196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855679e-060b-4578-8492-20492b4c6e22 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.685686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.686298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.686576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.686823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.687511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.689489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.719322 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3798/7642] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:42 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:42.723752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838352c5-6143-4591-9765-8515af98516a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.735507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838352c5-6143-4591-9765-8515af98516a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.794077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.794663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.794968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.795244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.795880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.797859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.821451 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3845/7643] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:42 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:42.825838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.836883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709013ba-77a4-4d03-b78f-146a7334ffa2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.842417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.843049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.843332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.843607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.844256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.845917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.857045 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3799/7644] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:42.861805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.872450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8f4e-efca-4b63-b327-6256a882df74 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.877340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.877988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.878267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.878536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] PATH_INFO: `/v3/projects/e6def473a7ee48eca824efb6c1bebee3/users/ff9f80e09be94ade9971d22ad661b81c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.894019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6def473a7ee48eca824efb6c1bebee3, user_id=ff9f80e09be94ade9971d22ad661b81c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:42.896728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:42.911636 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3846/7645] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:42 2022] PUT /identity/v3/projects/e6def473a7ee48eca824efb6c1bebee3/users/ff9f80e09be94ade9971d22ad661b81c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:42.916072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:42.926739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0888a90-1a5e-4384-88ea-2daad20d1fc4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:42.990803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b9841-3c04-4724-8a52-df459a4383e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:42.992126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b9841-3c04-4724-8a52-df459a4383e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:42.992726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b9841-3c04-4724-8a52-df459a4383e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:42.993174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b9841-3c04-4724-8a52-df459a4383e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:42.993985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b9841-3c04-4724-8a52-df459a4383e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.001246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b9841-3c04-4724-8a52-df459a4383e4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.012453 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3800/7646] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:43.013354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.025071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d65f22-c0d7-4c17-9c9c-f633791377c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.029630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.030357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.030637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.030904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] PATH_INFO: `/v3/projects/e6def473a7ee48eca824efb6c1bebee3/users/ff9f80e09be94ade9971d22ad661b81c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.033119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6def473a7ee48eca824efb6c1bebee3, user_id=ff9f80e09be94ade9971d22ad661b81c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.035675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.048148 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3847/7647] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:43 2022] PUT /identity/v3/projects/e6def473a7ee48eca824efb6c1bebee3/users/ff9f80e09be94ade9971d22ad661b81c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:43.060443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f9df-e7a0-4107-87e1-9021242c12ee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.060916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f9df-e7a0-4107-87e1-9021242c12ee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.061229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f9df-e7a0-4107-87e1-9021242c12ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.093051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2684f9df-e7a0-4107-87e1-9021242c12ee None None] MFA Rules not processed for user `ff9f80e09be94ade9971d22ad661b81c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:43.094248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2684f9df-e7a0-4107-87e1-9021242c12ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:43.167467 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3801/7648] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:43.172082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.183121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afba154-9ffa-4dc4-850a-0787e537c166 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.187670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.188267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.188540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.188846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.189515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.191106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.215886 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3848/7649] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:33:43 2022] POST /identity/v3/domains => generated 276 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:43.217283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684f9df-e7a0-4107-87e1-9021242c12ee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.228223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684f9df-e7a0-4107-87e1-9021242c12ee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.234027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:43.311005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.311633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.311949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.312218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.312902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.314968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.348453 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3802/7650] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:33:43 2022] POST /identity/v3/groups => generated 279 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:43.352505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.364034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b117977-a1fa-45ef-ac8e-1ca5d029fdc2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.368784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.369449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.369777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.370053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/a576b8f135974c44997d944bee94c15b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.375635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:update_group(group_id=a576b8f135974c44997d944bee94c15b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.376706 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:33:43.376706 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:33:43.376706 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:33:43.377921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.398001 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3849/7651] 10.222.0.26 () {60 vars in 1247 bytes} [Tue Sep 27 10:33:43 2022] PATCH /identity/v3/groups/a576b8f135974c44997d944bee94c15b => generated 292 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:33:43.402168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.413767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48509ac5-718c-4500-9641-3a01b7a512e2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.419297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.419964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.420250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.420527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/a576b8f135974c44997d944bee94c15b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.426496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:get_group(group_id=a576b8f135974c44997d944bee94c15b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.428786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.430568 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3803/7652] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:43 2022] GET /identity/v3/groups/a576b8f135974c44997d944bee94c15b => generated 292 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:43.434728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.445134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31850df-040b-4c3b-85df-489c7555e243 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.449842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.450539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.450827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.451103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/a576b8f135974c44997d944bee94c15b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.452283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:update_group(group_id=a576b8f135974c44997d944bee94c15b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.454221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.473351 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3850/7653] 10.222.0.26 () {60 vars in 1246 bytes} [Tue Sep 27 10:33:43 2022] PATCH /identity/v3/groups/a576b8f135974c44997d944bee94c15b => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:43.477612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.488067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc575-2b85-4bbb-aa0b-f6e96a68b2fb tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.493243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.493866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.494129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.494400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/a576b8f135974c44997d944bee94c15b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.499666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_group(group_id=a576b8f135974c44997d944bee94c15b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.501648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.550079 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3804/7654] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:43 2022] DELETE /identity/v3/groups/a576b8f135974c44997d944bee94c15b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:43.555225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.565607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014aca-a08f-4d24-a3a2-96eb873ea2c4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.606586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:UPDATE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:43.625218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.625861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.626141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.626416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.627117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.629066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.646343 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3851/7655] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:33:43 2022] POST /identity/v3/groups => generated 327 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:43.650996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.662259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e789be-ea37-4a1d-832f-f2cf8bc92751 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.667211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.667823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.668110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.668392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.669594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.670984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.702556 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3805/7656] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:43 2022] POST /identity/v3/users => generated 323 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:43.707015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.718055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6425d-3037-46a3-97f9-37e293724b01 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.723956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.724595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.724931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.725209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/3f9399daa84c4cef8d6471044ca941cd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.745885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=3f9399daa84c4cef8d6471044ca941cd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.747810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.774109 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3852/7657] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:33:43 2022] PUT /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/3f9399daa84c4cef8d6471044ca941cd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:43.778820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.789961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047625b8-fcd0-45ee-9cef-d1fc55e98ce8 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.852259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.852905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.853187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.853458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.854211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.856233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.884510 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3806/7658] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:43 2022] POST /identity/v3/users => generated 323 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:43.888766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.899506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622c2aa-5341-471c-8383-07e7a39f27d2 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:43.905068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:43.905713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:43.905992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:43.906288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/30785747a9484a6892feee8691f60c00` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:43.921799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=30785747a9484a6892feee8691f60c00)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:43.923656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:43.947404 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3853/7659] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:33:43 2022] PUT /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/30785747a9484a6892feee8691f60c00 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:43.951830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:43.958750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:INSERT=7,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:43.963665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f921c-6da4-4de7-8ca1-32065eac4b7e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.024895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.025495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.025783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.026066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.026674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.028705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.064979 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3807/7660] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:43 2022] POST /identity/v3/users => generated 323 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:44.067767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.081903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aee2b-c90e-402f-9e98-bbeb4382d2d6 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.087895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.088705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.089144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.089763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/70e3e6cc3e7542bd8be3fa6be1c50d72` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.107986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=70e3e6cc3e7542bd8be3fa6be1c50d72)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.109975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.138030 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3854/7661] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:33:44 2022] PUT /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/70e3e6cc3e7542bd8be3fa6be1c50d72 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:44.142440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.153976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbdabd-681c-429b-82dc-99ef75fc986d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.213529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.214222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.214855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.215182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.216492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=1f9c898922a5461a88345608845cf1ac)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.217750 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:33:44.217750 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:33:44.217750 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:33:44.219011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.237255 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3808/7662] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:33:44 2022] GET /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users => generated 1085 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:44.241657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.254073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9236af-85b6-46e4-bd5a-3c15fc1cbe4d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.263733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.264760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.265136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.265445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/3f9399daa84c4cef8d6471044ca941cd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.268226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=3f9399daa84c4cef8d6471044ca941cd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.269807 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:33:44.269807 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:33:44.269807 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:33:44.271540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.288821 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3855/7663] 10.222.0.26 () {58 vars in 1342 bytes} [Tue Sep 27 10:33:44 2022] HEAD /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/3f9399daa84c4cef8d6471044ca941cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:44.293684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.304635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659cd1f-8f89-46c7-aa10-35ad6e91e8e4 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.310703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.311375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.311658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.311935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/3f9399daa84c4cef8d6471044ca941cd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.313752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=3f9399daa84c4cef8d6471044ca941cd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.315677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.326287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '3f9399daa84c4cef8d6471044ca941cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.343789 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3809/7664] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:33:44 2022] DELETE /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/3f9399daa84c4cef8d6471044ca941cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:44.348145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.359008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fb471-faff-4d83-bfd5-16439f118b83 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.419127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.419698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.420002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.420279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/30785747a9484a6892feee8691f60c00` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.422005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=30785747a9484a6892feee8691f60c00)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.423938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.438834 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3856/7665] 10.222.0.26 () {58 vars in 1342 bytes} [Tue Sep 27 10:33:44 2022] HEAD /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/30785747a9484a6892feee8691f60c00 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:44.443219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.459556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50622029-6b3f-4c82-a1c9-056996de3029 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.459556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.460229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.460510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.460924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/30785747a9484a6892feee8691f60c00` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.462608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=30785747a9484a6892feee8691f60c00)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.465015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.475322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '30785747a9484a6892feee8691f60c00'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.491841 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3810/7666] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:33:44 2022] DELETE /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/30785747a9484a6892feee8691f60c00 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:44.495993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.506306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeac981-3f00-4bd7-a550-d7db1c8bfe6e tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.511822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:44.565689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.566340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.566622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.566900 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/70e3e6cc3e7542bd8be3fa6be1c50d72` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.568579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=70e3e6cc3e7542bd8be3fa6be1c50d72)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.570602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.583961 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3857/7667] 10.222.0.26 () {58 vars in 1342 bytes} [Tue Sep 27 10:33:44 2022] HEAD /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/70e3e6cc3e7542bd8be3fa6be1c50d72 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:44.588369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.599607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f7647-8149-4d68-af47-c13972646c0c tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.604559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.605282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.605569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.605873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users/70e3e6cc3e7542bd8be3fa6be1c50d72` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.607744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=1f9c898922a5461a88345608845cf1ac, user_id=70e3e6cc3e7542bd8be3fa6be1c50d72)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.609736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.620554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '70e3e6cc3e7542bd8be3fa6be1c50d72'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.636407 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3811/7668] 10.222.0.26 () {58 vars in 1344 bytes} [Tue Sep 27 10:33:44 2022] DELETE /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users/70e3e6cc3e7542bd8be3fa6be1c50d72 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:44.640561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.651023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed3564-b1e0-456a-8245-d6ff614c489f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.711818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.712376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.712664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.712967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.714177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=1f9c898922a5461a88345608845cf1ac)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.715544 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:33:44.715544 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:33:44.715544 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:33:44.716778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.726292 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3858/7669] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:33:44 2022] GET /identity/v3/groups/1f9c898922a5461a88345608845cf1ac/users => generated 145 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:44.730874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.742126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c8398-2330-4b2c-842e-d40d60790d70 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.747478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.748115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.748394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.748692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users/70e3e6cc3e7542bd8be3fa6be1c50d72` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.749875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_user(user_id=70e3e6cc3e7542bd8be3fa6be1c50d72)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.751807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.771433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:44.802740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e3e6cc3e7542bd8be3fa6be1c50d72'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.803531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e3e6cc3e7542bd8be3fa6be1c50d72'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.810503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e3e6cc3e7542bd8be3fa6be1c50d72'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.819700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e3e6cc3e7542bd8be3fa6be1c50d72'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.831934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e3e6cc3e7542bd8be3fa6be1c50d72'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.838281 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3812/7670] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:44 2022] DELETE /identity/v3/users/70e3e6cc3e7542bd8be3fa6be1c50d72 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:44.842991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:44.845409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:44.858169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb8a8f-e3f9-44b9-b47f-177a773e301f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:44.915733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:44.916364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:44.916639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:44.916950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users/30785747a9484a6892feee8691f60c00` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:44.918103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_user(user_id=30785747a9484a6892feee8691f60c00)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:44.920144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:44.961265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:44.969505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30785747a9484a6892feee8691f60c00'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.970360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30785747a9484a6892feee8691f60c00'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.980098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30785747a9484a6892feee8691f60c00'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.988118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30785747a9484a6892feee8691f60c00'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:44.988845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30785747a9484a6892feee8691f60c00'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:45.014956 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3859/7671] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:44 2022] DELETE /identity/v3/users/30785747a9484a6892feee8691f60c00 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:45.019267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.029732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f8185-094f-4f25-9ddd-1fba065b0b69 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.090615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.091329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.091697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.091948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users/3f9399daa84c4cef8d6471044ca941cd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.093225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3f9399daa84c4cef8d6471044ca941cd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.095164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.141811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9399daa84c4cef8d6471044ca941cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:45.142482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9399daa84c4cef8d6471044ca941cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:45.152843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9399daa84c4cef8d6471044ca941cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:45.162951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9399daa84c4cef8d6471044ca941cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:45.175999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9399daa84c4cef8d6471044ca941cd'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:45.182094 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3813/7672] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:45 2022] DELETE /identity/v3/users/3f9399daa84c4cef8d6471044ca941cd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:45.186563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.188207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:45.202094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30676286-5c0f-4b0a-aa41-42aa8ac59148 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.263884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.264541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.264730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.265023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/1f9c898922a5461a88345608845cf1ac` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.266376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_group(group_id=1f9c898922a5461a88345608845cf1ac)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.268317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.313142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3860/7673] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:45 2022] DELETE /identity/v3/groups/1f9c898922a5461a88345608845cf1ac => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:45.320364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.322703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:45.341322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff11d24-8d09-480d-a17f-d4c2fbab53ab tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.352640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:45.405277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.405887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.406166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.406436 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.407075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.408998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.426331 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3814/7674] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:33:45 2022] POST /identity/v3/groups => generated 325 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:45.430678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.441787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7854b6b-9a13-415f-b3a3-dfdebf3668f1 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.446731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.447340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.447656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.447943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.448570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.450638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.469565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3861/7675] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:33:45 2022] POST /identity/v3/groups => generated 326 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:45.475000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.485916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4450e7-4c9e-43a5-a53b-c19d087dcc61 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.491190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.491790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.492071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.492357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.493023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.494932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.511522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3815/7676] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:33:45 2022] POST /identity/v3/groups => generated 327 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:45.515973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.527491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4ff66-2050-48a8-b249-fd959cd9edf3 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.532973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.533570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.533853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.534140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.534780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.535802 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:33:45.535802 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:33:45.535802 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:33:45.536950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.544458 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3862/7677] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:33:45 2022] GET /identity/v3/groups => generated 1488 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:45.549119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.559540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3176be-cb2c-415a-adb1-d933c4f4df66 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.564550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.565248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.565535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.565817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/75194c9a259342caa7b4e08c587a9633` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.572171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_group(group_id=75194c9a259342caa7b4e08c587a9633)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.574151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.628982 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3816/7678] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:45 2022] DELETE /identity/v3/groups/75194c9a259342caa7b4e08c587a9633 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:45.633271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.643774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76af45-3737-4070-bfc7-bca11bce25da tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.708287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:45.719226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.719882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.720349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.720857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/d5d3445421cf45068f6e087fa24b3ef8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.725935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_group(group_id=d5d3445421cf45068f6e087fa24b3ef8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.727850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.772649 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3863/7679] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:45 2022] DELETE /identity/v3/groups/d5d3445421cf45068f6e087fa24b3ef8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:45.776919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.787691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2698aa-24c4-46e6-81d3-a7634a9b1775 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.848998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.849663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.849937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.850185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/b279343d75e34659a11cac311aa88702` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.857097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_group(group_id=b279343d75e34659a11cac311aa88702)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.859827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:45.899346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:45.909340 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3817/7680] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:45 2022] DELETE /identity/v3/groups/b279343d75e34659a11cac311aa88702 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:45.914693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:45.925808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2529e96-5e45-4068-a373-008ceb6fdc31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:45.988772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:45.989381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:45.989660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:45.989936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:45.990572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:45.992504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.020860 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3864/7681] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:45 2022] POST /identity/v3/users => generated 321 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:46.024987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.036067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69c50a-82a9-4845-9ef4-925add49b831 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.042134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.042740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.043015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.043284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.043980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.045947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.063661 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3818/7682] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:33:46 2022] POST /identity/v3/groups => generated 325 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:46.069603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.080760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc1875-2b31-4b22-8cd6-1b0de8466f74 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.092549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.093435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.093724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.094011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/0f2f22702dd84ff2837727f305fc1ace/users/3e5772ae5a38455e8883d280d1dcef1a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.116043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=0f2f22702dd84ff2837727f305fc1ace, user_id=3e5772ae5a38455e8883d280d1dcef1a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.118188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.146199 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3865/7683] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:33:46 2022] PUT /identity/v3/groups/0f2f22702dd84ff2837727f305fc1ace/users/3e5772ae5a38455e8883d280d1dcef1a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:46.150421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.162196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49e15b-c88f-4b71-a26a-05e540c726ba tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.228429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.229437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.229681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.230075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.230720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.232807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.252912 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3819/7684] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:33:46 2022] POST /identity/v3/groups => generated 327 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:46.257425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.268732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4aacd-94bd-4a03-8a4f-5e8fcbfd7f8d tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.273779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.274432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.274716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.275039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/a0307e0025ad4356bd07507324efe4ba/users/3e5772ae5a38455e8883d280d1dcef1a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.281025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=a0307e0025ad4356bd07507324efe4ba, user_id=3e5772ae5a38455e8883d280d1dcef1a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.282970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.308060 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3866/7685] 10.222.0.26 () {60 vars in 1360 bytes} [Tue Sep 27 10:33:46 2022] PUT /identity/v3/groups/a0307e0025ad4356bd07507324efe4ba/users/3e5772ae5a38455e8883d280d1dcef1a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:46.312595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.324225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da77fc-7a08-4ac3-9dab-34121c5f6cf0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.388518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.389253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.389534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.389820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users/3e5772ae5a38455e8883d280d1dcef1a/groups` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.391035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=3e5772ae5a38455e8883d280d1dcef1a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.393203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.412739 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3820/7686] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:33:46 2022] GET /identity/v3/users/3e5772ae5a38455e8883d280d1dcef1a/groups => generated 838 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:46.414210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.425232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d5ef-0a90-47fe-a748-8509d5700737 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.430060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.430806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.431090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.431363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/a0307e0025ad4356bd07507324efe4ba` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.432599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_group(group_id=a0307e0025ad4356bd07507324efe4ba)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.434630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.464603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:46.489850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3867/7687] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:46 2022] DELETE /identity/v3/groups/a0307e0025ad4356bd07507324efe4ba => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:46.493974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.504944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43915fcb-c839-44ab-b4f9-21cf3377a409 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.561748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.562510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.562791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.563063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/groups/0f2f22702dd84ff2837727f305fc1ace` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.564225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_group(group_id=0f2f22702dd84ff2837727f305fc1ace)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.566629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.576181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:46.620492 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3821/7688] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:46 2022] DELETE /identity/v3/groups/0f2f22702dd84ff2837727f305fc1ace => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:46.625006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.636316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b31304-4bbd-413e-8e49-e5bc4525eeee tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.708104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.708901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.709239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.709596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/users/3e5772ae5a38455e8883d280d1dcef1a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.710875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3e5772ae5a38455e8883d280d1dcef1a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.712910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.770387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5772ae5a38455e8883d280d1dcef1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.771315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5772ae5a38455e8883d280d1dcef1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.790804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5772ae5a38455e8883d280d1dcef1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.807359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5772ae5a38455e8883d280d1dcef1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.810458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5772ae5a38455e8883d280d1dcef1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.839710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3868/7689] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:46 2022] DELETE /identity/v3/users/3e5772ae5a38455e8883d280d1dcef1a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:46.846094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:46.848052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:46.864954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5474b-9f9b-4d99-b908-5452c2472f31 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:46.941012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:46.941769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:46.942049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:46.942324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/domains/bbe23d1cf8834789b8f8b87c18d44b27` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:46.943690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=bbe23d1cf8834789b8f8b87c18d44b27)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:46.945432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:46.967677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.968394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.996507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:46.997210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.002522 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3822/7690] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:33:46 2022] PATCH /identity/v3/domains/bbe23d1cf8834789b8f8b87c18d44b27 => generated 277 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:33:47.003862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:47.014923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27f8e7-95d4-4dec-83b5-d6662e350222 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:47.076894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff9f80e09be94ade9971d22ad661b81c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6def473a7ee48eca824efb6c1bebee3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:47.077548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:47.078002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:47.078254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] PATH_INFO: `/v3/domains/bbe23d1cf8834789b8f8b87c18d44b27` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:47.081688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:47.092414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=bbe23d1cf8834789b8f8b87c18d44b27)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:47.094765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:47.120776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.139447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.195222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27', 'request_id': 'req-692d0b35-e70c-4b66-bc41-03d9902e33d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.204527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27', 'request_id': 'req-692d0b35-e70c-4b66-bc41-03d9902e33d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.211631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27', 'request_id': 'req-692d0b35-e70c-4b66-bc41-03d9902e33d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.224573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] The token cache is being invalidate because project bbe23d1cf8834789b8f8b87c18d44b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:47.224894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.225593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.233767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27', 'request_id': 'req-692d0b35-e70c-4b66-bc41-03d9902e33d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.234506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27', 'request_id': 'req-692d0b35-e70c-4b66-bc41-03d9902e33d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.237886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-692d0b35-e70c-4b66-bc41-03d9902e33d0 tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'bbe23d1cf8834789b8f8b87c18d44b27', 'request_id': 'req-692d0b35-e70c-4b66-bc41-03d9902e33d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.243651 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3869/7691] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:33:47 2022] DELETE /identity/v3/domains/bbe23d1cf8834789b8f8b87c18d44b27 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:47.252092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:47.253710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:47.325768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874316c-0bbe-455e-be7b-3f29b677c57f tempest-GroupsV3TestJSON-648923333 tempest-GroupsV3TestJSON-648923333-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:47.361748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:47.387452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:47.388553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:47.388872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:47.389210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] PATH_INFO: `/v3/users/1712e1d632ac4a6bb0fefe6493cd022b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:47.390342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1712e1d632ac4a6bb0fefe6493cd022b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:47.392277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:47.456332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1712e1d632ac4a6bb0fefe6493cd022b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.457263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1712e1d632ac4a6bb0fefe6493cd022b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.468091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1712e1d632ac4a6bb0fefe6493cd022b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.478117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1712e1d632ac4a6bb0fefe6493cd022b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.490860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1712e1d632ac4a6bb0fefe6493cd022b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:47.497518 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3823/7692] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:47 2022] DELETE /identity/v3/users/1712e1d632ac4a6bb0fefe6493cd022b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:47.502284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:47.502723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:47.502828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:47.523377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:47.524946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:47.589701 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3870/7693] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:47.599851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:47.601514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:47.665193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6be2c3-dd95-4de5-ab6a-d8eb3756691e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:47.669818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:47.670440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:47.670705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:47.670974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:47.681144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:47.682578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:47.692438 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3824/7694] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:47.977288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:47.978753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:47.991807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9a510-d246-447e-8f3b-3e16d5e90a5d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:48.000790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:48.001580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:48.001873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:48.002137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] PATH_INFO: `/v3/projects/8fcab8074e364277b02ba683cd28ec85` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:48.003279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fcab8074e364277b02ba683cd28ec85)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:48.005328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:48.047409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:48.060705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fcab8074e364277b02ba683cd28ec85', 'request_id': 'req-32c3a689-f11a-4a7a-b718-81a4fa855fd0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.069233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fcab8074e364277b02ba683cd28ec85', 'request_id': 'req-32c3a689-f11a-4a7a-b718-81a4fa855fd0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.074939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fcab8074e364277b02ba683cd28ec85', 'request_id': 'req-32c3a689-f11a-4a7a-b718-81a4fa855fd0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.086467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] The token cache is being invalidate because project 8fcab8074e364277b02ba683cd28ec85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:48.086751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.087382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.089504 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3871/7695] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:47 2022] DELETE /identity/v3/projects/8fcab8074e364277b02ba683cd28ec85 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:48.093733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:48.104044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4ccf6-c996-4e52-bb24-c2f2b05df267 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:48.166762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:48.167307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:48.167592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:48.167866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] PATH_INFO: `/v3/users/ff9f80e09be94ade9971d22ad661b81c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:48.169270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9f80e09be94ade9971d22ad661b81c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:48.171268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:48.172930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:48.224295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9f80e09be94ade9971d22ad661b81c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.224986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9f80e09be94ade9971d22ad661b81c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.234346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9f80e09be94ade9971d22ad661b81c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.243119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9f80e09be94ade9971d22ad661b81c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.256460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a2968f4a-951b-47c8-a807-b56a8afa4fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9f80e09be94ade9971d22ad661b81c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.263431 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3825/7696] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:48 2022] DELETE /identity/v3/users/ff9f80e09be94ade9971d22ad661b81c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:48.559990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:48.561709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:48.595986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a689-f11a-4a7a-b718-81a4fa855fd0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:48.661719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:48.662382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:48.662812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:48.663090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] PATH_INFO: `/v3/projects/e6def473a7ee48eca824efb6c1bebee3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:48.664219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6def473a7ee48eca824efb6c1bebee3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:48.666298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:48.719307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6def473a7ee48eca824efb6c1bebee3', 'request_id': 'req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.727825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6def473a7ee48eca824efb6c1bebee3', 'request_id': 'req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.734011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6def473a7ee48eca824efb6c1bebee3', 'request_id': 'req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.745829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] The token cache is being invalidate because project e6def473a7ee48eca824efb6c1bebee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:48.746164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.747013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d54bb2df-23db-48ae-ac28-f25b02c1a6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:48.748607 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3872/7697] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:48 2022] DELETE /identity/v3/projects/e6def473a7ee48eca824efb6c1bebee3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:48.753737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7d22-0606-4577-a8ee-e642d44e0c30 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:48.754230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7d22-0606-4577-a8ee-e642d44e0c30 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:48.754359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7d22-0606-4577-a8ee-e642d44e0c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:48.776177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7d1e7d22-0606-4577-a8ee-e642d44e0c30 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:48.777422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7d1e7d22-0606-4577-a8ee-e642d44e0c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:48.843093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3826/7698] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:48.852310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601890a-7558-423a-9e3c-8f77b23f2064 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:48.853708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601890a-7558-423a-9e3c-8f77b23f2064 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:48.854330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601890a-7558-423a-9e3c-8f77b23f2064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:48.880429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1601890a-7558-423a-9e3c-8f77b23f2064 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:48.881741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1601890a-7558-423a-9e3c-8f77b23f2064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:48.890172 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3873/7699] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:48.894662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e7d22-0606-4577-a8ee-e642d44e0c30 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:48.905134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e7d22-0606-4577-a8ee-e642d44e0c30 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:48.911366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:48.911366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:48.911706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:48.911988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:48.912684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:48.913341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:48.914628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:48.931104 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3827/7700] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:48.935692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601890a-7558-423a-9e3c-8f77b23f2064 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:48.945806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601890a-7558-423a-9e3c-8f77b23f2064 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:48.951067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:48.951690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:48.951970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:48.952417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:48.953097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:48.955322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:48.989732 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3874/7701] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:48 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:48.993906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.006024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce116b1-1296-4d81-b6f7-ee0dc71663eb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.087154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.088602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.089371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.089737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.090596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.093501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.107789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:49.122874 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3828/7702] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:48 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:49.124358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.135435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50c656-2bfd-4358-a5da-117e2ebc698a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.140538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.141224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.141496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.141761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.142396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.143988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.155451 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3875/7703] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:49.159547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.170365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba3293-5922-42ab-b3f4-6774418bc07b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.176196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a612d-c238-4db7-b8f2-437cad3b01d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.176860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a612d-c238-4db7-b8f2-437cad3b01d3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.177139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a612d-c238-4db7-b8f2-437cad3b01d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.177411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a612d-c238-4db7-b8f2-437cad3b01d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.178063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a612d-c238-4db7-b8f2-437cad3b01d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.179677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a612d-c238-4db7-b8f2-437cad3b01d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.194787 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3829/7704] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:49.196338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.207156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fa45d-12f8-4533-af09-8dead68ab952 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.212098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.212786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.213067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.213341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] PATH_INFO: `/v3/projects/d9e28a9e9c2946fbb8a2cf7ee20d4238/users/2bf8b34e4aaf42ecb2dcded44de98841/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.225645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:49.229930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e28a9e9c2946fbb8a2cf7ee20d4238, user_id=2bf8b34e4aaf42ecb2dcded44de98841, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.233323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.249749 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3876/7705] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:49 2022] PUT /identity/v3/projects/d9e28a9e9c2946fbb8a2cf7ee20d4238/users/2bf8b34e4aaf42ecb2dcded44de98841/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:49.260263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2a89-59de-4d1c-8c5c-98128a81c8d9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.260931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2a89-59de-4d1c-8c5c-98128a81c8d9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.260931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2a89-59de-4d1c-8c5c-98128a81c8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.290167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-64cd2a89-59de-4d1c-8c5c-98128a81c8d9 None None] MFA Rules not processed for user `2bf8b34e4aaf42ecb2dcded44de98841`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:49.291432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-64cd2a89-59de-4d1c-8c5c-98128a81c8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:49.365053 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3830/7706] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:49.369852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.452708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd19e64-ae31-41a2-a0cc-d7e28733523d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.458425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.459136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.459476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.461828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.462590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.465091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.516523 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3877/7707] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:49 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:49.520880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2a89-59de-4d1c-8c5c-98128a81c8d9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.537847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2a89-59de-4d1c-8c5c-98128a81c8d9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.632430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.633359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.633759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.634030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.636058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.636990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.667239 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3831/7708] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:49 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:49.671541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.685622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce86f9-61d3-4f5e-8285-6647ceb1ce3e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.690971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.691764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.692148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.692527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.693315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.695171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.711048 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3878/7709] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:49.716263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.729857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3b660-879d-471c-8f40-91d0745c11be admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.736094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.737716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.737716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.738358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] PATH_INFO: `/v3/projects/3e07e905392f40b9b0b1f48e054d6125/users/dd95f9d7dd80482abc4f1aff2b09ecaf/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.761709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e07e905392f40b9b0b1f48e054d6125, user_id=dd95f9d7dd80482abc4f1aff2b09ecaf, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.764796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.783721 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3832/7710] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:49 2022] PUT /identity/v3/projects/3e07e905392f40b9b0b1f48e054d6125/users/dd95f9d7dd80482abc4f1aff2b09ecaf/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:49.788082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.799270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb874f1-37da-4c45-92d8-908366f13495 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.880700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3b814-bd29-4953-82c8-f139d9bbd69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.880700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3b814-bd29-4953-82c8-f139d9bbd69a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.880700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3b814-bd29-4953-82c8-f139d9bbd69a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.880700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3b814-bd29-4953-82c8-f139d9bbd69a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.881250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3b814-bd29-4953-82c8-f139d9bbd69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.881345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3b814-bd29-4953-82c8-f139d9bbd69a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.903418 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3879/7711] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:49.905545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:49.920092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471138b3-b107-42a1-9497-7dc9987cb508 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:49.925298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:49.926207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.926437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.926806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] PATH_INFO: `/v3/projects/3e07e905392f40b9b0b1f48e054d6125/users/dd95f9d7dd80482abc4f1aff2b09ecaf/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.929042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e07e905392f40b9b0b1f48e054d6125, user_id=dd95f9d7dd80482abc4f1aff2b09ecaf, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:49.931784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:49.946123 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3833/7712] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:49 2022] PUT /identity/v3/projects/3e07e905392f40b9b0b1f48e054d6125/users/dd95f9d7dd80482abc4f1aff2b09ecaf/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:49.960764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08d7b-7681-4d08-9516-099b41599395 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:49.961298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08d7b-7681-4d08-9516-099b41599395 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:49.962256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08d7b-7681-4d08-9516-099b41599395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:49.996532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-26b08d7b-7681-4d08-9516-099b41599395 None None] MFA Rules not processed for user `dd95f9d7dd80482abc4f1aff2b09ecaf`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:49.997733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-26b08d7b-7681-4d08-9516-099b41599395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:50.074386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:50.116377 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3880/7713] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:50.121525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.134243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0912a04-9704-4228-9efd-1c3c1df47f02 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.142708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.144226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.144226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.144226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.144795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.146691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.193352 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3834/7714] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:50 2022] POST /identity/v3/projects => generated 358 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:50.198345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08d7b-7681-4d08-9516-099b41599395 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.209097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08d7b-7681-4d08-9516-099b41599395 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.279997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.281065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.281340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.281608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-821109626` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.282910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=a78bdb02568e43caa2267c7d15854154, value=tempest-tag-821109626)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.285056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.355331 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-821109626. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:33:50.355331 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:33:50.355331 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:33:50.363642 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3881/7715] 10.222.0.26 () {60 vars in 1330 bytes} [Tue Sep 27 10:33:50 2022] PUT /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-821109626 => generated 0 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 290 bytes (1 switches on core 0) Sep 27 10:33:50.370195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.378560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:50.383315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362776-fcb0-4948-badf-f5e750ad944f tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.391976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.392650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.393617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.393617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-821109626` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.406013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=a78bdb02568e43caa2267c7d15854154, value=tempest-tag-821109626)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.408241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.425286 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3835/7716] 10.222.0.26 () {58 vars in 1311 bytes} [Tue Sep 27 10:33:50 2022] GET /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-821109626 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:50.430122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.441459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d866cf-a6ff-4160-a4fa-f41fa3a39ed4 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.446491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.447214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.447504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.448397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.449768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=a78bdb02568e43caa2267c7d15854154)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.453630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.527408 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3882/7717] 10.222.0.26 () {60 vars in 1265 bytes} [Tue Sep 27 10:33:50 2022] PUT /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags => generated 141 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:50.531987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.546427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76fa77-93de-4aa3-8ae7-db1d64dc0d34 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.556893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.557982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.558257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.558669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.571008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=a78bdb02568e43caa2267c7d15854154)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.573252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.584960 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3836/7718] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:33:50 2022] GET /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags => generated 141 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:50.589366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.600596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d970176-d846-46b6-9a9a-56b0feab4d75 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.608091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.608846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.609134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.609400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-693321316` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.614811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=a78bdb02568e43caa2267c7d15854154, value=tempest-tag-693321316)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.617027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.675572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.677138 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-693321316. To ensure interoperability, identifiers should be a valid uuid. Sep 27 10:33:50.677138 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 27 10:33:50.677138 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:33:50.682646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:50.686328 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3883/7719] 10.222.0.26 () {58 vars in 1314 bytes} [Tue Sep 27 10:33:50 2022] DELETE /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-693321316 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:50.687271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.699261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5260f-acb5-4e1e-856c-1e889a7f2455 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.706091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.707440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.707893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.708322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-693321316` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.726064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=a78bdb02568e43caa2267c7d15854154, value=tempest-tag-693321316)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.729354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.737924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Could not find project tag: tempest-tag-693321316.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-693321316. Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Sep 27 10:33:50.748702 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Sep 27 10:33:50.750374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:33:50.750374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:33:50.750374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Sep 27 10:33:50.750374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Sep 27 10:33:50.750374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-693321316. Sep 27 10:33:50.750374 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:33:50.753444 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3884/7720] 10.222.0.26 () {58 vars in 1311 bytes} [Tue Sep 27 10:33:50 2022] GET /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags/tempest-tag-693321316 => generated 106 bytes in 67 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:33:50.760007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.772730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822445e-adea-4748-b334-627659239edd tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.778064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.778984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.779416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.779767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.781949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=a78bdb02568e43caa2267c7d15854154)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.785002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.818582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70c9a-0b33-4bb7-904d-7fd2cbc586ab tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.826246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.826896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.827472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.827659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.830057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:50.848257 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3885/7721] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:33:50 2022] DELETE /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:50.849533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.861393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da397415-224b-4962-8c7c-e526aec8e132 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.884229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.884931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.885347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:50.885625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154/tags` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:50.924590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=a78bdb02568e43caa2267c7d15854154)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.926982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.936483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:50.938385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:50.950660 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3837/7722] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:50 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:50.972776 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3886/7723] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:33:50 2022] GET /identity/v3/projects/a78bdb02568e43caa2267c7d15854154/tags => generated 66 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 27 10:33:50.974087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:50.991912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d1c77-1a52-4a20-80b3-ccfd51199fc3 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:50.998227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd95f9d7dd80482abc4f1aff2b09ecaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e07e905392f40b9b0b1f48e054d6125', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:50.999463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:50.999941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:51.000358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] PATH_INFO: `/v3/projects/a78bdb02568e43caa2267c7d15854154` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:51.001955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a78bdb02568e43caa2267c7d15854154)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:51.004331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:51.115839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:51.120757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78bdb02568e43caa2267c7d15854154', 'request_id': 'req-4c735857-6ca3-44fa-865e-8db2af40b437'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.130117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78bdb02568e43caa2267c7d15854154', 'request_id': 'req-4c735857-6ca3-44fa-865e-8db2af40b437'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.140997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78bdb02568e43caa2267c7d15854154', 'request_id': 'req-4c735857-6ca3-44fa-865e-8db2af40b437'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.159216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] The token cache is being invalidate because project a78bdb02568e43caa2267c7d15854154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:51.159511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.160652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.167685 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3838/7724] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:50 2022] DELETE /identity/v3/projects/a78bdb02568e43caa2267c7d15854154 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:51.169337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:51.171445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.283614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce04ec-f9f3-4ca2-a04b-ce6770c154e1 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:51.333321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:51.334766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.360868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:51.361585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:51.361873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:51.362198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] PATH_INFO: `/v3/users/2bf8b34e4aaf42ecb2dcded44de98841` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:51.363842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf8b34e4aaf42ecb2dcded44de98841)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:51.365823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:51.465962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf8b34e4aaf42ecb2dcded44de98841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.466598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf8b34e4aaf42ecb2dcded44de98841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.468559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:51.480463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf8b34e4aaf42ecb2dcded44de98841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.485339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735857-6ca3-44fa-865e-8db2af40b437 tempest-IdentityV3ProjectTagsTest-633521057 tempest-IdentityV3ProjectTagsTest-633521057-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:51.489249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf8b34e4aaf42ecb2dcded44de98841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.489942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf8b34e4aaf42ecb2dcded44de98841'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.493255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afaba9-0de1-4b18-b942-aa0879279e60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:51.493892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afaba9-0de1-4b18-b942-aa0879279e60 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:51.494176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afaba9-0de1-4b18-b942-aa0879279e60 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:51.494616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afaba9-0de1-4b18-b942-aa0879279e60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:51.496799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f7afaba9-0de1-4b18-b942-aa0879279e60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.511239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afaba9-0de1-4b18-b942-aa0879279e60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:51.511807 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3887/7725] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:51 2022] DELETE /identity/v3/users/2bf8b34e4aaf42ecb2dcded44de98841 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:51.513474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:51.513735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afaba9-0de1-4b18-b942-aa0879279e60 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:51.515886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.527697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3839/7726] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:33:51.539718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:51.540001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:51.540280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:51.581430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:51.582772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.606400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcac1b-5430-451f-88a4-f8e421805014 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:51.615269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:51.616007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:51.616321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:51.616600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] PATH_INFO: `/v3/users/6771df45b4ee4426b2f76a09eab2998a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:51.618094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6771df45b4ee4426b2f76a09eab2998a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:51.620553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:51.677192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6771df45b4ee4426b2f76a09eab2998a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.678303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6771df45b4ee4426b2f76a09eab2998a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.680721 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3840/7727] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:51.688415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6771df45b4ee4426b2f76a09eab2998a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.688982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:51.691540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.697707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6771df45b4ee4426b2f76a09eab2998a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.698459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6771df45b4ee4426b2f76a09eab2998a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:51.719465 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3888/7728] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:51 2022] DELETE /identity/v3/users/6771df45b4ee4426b2f76a09eab2998a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:51.778448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2330ae-e783-4d06-a14f-b8366d3129fc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:51.800700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:51.844403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54f556-fdb4-405d-b1c2-73245ca7f011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:51.845131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54f556-fdb4-405d-b1c2-73245ca7f011 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:51.845452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54f556-fdb4-405d-b1c2-73245ca7f011 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:51.845632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54f556-fdb4-405d-b1c2-73245ca7f011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:51.847894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5a54f556-fdb4-405d-b1c2-73245ca7f011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.923496 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:51.926031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:51.952717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54f556-fdb4-405d-b1c2-73245ca7f011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:51.954135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54f556-fdb4-405d-b1c2-73245ca7f011 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:51.966127 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3841/7729] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:33:51.999646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:52.026810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d14dd2-6f7b-4e3f-8e31-99758b319600 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.033530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.034191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.034572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.034845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] PATH_INFO: `/v3/projects/40247edd5a444fd8b1c937c13d39393a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.036005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40247edd5a444fd8b1c937c13d39393a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.038232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.095344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40247edd5a444fd8b1c937c13d39393a', 'request_id': 'req-fc209a6a-deed-4f50-8f36-27157fffeaf7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.104356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40247edd5a444fd8b1c937c13d39393a', 'request_id': 'req-fc209a6a-deed-4f50-8f36-27157fffeaf7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.112012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40247edd5a444fd8b1c937c13d39393a', 'request_id': 'req-fc209a6a-deed-4f50-8f36-27157fffeaf7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.125046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] The token cache is being invalidate because project 40247edd5a444fd8b1c937c13d39393a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:52.125420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.126101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fc209a6a-deed-4f50-8f36-27157fffeaf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.131055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3889/7730] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:51 2022] DELETE /identity/v3/projects/40247edd5a444fd8b1c937c13d39393a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:52.134376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895166-97b3-4488-93fa-986511078846 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.134678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895166-97b3-4488-93fa-986511078846 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.134953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895166-97b3-4488-93fa-986511078846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.164057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-54895166-97b3-4488-93fa-986511078846 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:52.165506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-54895166-97b3-4488-93fa-986511078846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:52.239536 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3842/7731] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:52.247923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.248289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.248634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.270141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:52.271615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:52.280743 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3890/7732] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:52.285987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895166-97b3-4488-93fa-986511078846 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.294898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.296740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:52.298644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895166-97b3-4488-93fa-986511078846 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.304766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.305422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.305714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.306120 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.306780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:52.307079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.309606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.317448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974d91-01b1-4e19-a344-211334e9d407 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.327048 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3843/7733] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:52.327206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.327206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.327206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.327206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] PATH_INFO: `/v3/projects/d9e28a9e9c2946fbb8a2cf7ee20d4238` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.327629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e28a9e9c2946fbb8a2cf7ee20d4238)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.327761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.332134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.339700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300eaf2-e1df-4ffc-bf79-b5d50f15c588 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.345270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.345890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.346214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.346491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.347122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.349327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.384342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3844/7734] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:52 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:52.385532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.397553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e28a9e9c2946fbb8a2cf7ee20d4238', 'request_id': 'req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.409983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e28a9e9c2946fbb8a2cf7ee20d4238', 'request_id': 'req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.417267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e28a9e9c2946fbb8a2cf7ee20d4238', 'request_id': 'req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.430295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] The token cache is being invalidate because project d9e28a9e9c2946fbb8a2cf7ee20d4238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:52.430624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.431279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.432093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:52.433088 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3891/7735] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:52 2022] DELETE /identity/v3/projects/d9e28a9e9c2946fbb8a2cf7ee20d4238 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:52.437669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.438744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:52.458642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53da75-2dc3-49ef-9322-676ea11f71af admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.464989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.465827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.466134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.466382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.467075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.469058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.493973 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3845/7736] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:52 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:52.498115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.501176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:52.505581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:52.514900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb064e-28d4-488d-827e-b85667a61e1a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.520480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.520913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ba4ae-14b0-41dc-8827-dd8ca67ee7b3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.521033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.521033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.521333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.521926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.523726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.526931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.528569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.528974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.529351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] PATH_INFO: `/v3/users/dd95f9d7dd80482abc4f1aff2b09ecaf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.530617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd95f9d7dd80482abc4f1aff2b09ecaf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.532574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.544859 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3846/7737] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:52.548679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.559681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5921f7-2c4b-4cef-b5c5-e16dc70ab95f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.565003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47192074-dace-416a-91a1-30e801977357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.565618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47192074-dace-416a-91a1-30e801977357 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.565898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47192074-dace-416a-91a1-30e801977357 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.566169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47192074-dace-416a-91a1-30e801977357 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.567007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47192074-dace-416a-91a1-30e801977357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.568400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47192074-dace-416a-91a1-30e801977357 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.582611 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3847/7738] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:52.586763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47192074-dace-416a-91a1-30e801977357 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.597772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd95f9d7dd80482abc4f1aff2b09ecaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.598539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd95f9d7dd80482abc4f1aff2b09ecaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.602394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47192074-dace-416a-91a1-30e801977357 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.608966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.609726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.611219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd95f9d7dd80482abc4f1aff2b09ecaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.614277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.614451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] PATH_INFO: `/v3/projects/f91c166b86a84e9fbd3af3de3a820288/users/f38b6a57d5e446e7bbd171fcdefbbeb6/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.626105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd95f9d7dd80482abc4f1aff2b09ecaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.627036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8d886c02-2f4b-435d-95c0-6fb560734e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd95f9d7dd80482abc4f1aff2b09ecaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:52.635746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91c166b86a84e9fbd3af3de3a820288, user_id=f38b6a57d5e446e7bbd171fcdefbbeb6, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.639485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:52.649761 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3892/7739] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:52 2022] DELETE /identity/v3/users/dd95f9d7dd80482abc4f1aff2b09ecaf => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:52.676697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3848/7740] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:52 2022] PUT /identity/v3/projects/f91c166b86a84e9fbd3af3de3a820288/users/f38b6a57d5e446e7bbd171fcdefbbeb6/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:52.686426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.687269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.687731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.732874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] MFA Rules not processed for user `f38b6a57d5e446e7bbd171fcdefbbeb6`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:52.735745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:52.833991 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3893/7741] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:52.845741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:52.848365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:52.969897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d9796-5ac7-4632-9285-6123eed7f8ac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:52.976566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:52.977241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:52.977920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:52.978344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:52.988917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:52.990954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.002656 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3849/7742] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:33:53.059700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.061422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:53.126185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.128188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:53.182505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39512-268d-4c82-900e-c65a8302a436 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.189009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.190469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.190752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.191192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] PATH_INFO: `/v3/users/f38b6a57d5e446e7bbd171fcdefbbeb6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.192355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38b6a57d5e446e7bbd171fcdefbbeb6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.194398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.196600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:53.228835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fd6d1-dd9b-446f-b48d-9c42a704c098 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.234905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.235627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.236039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.236322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] PATH_INFO: `/v3/projects/3e07e905392f40b9b0b1f48e054d6125` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.237563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e07e905392f40b9b0b1f48e054d6125)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.239725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.249438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:53.251053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38b6a57d5e446e7bbd171fcdefbbeb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.252236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38b6a57d5e446e7bbd171fcdefbbeb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.263476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38b6a57d5e446e7bbd171fcdefbbeb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.272690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38b6a57d5e446e7bbd171fcdefbbeb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.273591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0dd96087-cdef-414d-9461-e2d59e60485a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38b6a57d5e446e7bbd171fcdefbbeb6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.294413 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3894/7743] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:53 2022] DELETE /identity/v3/users/f38b6a57d5e446e7bbd171fcdefbbeb6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:53.296294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e07e905392f40b9b0b1f48e054d6125', 'request_id': 'req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.299627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.299975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.300192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.303051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e07e905392f40b9b0b1f48e054d6125', 'request_id': 'req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.313026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e07e905392f40b9b0b1f48e054d6125', 'request_id': 'req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.323341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:53.325488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:53.331776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] The token cache is being invalidate because project 3e07e905392f40b9b0b1f48e054d6125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:53.331927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.332841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bb14af3d-2d73-4fcc-b10c-8f6182fd1131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:53.335623 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3850/7744] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:53 2022] DELETE /identity/v3/projects/3e07e905392f40b9b0b1f48e054d6125 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:53.342055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b00ee-0875-443d-93f0-415c44876a81 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.342269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b00ee-0875-443d-93f0-415c44876a81 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.342547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b00ee-0875-443d-93f0-415c44876a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.366883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d60b00ee-0875-443d-93f0-415c44876a81 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:53.368164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d60b00ee-0875-443d-93f0-415c44876a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:53.405466 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3895/7745] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:53.418622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.420608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:53.437255 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3851/7746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:53.446358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfa55-c068-4c41-957f-62a6a5361ab5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.446735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfa55-c068-4c41-957f-62a6a5361ab5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.446910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfa55-c068-4c41-957f-62a6a5361ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.470406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d99bfa55-c068-4c41-957f-62a6a5361ab5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:53.471626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d99bfa55-c068-4c41-957f-62a6a5361ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:53.481969 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3852/7747] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:53.486586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bfa55-c068-4c41-957f-62a6a5361ab5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.499245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7a312-7995-40d9-9cfb-3b994db7f887 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.500503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bfa55-c068-4c41-957f-62a6a5361ab5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.503228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.503857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.504134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.504412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.506238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.507162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.507447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.507729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.508385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:53.508715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.510333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.516781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.518238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.529234 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3853/7748] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:33:53.530925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.533613 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3896/7749] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:33:53.544096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6b935-416c-41d9-91e7-20850e6e3fd2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.549702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.550338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.550618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.550895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.551543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.553726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.584636 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3854/7750] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:53 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:53.589395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.605671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2584-c962-4788-90d2-97acbc0be4a5 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.649063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:53.706851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.708492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.709078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.709353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.710001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.712606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.739972 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3897/7751] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:53 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:53.746026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.761952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc86dd-ead0-414f-8e4b-578dcbae44ca admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.769368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.770015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.770335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.770564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.771234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.772934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.786014 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3855/7752] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:53.789800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.801395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199143-ab42-4f8e-ad6d-b49082f4f09e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.812992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.813628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.813908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.814181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.814990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.816821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.829470 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3898/7753] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:53.833997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.846340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24a95f-eed8-4e86-8c18-b385c9857990 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.851961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f42c8c-cc2b-4166-89da-bc60d0061226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:53.852708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f42c8c-cc2b-4166-89da-bc60d0061226 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.853133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f42c8c-cc2b-4166-89da-bc60d0061226 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.853420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f42c8c-cc2b-4166-89da-bc60d0061226 admin admin] PATH_INFO: `/v3/projects/2a930d05ce814db6be5a3005ab6b9662/users/2e8f50d7ebee4a5fa2d302a53855f1d7/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.872543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f42c8c-cc2b-4166-89da-bc60d0061226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a930d05ce814db6be5a3005ab6b9662, user_id=2e8f50d7ebee4a5fa2d302a53855f1d7, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:53.875271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f42c8c-cc2b-4166-89da-bc60d0061226 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:53.884187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:53.886366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:53.902238 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3856/7754] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:53 2022] PUT /identity/v3/projects/2a930d05ce814db6be5a3005ab6b9662/users/2e8f50d7ebee4a5fa2d302a53855f1d7/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:53.903852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184b1ab-2ec6-4b2d-b7df-0a97a2735bc2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:53.923506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e46f5-a74e-4c36-8639-f9e6c016a6b6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:53.924318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e46f5-a74e-4c36-8639-f9e6c016a6b6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:53.924871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e46f5-a74e-4c36-8639-f9e6c016a6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:53.993678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-986e46f5-a74e-4c36-8639-f9e6c016a6b6 None None] MFA Rules not processed for user `2e8f50d7ebee4a5fa2d302a53855f1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:53.995004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-986e46f5-a74e-4c36-8639-f9e6c016a6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:54.006404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:54.008108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.009209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.009500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.009777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] PATH_INFO: `/v3/projects/f91c166b86a84e9fbd3af3de3a820288` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.011092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91c166b86a84e9fbd3af3de3a820288)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.014309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.080098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91c166b86a84e9fbd3af3de3a820288', 'request_id': 'req-32c668bb-354e-41a0-9d0c-406b5a904e5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:54.084715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3857/7755] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:54.089275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e46f5-a74e-4c36-8639-f9e6c016a6b6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.093187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91c166b86a84e9fbd3af3de3a820288', 'request_id': 'req-32c668bb-354e-41a0-9d0c-406b5a904e5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:54.100099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91c166b86a84e9fbd3af3de3a820288', 'request_id': 'req-32c668bb-354e-41a0-9d0c-406b5a904e5f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:54.109652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e46f5-a74e-4c36-8639-f9e6c016a6b6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.113819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] The token cache is being invalidate because project f91c166b86a84e9fbd3af3de3a820288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:54.114117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:54.115417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-32c668bb-354e-41a0-9d0c-406b5a904e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:54.116549 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3899/7756] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:53 2022] DELETE /identity/v3/projects/f91c166b86a84e9fbd3af3de3a820288 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:54.122893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0972fc-3429-44b1-92b6-af5743dbfce5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.123174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0972fc-3429-44b1-92b6-af5743dbfce5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.123448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0972fc-3429-44b1-92b6-af5743dbfce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.144295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-df0972fc-3429-44b1-92b6-af5743dbfce5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:54.145559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df0972fc-3429-44b1-92b6-af5743dbfce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:54.173998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:54.190116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.190742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.191019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.191489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.192128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.195566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.221185 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3900/7757] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:54.233970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd98957-db68-4ea6-aa72-9dacbd846596 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.234280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd98957-db68-4ea6-aa72-9dacbd846596 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.234517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd98957-db68-4ea6-aa72-9dacbd846596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.246638 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3858/7758] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:54.247647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.250876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:54.270069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4dd98957-db68-4ea6-aa72-9dacbd846596 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:54.271415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4dd98957-db68-4ea6-aa72-9dacbd846596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:54.330581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dff799-4a92-4202-8295-9bff2593ca94 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.335649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.336367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.336742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.337083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.337657 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3901/7759] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:54.337859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.340405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.342572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd98957-db68-4ea6-aa72-9dacbd846596 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.354031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd98957-db68-4ea6-aa72-9dacbd846596 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.358919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.359734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.360016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.360289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.360976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:54.361302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.362937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.376085 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3902/7760] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:54.379054 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3859/7761] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:54.380392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.383014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.392292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f8471-b589-4ba3-89f5-640447c53600 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.393296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465baf83-4470-4ff1-9744-a4aa5966f973 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.397148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.397850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.398005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.398172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.398444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.398569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.398779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.399048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.399156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.399685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.400987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.401292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.413348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3860/7762] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:54.417369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.428009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ef4a7-2ac9-48cc-b610-a9edb0749eca admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.433223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.434078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.434605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.434945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] PATH_INFO: `/v3/projects/a8b9bbbab69148698bddd34616b4c1b3/users/5330141373d44166a2185dbe591da13a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.437347 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3903/7763] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:54.441374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.453212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:54.458282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b9bbbab69148698bddd34616b4c1b3, user_id=5330141373d44166a2185dbe591da13a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.461990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.482404 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3861/7764] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:54 2022] PUT /identity/v3/projects/a8b9bbbab69148698bddd34616b4c1b3/users/5330141373d44166a2185dbe591da13a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:54.487688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.516409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9096c0-1133-4502-99a5-bf7037a918c6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.559346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d11288-140d-4cb8-b023-1d30a1d8ee1c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.564678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.565329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.565608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.565879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.566525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.569162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.570414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:54.587368 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3862/7765] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:54.589165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.590351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.591125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.591486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.591824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.592529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.594736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.599884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddfbd-a150-47ef-a28d-91b4fff88423 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.605683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426012ee-c53c-4a72-8d1d-159707da4857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.606473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426012ee-c53c-4a72-8d1d-159707da4857 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.606736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426012ee-c53c-4a72-8d1d-159707da4857 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.607429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426012ee-c53c-4a72-8d1d-159707da4857 admin admin] PATH_INFO: `/v3/projects/a8b9bbbab69148698bddd34616b4c1b3/users/5330141373d44166a2185dbe591da13a/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.609209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426012ee-c53c-4a72-8d1d-159707da4857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b9bbbab69148698bddd34616b4c1b3, user_id=5330141373d44166a2185dbe591da13a, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.613115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426012ee-c53c-4a72-8d1d-159707da4857 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.633606 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3863/7766] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:54 2022] PUT /identity/v3/projects/a8b9bbbab69148698bddd34616b4c1b3/users/5330141373d44166a2185dbe591da13a/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:54.636254 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3904/7767] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/users => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:54.637546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.637869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a9d49-53a6-4b87-a0e6-f7d22ae089c7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.638222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a9d49-53a6-4b87-a0e6-f7d22ae089c7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.638568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a9d49-53a6-4b87-a0e6-f7d22ae089c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.650097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528988c-5e11-48e8-92f5-8cffc5d1bf1e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.682499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-aa1a9d49-53a6-4b87-a0e6-f7d22ae089c7 None None] MFA Rules not processed for user `5330141373d44166a2185dbe591da13a`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:54.685113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aa1a9d49-53a6-4b87-a0e6-f7d22ae089c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:54.730911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.731590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.731840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.732075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.732759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.734893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.746540 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3905/7768] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:54.751012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.762347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17acc1-6a77-4335-846d-67f925fd84e6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.768218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.769297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.769297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.769530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.770163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.771365 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3864/7769] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:54.776118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.776511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a9d49-53a6-4b87-a0e6-f7d22ae089c7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.791075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3906/7770] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:54.791270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a9d49-53a6-4b87-a0e6-f7d22ae089c7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.792200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.795380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5330141373d44166a2185dbe591da13a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8b9bbbab69148698bddd34616b4c1b3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.796229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.796957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.801083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.801767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.802697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824de355-032e-48c0-ba54-6b9f3ab27f0d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.803730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.809146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65fd93-dc57-48b7-94f0-5c94ef399bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:54.810407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65fd93-dc57-48b7-94f0-5c94ef399bff admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.810711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65fd93-dc57-48b7-94f0-5c94ef399bff admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.811370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65fd93-dc57-48b7-94f0-5c94ef399bff admin admin] PATH_INFO: `/v3/projects/0c4391bbbcfa417697dab0922bf87931/users/91717560957a4e68ae31cee8f6af43f3/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.833779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65fd93-dc57-48b7-94f0-5c94ef399bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4391bbbcfa417697dab0922bf87931, user_id=91717560957a4e68ae31cee8f6af43f3, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:54.836790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65fd93-dc57-48b7-94f0-5c94ef399bff admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:54.847549 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3865/7771] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/projects => generated 359 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:54.852064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:54.854063 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3907/7772] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:54 2022] PUT /identity/v3/projects/0c4391bbbcfa417697dab0922bf87931/users/91717560957a4e68ae31cee8f6af43f3/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:54.855362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:54.864069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cccb4-12d3-49c6-a4a1-25ebf5766ea0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:54.864380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cccb4-12d3-49c6-a4a1-25ebf5766ea0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:54.864616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cccb4-12d3-49c6-a4a1-25ebf5766ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:54.895392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f24cccb4-12d3-49c6-a4a1-25ebf5766ea0 None None] MFA Rules not processed for user `91717560957a4e68ae31cee8f6af43f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:54.896460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f24cccb4-12d3-49c6-a4a1-25ebf5766ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:54.909899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:54.941000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54956669-7df2-4072-a969-dfa87855a2d9 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:54.975566 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3908/7773] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:54.980251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cccb4-12d3-49c6-a4a1-25ebf5766ea0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.006140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8f50d7ebee4a5fa2d302a53855f1d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a930d05ce814db6be5a3005ab6b9662', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.006806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.007586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.007586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] PATH_INFO: `/v3/projects/cd244e62f2824ffc8a4345fc45c47dc7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.008551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] RBAC: Authorizing `identity:delete_project(project_id=cd244e62f2824ffc8a4345fc45c47dc7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.009707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Sep 27 10:33:55.015533 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:33:55.016755 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3866/7774] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:54 2022] DELETE /identity/v3/projects/cd244e62f2824ffc8a4345fc45c47dc7 => generated 136 bytes in 165 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:33:55.021412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.062064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cccb4-12d3-49c6-a4a1-25ebf5766ea0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.068002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.068986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.069835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.070993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.071160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.074034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.096962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6f6f-849d-4bc8-a317-ada4f94716d4 tempest-ProjectsNegativeTestJSON-1310945182 tempest-ProjectsNegativeTestJSON-1310945182-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.103187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5330141373d44166a2185dbe591da13a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8b9bbbab69148698bddd34616b4c1b3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.103967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.104316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.104522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] PATH_INFO: `/v3/projects/cd244e62f2824ffc8a4345fc45c47dc7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.105975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cd244e62f2824ffc8a4345fc45c47dc7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.108147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.113829 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3909/7775] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:54 2022] POST /identity/v3/projects => generated 401 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:55.118500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.169898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd244e62f2824ffc8a4345fc45c47dc7', 'request_id': 'req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.178687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd244e62f2824ffc8a4345fc45c47dc7', 'request_id': 'req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.188508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd244e62f2824ffc8a4345fc45c47dc7', 'request_id': 'req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.192124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7950-b8be-4948-9b78-c823c058269e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.203801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] The token cache is being invalidate because project cd244e62f2824ffc8a4345fc45c47dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:55.204161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.205073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.207123 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3867/7776] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:55 2022] DELETE /identity/v3/projects/cd244e62f2824ffc8a4345fc45c47dc7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:55.211959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.213813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:55.250097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:55.256805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.257455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.257723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.258031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.258675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.260910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.285244 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3910/7777] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:55 2022] POST /identity/v3/users => generated 307 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:33:55.289625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.291749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:55.298617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9ef32-a5c2-475b-bedd-6c5302f174a4 tempest-ProjectsNegativeTestJSON-312843589 tempest-ProjectsNegativeTestJSON-312843589-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.304599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.305024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3e4a3-f771-443f-9337-85a2bbb4663b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.305284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.305566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.305841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] PATH_INFO: `/v3/users/2e8f50d7ebee4a5fa2d302a53855f1d7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.306972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8f50d7ebee4a5fa2d302a53855f1d7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.309982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.311425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.311627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.312052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.312386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.313172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.314866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.330476 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3911/7778] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:55.331928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.343329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce383-3b25-49fe-9930-fa9ca22e7160 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.349379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.350119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.350570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.350922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] PATH_INFO: `/v3/projects/a51079ec30bb4509b2f53f3e0f13307d/users/88f418de27a042fd8a4bdf81612962df/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.373801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51079ec30bb4509b2f53f3e0f13307d, user_id=88f418de27a042fd8a4bdf81612962df, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.377354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.394301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8f50d7ebee4a5fa2d302a53855f1d7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.395059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8f50d7ebee4a5fa2d302a53855f1d7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.396468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3912/7779] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:55 2022] PUT /identity/v3/projects/a51079ec30bb4509b2f53f3e0f13307d/users/88f418de27a042fd8a4bdf81612962df/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:55.400484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.404065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:55.409662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8f50d7ebee4a5fa2d302a53855f1d7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.422513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:55.435349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8f50d7ebee4a5fa2d302a53855f1d7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.455474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7ca99f04-b6d2-4909-9924-ab044f353f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8f50d7ebee4a5fa2d302a53855f1d7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:55.466501 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3868/7780] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:55 2022] DELETE /identity/v3/users/2e8f50d7ebee4a5fa2d302a53855f1d7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:55.471832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.472124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.472401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.495510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17851e5-862e-4ba2-a28a-a0fe0ab34bb3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.508694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:55.509998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:55.559440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.560452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.560914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.561188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.562079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.563876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.575856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3913/7781] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:55.579323 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3869/7782] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:55.579641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.590254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.590858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a4ccc-2796-4c94-a622-ce9107427f39 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.591661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:55.597095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8eeff-7c4f-46b8-904e-2e3fa48764e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.598484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8eeff-7c4f-46b8-904e-2e3fa48764e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.598777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8eeff-7c4f-46b8-904e-2e3fa48764e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.599548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8eeff-7c4f-46b8-904e-2e3fa48764e0 admin admin] PATH_INFO: `/v3/projects/a51079ec30bb4509b2f53f3e0f13307d/users/88f418de27a042fd8a4bdf81612962df/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.601603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8eeff-7c4f-46b8-904e-2e3fa48764e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51079ec30bb4509b2f53f3e0f13307d, user_id=88f418de27a042fd8a4bdf81612962df, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.604331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8eeff-7c4f-46b8-904e-2e3fa48764e0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.617878 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3914/7783] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:55 2022] PUT /identity/v3/projects/a51079ec30bb4509b2f53f3e0f13307d/users/88f418de27a042fd8a4bdf81612962df/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:55.628849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfad4a-1835-4274-bae9-75ef7b8ba42c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.629230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfad4a-1835-4274-bae9-75ef7b8ba42c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.629611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfad4a-1835-4274-bae9-75ef7b8ba42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.663565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-36dfad4a-1835-4274-bae9-75ef7b8ba42c None None] MFA Rules not processed for user `88f418de27a042fd8a4bdf81612962df`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:55.664573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:55.664893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36dfad4a-1835-4274-bae9-75ef7b8ba42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:55.683027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570aed9-0f61-4fc6-bea7-1fa84a07ec5f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.747142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.747797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.748073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.748341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.756290 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3915/7784] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:55 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:55.761914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfad4a-1835-4274-bae9-75ef7b8ba42c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.771478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfad4a-1835-4274-bae9-75ef7b8ba42c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.776627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88f418de27a042fd8a4bdf81612962df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a51079ec30bb4509b2f53f3e0f13307d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.777350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.777631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.777906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.778589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.780524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.805378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3916/7785] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:33:55 2022] POST /identity/v3/roles => generated 239 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:55.806552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.818883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfbd28-3c30-43e0-9f8d-829f76a65893 tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:55.821302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.823106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.824946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f9468-f02f-4a34-abc9-80327e7bc3df tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88f418de27a042fd8a4bdf81612962df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a51079ec30bb4509b2f53f3e0f13307d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:55.825814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9468-f02f-4a34-abc9-80327e7bc3df tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:55.826172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9468-f02f-4a34-abc9-80327e7bc3df tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:55.826561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9468-f02f-4a34-abc9-80327e7bc3df tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] PATH_INFO: `/v3/projects/0c4391bbbcfa417697dab0922bf87931/users/91717560957a4e68ae31cee8f6af43f3/roles/05cb858ceae744e6b79561db81072ffd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:55.829072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f9468-f02f-4a34-abc9-80327e7bc3df tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: Authorizing `identity:create_grant(project_id=0c4391bbbcfa417697dab0922bf87931, user_id=91717560957a4e68ae31cee8f6af43f3, role_id=05cb858ceae744e6b79561db81072ffd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:55.831959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f9468-f02f-4a34-abc9-80327e7bc3df tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:55.834869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3870/7786] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:33:55.847669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:55.854148 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3917/7787] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:55 2022] PUT /identity/v3/projects/0c4391bbbcfa417697dab0922bf87931/users/91717560957a4e68ae31cee8f6af43f3/roles/05cb858ceae744e6b79561db81072ffd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:55.858280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:55.861747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:55.959664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168920ac-8b32-4516-a19b-f38ce21ac212 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.028282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:56.037618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91717560957a4e68ae31cee8f6af43f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c4391bbbcfa417697dab0922bf87931', 'project_domain_id': 'default', 'roles': ['tempest-test_role-756783222', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:56.038370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.038568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.038852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] PATH_INFO: `/v3/users/91717560957a4e68ae31cee8f6af43f3/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.039556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] RBAC: Authorizing `identity:create_application_credential(user_id=91717560957a4e68ae31cee8f6af43f3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:56.040741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:56.131832 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3871/7788] 10.222.0.26 () {60 vars in 1315 bytes} [Tue Sep 27 10:33:55 2022] POST /identity/v3/users/91717560957a4e68ae31cee8f6af43f3/application_credentials => generated 658 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:56.133621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c139e-0ddb-4150-86a4-62028abd60cf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.133941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c139e-0ddb-4150-86a4-62028abd60cf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.134211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c139e-0ddb-4150-86a4-62028abd60cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.158392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:56.160022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a87c139e-0ddb-4150-86a4-62028abd60cf None None] MFA Rules not processed for user `91717560957a4e68ae31cee8f6af43f3`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:56.160187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.161314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a87c139e-0ddb-4150-86a4-62028abd60cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.232356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d74ff4-214a-46f5-8c3e-10a980554f9c tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.237102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:56.237735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.238014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.238291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] PATH_INFO: `/v3/projects/2a930d05ce814db6be5a3005ab6b9662` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.239411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a930d05ce814db6be5a3005ab6b9662)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:56.241371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:56.288949 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3918/7789] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:56 2022] POST /identity/v3/auth/tokens => generated 3395 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 27 10:33:56.293523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a930d05ce814db6be5a3005ab6b9662', 'request_id': 'req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.293676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c139e-0ddb-4150-86a4-62028abd60cf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:56.300002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a930d05ce814db6be5a3005ab6b9662', 'request_id': 'req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.308095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a930d05ce814db6be5a3005ab6b9662', 'request_id': 'req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.320264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] The token cache is being invalidate because project 2a930d05ce814db6be5a3005ab6b9662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:56.320556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.321261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.322799 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3872/7790] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:56 2022] DELETE /identity/v3/projects/2a930d05ce814db6be5a3005ab6b9662 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:56.327619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:56.329257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.357903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c139e-0ddb-4150-86a4-62028abd60cf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.362776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91717560957a4e68ae31cee8f6af43f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c4391bbbcfa417697dab0922bf87931', 'project_domain_id': 'default', 'roles': ['tempest-test_role-756783222', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:56.363389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:56.363581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.364284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.364565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] PATH_INFO: `/v3/users/91717560957a4e68ae31cee8f6af43f3/application_credentials/c63b975572294f1d8d6e9f329e7d1c1e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.366712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] RBAC: Authorizing `identity:delete_application_credential(user_id=91717560957a4e68ae31cee8f6af43f3, application_credential_id=c63b975572294f1d8d6e9f329e7d1c1e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:56.368601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:56.388858 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3919/7791] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:33:56 2022] DELETE /identity/v3/users/91717560957a4e68ae31cee8f6af43f3/application_credentials/c63b975572294f1d8d6e9f329e7d1c1e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:56.393098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:56.394699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.401197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b72e2-55c1-494b-a35a-c6c7ac8235d2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.406540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:56.407175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.407445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.407710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] PATH_INFO: `/v3/users/5330141373d44166a2185dbe591da13a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.408900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5330141373d44166a2185dbe591da13a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:56.410812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:56.421775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:56.465554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5330141373d44166a2185dbe591da13a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.466172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5330141373d44166a2185dbe591da13a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.466920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d9f26-3e63-4964-8343-f1e03a480ff9 tempest-ApplicationCredentialsV3AdminTest-1758170070 tempest-ApplicationCredentialsV3AdminTest-1758170070-project] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.471521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88f418de27a042fd8a4bdf81612962df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a51079ec30bb4509b2f53f3e0f13307d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:56.472158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.472434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.472752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] PATH_INFO: `/v3/roles/05cb858ceae744e6b79561db81072ffd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.473485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5330141373d44166a2185dbe591da13a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.474115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: Authorizing `identity:delete_role(role_id=05cb858ceae744e6b79561db81072ffd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:56.475755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:56.483552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5330141373d44166a2185dbe591da13a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.496791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5330141373d44166a2185dbe591da13a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.503632 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3873/7792] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:56 2022] DELETE /identity/v3/users/5330141373d44166a2185dbe591da13a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:56.516541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] Invalidating the token cache because role 05cb858ceae744e6b79561db81072ffd has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:56.516853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.517504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-47920594-1695-4977-b3bd-ccfdbcd476fd tempest-ApplicationCredentialsV3AdminTest-893959903 tempest-ApplicationCredentialsV3AdminTest-893959903-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.519308 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3920/7793] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:56 2022] DELETE /identity/v3/roles/05cb858ceae744e6b79561db81072ffd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:56.525164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:56.526782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.610538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e7e1-81b0-4c29-b011-1a8ab24e5f4a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.668976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:56.681088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:56.681936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.682217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.682497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] PATH_INFO: `/v3/users/91717560957a4e68ae31cee8f6af43f3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.684227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91717560957a4e68ae31cee8f6af43f3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:56.686815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:56.752633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91717560957a4e68ae31cee8f6af43f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.753515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91717560957a4e68ae31cee8f6af43f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.762127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91717560957a4e68ae31cee8f6af43f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.776880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91717560957a4e68ae31cee8f6af43f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.794095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91717560957a4e68ae31cee8f6af43f3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:56.801397 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3874/7794] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:56 2022] DELETE /identity/v3/users/91717560957a4e68ae31cee8f6af43f3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:56.806658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.807255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.807874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.836102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:56.836102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.839560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:56.842227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.906838 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3921/7795] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:56.916484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:56.918224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:56.926341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f05b3-72a8-411f-bdef-262011f84e34 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.931768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:56.932412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:56.932715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:56.933057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] PATH_INFO: `/v3/projects/a8b9bbbab69148698bddd34616b4c1b3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:56.934154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b9bbbab69148698bddd34616b4c1b3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:56.936053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:56.986522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0de050-261d-4f2c-a7c4-b6c7d285b6e1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:56.991475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b9bbbab69148698bddd34616b4c1b3', 'request_id': 'req-6fba3162-ee94-4de9-891f-d61b2dc0a019'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.001616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b9bbbab69148698bddd34616b4c1b3', 'request_id': 'req-6fba3162-ee94-4de9-891f-d61b2dc0a019'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.013737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b9bbbab69148698bddd34616b4c1b3', 'request_id': 'req-6fba3162-ee94-4de9-891f-d61b2dc0a019'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.028156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] The token cache is being invalidate because project a8b9bbbab69148698bddd34616b4c1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:57.028583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.029694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6fba3162-ee94-4de9-891f-d61b2dc0a019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.031033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:57.035810 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3875/7796] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:56 2022] DELETE /identity/v3/projects/a8b9bbbab69148698bddd34616b4c1b3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:57.037873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f304-f489-41b7-a2b9-dbd615d68c28 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.038216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f304-f489-41b7-a2b9-dbd615d68c28 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.038553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f304-f489-41b7-a2b9-dbd615d68c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.048167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52eb50e-3463-4a09-ba04-76f8d7c344ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.048867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:57.050346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eb50e-3463-4a09-ba04-76f8d7c344ee service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.050771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eb50e-3463-4a09-ba04-76f8d7c344ee service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.051086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eb50e-3463-4a09-ba04-76f8d7c344ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.053803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e52eb50e-3463-4a09-ba04-76f8d7c344ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.065666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7cf6f304-f489-41b7-a2b9-dbd615d68c28 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:57.066899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7cf6f304-f489-41b7-a2b9-dbd615d68c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.125737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52eb50e-3463-4a09-ba04-76f8d7c344ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.127709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52eb50e-3463-4a09-ba04-76f8d7c344ee service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.139191 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3922/7797] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:33:57.141474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3876/7798] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:57.149392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36493079-6129-4826-8d34-3880590f0565 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.149943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36493079-6129-4826-8d34-3880590f0565 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.150405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36493079-6129-4826-8d34-3880590f0565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.173252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-36493079-6129-4826-8d34-3880590f0565 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:57.174489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36493079-6129-4826-8d34-3880590f0565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.184251 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3923/7799] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:57.188426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f304-f489-41b7-a2b9-dbd615d68c28 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.200707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f304-f489-41b7-a2b9-dbd615d68c28 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.206883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.207575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.207843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.208113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.209117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:57.209117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.211107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.225731 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3877/7800] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:33:57.230226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36493079-6129-4826-8d34-3880590f0565 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.240197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36493079-6129-4826-8d34-3880590f0565 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.245981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.246531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.246816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.247064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.247704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.249658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.280364 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3924/7801] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:57 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:57.284576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.298119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebad50-9ec4-4b54-957d-cc8b8de174b5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.370803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.371631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.371828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.372140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.372817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.374751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.404993 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3878/7802] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:57 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:57.408622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.419842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbab83b-e99e-468c-865c-567911f82c02 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.426240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.426864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.427136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.427458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.428353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.430047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.445447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3925/7803] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:57.446029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.446616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.448345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.458617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df1c7b-60d3-41ac-80d2-7fd44d5ea9ef admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.471935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.472338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.472627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.473187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.473624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.475365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.477489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3ed61-c4c9-4689-adc1-e2123c010848 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.486698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.487604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.488342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.488756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] PATH_INFO: `/v3/projects/0c4391bbbcfa417697dab0922bf87931` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.489954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4391bbbcfa417697dab0922bf87931)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.490440 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3879/7804] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:57.494338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.494685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.506704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f2090-a5ac-482a-bf37-b1b9f02d9828 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.512516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f8ee8-2d7b-4380-a96e-e66bc0257958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.513451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8ee8-2d7b-4380-a96e-e66bc0257958 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.513725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8ee8-2d7b-4380-a96e-e66bc0257958 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.514337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8ee8-2d7b-4380-a96e-e66bc0257958 admin admin] PATH_INFO: `/v3/projects/097adec8b87c4ce5ad9d0d2cd8d5af88/users/84f210eb637d43edb55fd8ad778d9590/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.530804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f8ee8-2d7b-4380-a96e-e66bc0257958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=097adec8b87c4ce5ad9d0d2cd8d5af88, user_id=84f210eb637d43edb55fd8ad778d9590, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.533694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f8ee8-2d7b-4380-a96e-e66bc0257958 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.549338 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3880/7805] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:57 2022] PUT /identity/v3/projects/097adec8b87c4ce5ad9d0d2cd8d5af88/users/84f210eb637d43edb55fd8ad778d9590/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:57.550895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4391bbbcfa417697dab0922bf87931', 'request_id': 'req-64e6c336-d16b-45b7-9548-71ddb0b7ff11'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.561643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.561947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.562173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4391bbbcfa417697dab0922bf87931', 'request_id': 'req-64e6c336-d16b-45b7-9548-71ddb0b7ff11'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.562310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.568724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4391bbbcfa417697dab0922bf87931', 'request_id': 'req-64e6c336-d16b-45b7-9548-71ddb0b7ff11'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.584342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] The token cache is being invalidate because project 0c4391bbbcfa417697dab0922bf87931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:57.584714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.586232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.588468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3926/7806] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:57 2022] DELETE /identity/v3/projects/0c4391bbbcfa417697dab0922bf87931 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:57.593713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.596397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.600020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] MFA Rules not processed for user `84f210eb637d43edb55fd8ad778d9590`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:57.601268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.675188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:57.678628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:57.716799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c336-d16b-45b7-9548-71ddb0b7ff11 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.726291 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3881/7807] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:57.731299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.732906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.815596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.816533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.816944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.817315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] PATH_INFO: `/v3/users/88f418de27a042fd8a4bdf81612962df` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.818708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f418de27a042fd8a4bdf81612962df)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.821100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.832985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fcef8-c445-459e-8812-98ac2e806734 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:57.843242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:57.844045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:57.844323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:57.844763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:57.845410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:57.848757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:57.883672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f418de27a042fd8a4bdf81612962df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.884340 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3882/7808] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:57 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:57.884874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f418de27a042fd8a4bdf81612962df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.889227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:57.891749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:57.895211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f418de27a042fd8a4bdf81612962df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.903554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f418de27a042fd8a4bdf81612962df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.904393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f418de27a042fd8a4bdf81612962df'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:57.923721 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3927/7809] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:33:57 2022] DELETE /identity/v3/users/88f418de27a042fd8a4bdf81612962df => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:57.966266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caea30-268b-425a-aedb-4b9194c23fd1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.026256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.026949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.027226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.027489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.028137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.030209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.042386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:58.058318 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3883/7810] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:57 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:58.061400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.072086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf70ae-b573-41ba-a4a0-9fb63c55dde1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.076861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.077462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.077734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.077999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.078629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.080431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.092827 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3928/7811] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:58.097415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.108492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5664f-2ef7-42e8-8bac-5c15eaac380c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.113927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.114610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.114879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.115153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] PATH_INFO: `/v3/projects/7ea0952f7a244043b1f5ca9b44acc9fa/users/77e492b6063d4c949a2f2116d51ba603/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.131357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea0952f7a244043b1f5ca9b44acc9fa, user_id=77e492b6063d4c949a2f2116d51ba603, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.134076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.149716 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3884/7812] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:58 2022] PUT /identity/v3/projects/7ea0952f7a244043b1f5ca9b44acc9fa/users/77e492b6063d4c949a2f2116d51ba603/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:58.154420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.166507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b10c87-e54f-4824-803f-be8a68b642d2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.229154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:58.243870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.244568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.245324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.245717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.246788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.250531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.261358 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3929/7813] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:58.266050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.277141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724c7e-e84d-490e-be23-98a679cd157f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.282196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890dde4-39de-40a1-a3ad-d788e0e6bc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.282806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890dde4-39de-40a1-a3ad-d788e0e6bc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.283086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890dde4-39de-40a1-a3ad-d788e0e6bc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.283399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890dde4-39de-40a1-a3ad-d788e0e6bc88 admin admin] PATH_INFO: `/v3/projects/7ea0952f7a244043b1f5ca9b44acc9fa/users/77e492b6063d4c949a2f2116d51ba603/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.285344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890dde4-39de-40a1-a3ad-d788e0e6bc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea0952f7a244043b1f5ca9b44acc9fa, user_id=77e492b6063d4c949a2f2116d51ba603, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.288165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890dde4-39de-40a1-a3ad-d788e0e6bc88 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.299672 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3885/7814] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:58 2022] PUT /identity/v3/projects/7ea0952f7a244043b1f5ca9b44acc9fa/users/77e492b6063d4c949a2f2116d51ba603/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:58.368943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.386283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:58.400566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f695dd-dfd3-46e3-b503-6e3ccb0da301 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.400566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f695dd-dfd3-46e3-b503-6e3ccb0da301 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.400566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f695dd-dfd3-46e3-b503-6e3ccb0da301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.415722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006af5-0910-4d33-9b62-f4ade89502d9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.461259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-53f695dd-dfd3-46e3-b503-6e3ccb0da301 None None] MFA Rules not processed for user `77e492b6063d4c949a2f2116d51ba603`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:58.462505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-53f695dd-dfd3-46e3-b503-6e3ccb0da301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:58.498463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.499165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.499441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.499759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] PATH_INFO: `/v3/projects/a51079ec30bb4509b2f53f3e0f13307d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.501188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51079ec30bb4509b2f53f3e0f13307d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.503233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.592995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51079ec30bb4509b2f53f3e0f13307d', 'request_id': 'req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:58.608600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51079ec30bb4509b2f53f3e0f13307d', 'request_id': 'req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:58.618752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51079ec30bb4509b2f53f3e0f13307d', 'request_id': 'req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:58.621111 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3886/7815] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:58.627588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f695dd-dfd3-46e3-b503-6e3ccb0da301 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.638447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f695dd-dfd3-46e3-b503-6e3ccb0da301 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.645948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] The token cache is being invalidate because project a51079ec30bb4509b2f53f3e0f13307d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:33:58.646658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:58.647640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2c6728be-f2ef-4b57-8d3d-e6146efc50a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:33:58.650948 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3930/7816] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:33:58 2022] DELETE /identity/v3/projects/a51079ec30bb4509b2f53f3e0f13307d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:58.657713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b6c5-ca7b-4a23-a70b-0972c412cd69 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.658531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b6c5-ca7b-4a23-a70b-0972c412cd69 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.659193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b6c5-ca7b-4a23-a70b-0972c412cd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.696605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9bf9b6c5-ca7b-4a23-a70b-0972c412cd69 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:58.698416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9bf9b6c5-ca7b-4a23-a70b-0972c412cd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:58.738848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.739985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.740513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.741066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.741969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.743306 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:33:58.743306 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:33:58.743306 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:33:58.744840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.780201 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3887/7817] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:33:58 2022] POST /identity/v3/regions => generated 224 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:58.784211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.788149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:58.818694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181e48f-a748-4f07-b20b-ef2a91e8db81 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.824290 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3931/7818] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:58.832374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.833057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.835265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce2eb-d977-4013-98ae-0c3f2f862bb7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.835432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce2eb-d977-4013-98ae-0c3f2f862bb7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.835701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce2eb-d977-4013-98ae-0c3f2f862bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.837209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.837493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.838503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.845125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.866022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6e4ce2eb-d977-4013-98ae-0c3f2f862bb7 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:58.867384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e4ce2eb-d977-4013-98ae-0c3f2f862bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:58.882845 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3932/7819] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:33:58.884929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce2eb-d977-4013-98ae-0c3f2f862bb7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.892967 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3888/7820] 10.222.0.26 () {60 vars in 1149 bytes} [Tue Sep 27 10:33:58 2022] POST /identity/v3/regions => generated 223 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:58.893170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:58.895064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.902047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce2eb-d977-4013-98ae-0c3f2f862bb7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.927483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:58.929116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.929856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.930263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.930609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.931270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:58.931575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.935833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.953044 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3933/7821] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:58.957484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:58.958232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35811db-b14f-4cec-877d-1f8396e12a93 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.965255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.966627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.967159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.967159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/647e2653-839a-42fd-972a-4099fbdf4bda` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.967859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:create_region(region_id=647e2653-839a-42fd-972a-4099fbdf4bda)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.970164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:58.973608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e801a-5a33-41ea-9f9a-5fca87732d88 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:58.984078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:58.984986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:58.985284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:58.985636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:58.986626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:58.998560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.006343 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3889/7822] 10.222.0.26 () {60 vars in 1259 bytes} [Tue Sep 27 10:33:58 2022] PUT /identity/v3/regions/647e2653-839a-42fd-972a-4099fbdf4bda => generated 232 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:59.010631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.042675 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3934/7823] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:58 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:59.043856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.058176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62012629-6b17-4864-aebe-4af07ce46bea tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.130085 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.131213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.131514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.131792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/647e2653-839a-42fd-972a-4099fbdf4bda` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.132548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:delete_region(region_id=647e2653-839a-42fd-972a-4099fbdf4bda)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.134308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.148809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d7f9-9843-49e2-9173-5e207bef6caa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.173747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.174378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.174526 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3890/7824] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:33:59 2022] DELETE /identity/v3/regions/647e2653-839a-42fd-972a-4099fbdf4bda => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:59.174959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.176192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.177301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.179783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.181956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.218689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3b949-2505-402a-abd6-ee33e7ffd9b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.229061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.229512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.229965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.230300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.230964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.232589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.241066 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3935/7825] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:59 2022] POST /identity/v3/users => generated 284 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:59.244752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.256724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3fb34-9034-411e-a8ce-4409cfda216c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.278256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3891/7826] 10.222.0.26 () {60 vars in 1150 bytes} [Tue Sep 27 10:33:59 2022] POST /identity/v3/regions => generated 254 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:59.279845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.281799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.282513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.282795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.283114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.284061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.288935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.307200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84595f-e179-4483-8450-b085d05ff383 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.307975 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3936/7827] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:33:59.311781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.322206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.324164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.324814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.325324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.326066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:update_region(region_id=dc9c68ecee7048a7b23b8a326bd661ee)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.326348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56acc8-a791-4c9e-9566-5c16cc62626b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.327784 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:33:59.327784 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:33:59.327784 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:33:59.329106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.345754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.346352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.347259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.347592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.348345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.350316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.363519 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3937/7828] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:33:59.367415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.368685 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3892/7829] 10.222.0.26 () {60 vars in 1250 bytes} [Tue Sep 27 10:33:59 2022] PATCH /identity/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee => generated 253 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:59.370823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.384886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62c997-8412-467c-9614-6b45523d3f68 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.409994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2497-931f-4686-8171-ca95bf1c3145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.411706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2497-931f-4686-8171-ca95bf1c3145 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.412211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2497-931f-4686-8171-ca95bf1c3145 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.412628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2497-931f-4686-8171-ca95bf1c3145 admin admin] PATH_INFO: `/v3/projects/b1c7b7929ecf44d88f9ace0cc3415fed/users/54df35a7ab8543588f5a898035db3d62/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.418159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8d785-84d8-4bd2-8254-3b68dc44e2b1 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.425266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.425921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.426201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.426473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.428493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:get_region(region_id=dc9c68ecee7048a7b23b8a326bd661ee)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.429627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.433003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2497-931f-4686-8171-ca95bf1c3145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c7b7929ecf44d88f9ace0cc3415fed, user_id=54df35a7ab8543588f5a898035db3d62, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.436394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2497-931f-4686-8171-ca95bf1c3145 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.438234 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3893/7830] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:59 2022] GET /identity/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee => generated 253 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:33:59.442276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.453669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fda7ef-af43-4e88-b598-f324e891daf7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.457232 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3938/7831] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:33:59 2022] PUT /identity/v3/projects/b1c7b7929ecf44d88f9ace0cc3415fed/users/54df35a7ab8543588f5a898035db3d62/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:33:59.468394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bef1e-e857-4bc0-8ea6-02d1ccab6526 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.469398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bef1e-e857-4bc0-8ea6-02d1ccab6526 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.470924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bef1e-e857-4bc0-8ea6-02d1ccab6526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.503869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c94bef1e-e857-4bc0-8ea6-02d1ccab6526 None None] MFA Rules not processed for user `54df35a7ab8543588f5a898035db3d62`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:33:59.505179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c94bef1e-e857-4bc0-8ea6-02d1ccab6526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:33:59.505796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:59.524425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:59.531889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.532971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.533401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.533763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.535114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:delete_region(region_id=dc9c68ecee7048a7b23b8a326bd661ee)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.536570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.578536 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3894/7832] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:33:59 2022] DELETE /identity/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:59.582515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.621106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6a0e-eae4-4065-a6d7-8179a89c92bb tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.626745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.627678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.629080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.629080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.629694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.630919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.638407 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3895/7833] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 10:33:59 2022] GET /identity/v3/regions => generated 669 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:59.642687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.650260 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3939/7834] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:33:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:33:59.655645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bef1e-e857-4bc0-8ea6-02d1ccab6526 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.662329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b44e48-59e7-4888-98da-8ac64535088d tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.668012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.668740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.669026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.669302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.669901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:delete_region(region_id=dc9c68ecee7048a7b23b8a326bd661ee)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.671552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Could not find region: dc9c68ecee7048a7b23b8a326bd661ee.: keystone.exception.RegionNotFound: Could not find region: dc9c68ecee7048a7b23b8a326bd661ee. Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: dc9c68ecee7048a7b23b8a326bd661ee. Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:33:59.688086 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: dc9c68ecee7048a7b23b8a326bd661ee. Sep 27 10:33:59.689736 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:33:59.691748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3896/7835] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:33:59 2022] DELETE /identity/v3/regions/dc9c68ecee7048a7b23b8a326bd661ee => generated 112 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:33:59.698567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.712483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388d610-912b-4429-89b1-cb97d1a2af5c tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.719902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.720631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.721081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.721355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.722439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.723572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.730847 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3897/7836] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 10:33:59 2022] GET /identity/v3/regions => generated 669 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:59.737522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.756526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba7b30-6ea0-4b62-a5fb-f11fcf659cec tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.766327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.766991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.767687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.767687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.770019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.771483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.791195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bef1e-e857-4bc0-8ea6-02d1ccab6526 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.797544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.798165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.798558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.798831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.799539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.801796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.806536 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3898/7837] 10.222.0.26 () {60 vars in 1150 bytes} [Tue Sep 27 10:33:59 2022] POST /identity/v3/regions => generated 253 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:59.807983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.835234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c7289-2605-49fe-af3f-b02849fdce16 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.844124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.845211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.845619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.846033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.847484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Adding query filter params (parent_region_id=e550016207054a528bac8143587b90fd) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:33:59.847955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.850101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.857108 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3940/7838] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:33:59 2022] POST /identity/v3/projects => generated 369 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:33:59.859969 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3899/7839] 10.222.0.26 () {58 vars in 1227 bytes} [Tue Sep 27 10:33:59 2022] GET /identity/v3/regions?parent_region_id=e550016207054a528bac8143587b90fd => generated 399 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:33:59.860162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.864891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:33:59.878713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fda34-f024-4889-b37e-4e0b0b76d7bc tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.933938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1480e-1c11-4364-b5c2-84d6651e664b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:33:59.951069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.952213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.952724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.953187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/e97bbb517e084445a0370fabd31b6867` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.953956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:delete_region(region_id=e97bbb517e084445a0370fabd31b6867)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.956000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.963390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:33:59.965002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:33:59.965380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:33:59.965711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:33:59.967343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:33:59.970181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:33:59.985129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3900/7840] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:33:59 2022] DELETE /identity/v3/regions/e97bbb517e084445a0370fabd31b6867 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:33:59.989755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.005149 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3941/7841] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:33:59 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:00.006315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.012584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f620fd5-115d-4aa6-90e9-568eedae240a tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.019935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.019935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.019935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.020343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ec306-8a4e-4581-b68f-0ead7de49d4d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.020485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/0dd86653a9a341df9900a9dbc3d71906` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.020485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:delete_region(region_id=0dd86653a9a341df9900a9dbc3d71906)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.021525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.041505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.042709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.043140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.043531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.044391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.046500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.048156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:00.048560 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3901/7842] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:33:59 2022] DELETE /identity/v3/regions/0dd86653a9a341df9900a9dbc3d71906 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:00.053565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.068355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:00.068843 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3942/7843] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:00.071986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.085256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be615-2840-474f-aaf6-1a181a91b96e tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.092088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107049e-b87f-4303-b387-cd611284b482 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.092817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e492b6063d4c949a2f2116d51ba603', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ea0952f7a244043b1f5ca9b44acc9fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.093761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.094554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.094961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] PATH_INFO: `/v3/regions/e550016207054a528bac8143587b90fd` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.095832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorizing `identity:delete_region(region_id=e550016207054a528bac8143587b90fd)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.101830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.116961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.117727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.118140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.118339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] PATH_INFO: `/v3/projects/a7fc0004ff7f42db8fbdd93c0fc80eac/users/fef31e12e91d4a59ad54d1ba987d62ae/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.134068 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3902/7844] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:34:00 2022] DELETE /identity/v3/regions/e550016207054a528bac8143587b90fd => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:00.139649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.141440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7fc0004ff7f42db8fbdd93c0fc80eac, user_id=fef31e12e91d4a59ad54d1ba987d62ae, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.142402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:00.144315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.162144 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3943/7845] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:00 2022] PUT /identity/v3/projects/a7fc0004ff7f42db8fbdd93c0fc80eac/users/fef31e12e91d4a59ad54d1ba987d62ae/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:00.166406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.178650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96a4c-7f87-4ba1-afd0-1d2a1b4dfce7 tempest-RegionsTestJSON-1292587720 tempest-RegionsTestJSON-1292587720-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.252302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56e1ae-b886-47a5-a927-451e76e20296 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.257448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.258298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.258736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.259163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.259610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.260261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.260417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.260539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.261054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] PATH_INFO: `/v3/users/84f210eb637d43edb55fd8ad778d9590` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.262145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.262362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f210eb637d43edb55fd8ad778d9590)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.264298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.280522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3944/7846] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:00.281686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.296330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82859fb3-c2f1-4a2b-bc2f-d474d88ff187 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.303473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb12cf0-5186-4f2b-8237-0bbc7b608a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.304156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb12cf0-5186-4f2b-8237-0bbc7b608a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.304409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb12cf0-5186-4f2b-8237-0bbc7b608a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.304771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb12cf0-5186-4f2b-8237-0bbc7b608a2c admin admin] PATH_INFO: `/v3/projects/a7fc0004ff7f42db8fbdd93c0fc80eac/users/fef31e12e91d4a59ad54d1ba987d62ae/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.310320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb12cf0-5186-4f2b-8237-0bbc7b608a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7fc0004ff7f42db8fbdd93c0fc80eac, user_id=fef31e12e91d4a59ad54d1ba987d62ae, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.315087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb12cf0-5186-4f2b-8237-0bbc7b608a2c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.331485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f210eb637d43edb55fd8ad778d9590'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:00.333487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f210eb637d43edb55fd8ad778d9590'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:00.335443 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3945/7847] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:00 2022] PUT /identity/v3/projects/a7fc0004ff7f42db8fbdd93c0fc80eac/users/fef31e12e91d4a59ad54d1ba987d62ae/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:00.340817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f210eb637d43edb55fd8ad778d9590'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:00.347517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0faf7f-2186-4172-9e17-45d47d450c33 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.347800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0faf7f-2186-4172-9e17-45d47d450c33 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.348070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0faf7f-2186-4172-9e17-45d47d450c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.352318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f210eb637d43edb55fd8ad778d9590'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:00.369400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3af38e72-7822-42ee-9053-8b82dcd695e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f210eb637d43edb55fd8ad778d9590'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:00.375649 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3903/7848] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:00 2022] DELETE /identity/v3/users/84f210eb637d43edb55fd8ad778d9590 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:00.380812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.381136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.381537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.382355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ea0faf7f-2186-4172-9e17-45d47d450c33 None None] MFA Rules not processed for user `fef31e12e91d4a59ad54d1ba987d62ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:00.383723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea0faf7f-2186-4172-9e17-45d47d450c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:00.404597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:00.406365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:00.470708 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3946/7849] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:00.475049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:00.475768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0faf7f-2186-4172-9e17-45d47d450c33 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.486808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0faf7f-2186-4172-9e17-45d47d450c33 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.492779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.493411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.493692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.494076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.495554 ncdv-master-1 devstack@keystone.service[104132]: WARNING oslo_log.versionutils [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 27 10:34:00.495896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.497182 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:00.497182 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:00.497182 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:00.498093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.498647 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3904/7850] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:00.508719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.510212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:00.523318 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3947/7851] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:00 2022] POST /identity/v3/policies => generated 228 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:00.528086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.546066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacc2f6-c819-4ada-90f9-3239837d0394 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.553823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.554487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.554767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.555047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies/d47fe7a617d74ca498a5a48c4e42ba41` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.557034 ncdv-master-1 devstack@keystone.service[104132]: WARNING oslo_log.versionutils [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 27 10:34:00.557573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=d47fe7a617d74ca498a5a48c4e42ba41)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.559794 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:00.559794 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:00.559794 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:00.564976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.599472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3948/7852] 10.222.0.26 () {60 vars in 1252 bytes} [Tue Sep 27 10:34:00 2022] PATCH /identity/v3/policies/d47fe7a617d74ca498a5a48c4e42ba41 => generated 235 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:00.608012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.616789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:00.618035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dc28f-2102-4f9d-84a7-993fdf27b43a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.622412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.623246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.623690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.624133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a9eb4-1651-47d1-b1a7-d2e9aedab1cc tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.624320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.631940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.632739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.633044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.633393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies/d47fe7a617d74ca498a5a48c4e42ba41` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.634227 ncdv-master-1 devstack@keystone.service[104132]: WARNING oslo_log.versionutils [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 27 10:34:00.635055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=d47fe7a617d74ca498a5a48c4e42ba41)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.636105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.637174 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:00.637174 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:00.637174 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:00.638176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.638515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.648051 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3949/7853] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:34:00 2022] GET /identity/v3/policies/d47fe7a617d74ca498a5a48c4e42ba41 => generated 235 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:00.652388 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3905/7854] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:00.657165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.668886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a993b4-511a-4ddf-b24f-32a352c87d12 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.677912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.678670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.679019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.679401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies/d47fe7a617d74ca498a5a48c4e42ba41` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.680363 ncdv-master-1 devstack@keystone.service[104132]: WARNING oslo_log.versionutils [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 27 10:34:00.680958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=d47fe7a617d74ca498a5a48c4e42ba41)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.682275 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:00.682275 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:00.682275 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:00.683668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.717726 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3950/7855] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:00 2022] DELETE /identity/v3/policies/d47fe7a617d74ca498a5a48c4e42ba41 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:00.723847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.744697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59779-8842-4fa4-ae59-857e614fe875 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.753994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.755203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.755203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.755203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.755887 ncdv-master-1 devstack@keystone.service[104131]: WARNING oslo_log.versionutils [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 27 10:34:00.756396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.758038 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:00.758038 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:34:00.758038 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:34:00.758882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.785599 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3906/7856] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:00 2022] POST /identity/v3/policies => generated 230 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:00.789951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.802019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589a7fc-326c-49f6-a1ee-a8a061c99619 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.807711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.808038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.808325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.808599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.809759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.811580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.844412 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3951/7857] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:00 2022] POST /identity/v3/policies => generated 229 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:00.844951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.862536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f64642-155b-45b7-a3d2-0017863590c9 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.868229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.869105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.869488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.869868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.871456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.873617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.927708 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3907/7858] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:00 2022] POST /identity/v3/policies => generated 230 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:00.928573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.942086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b83b-f076-42ba-8d85-9bc78b159a2c tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.947144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.947605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.947892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.948215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.949467 ncdv-master-1 devstack@keystone.service[104132]: WARNING oslo_log.versionutils [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 27 10:34:00.950013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.951302 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:00.951302 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:00.951302 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:00.952207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:00.966541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3952/7859] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:34:00 2022] GET /identity/v3/policies => generated 765 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:00.970914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:00.983705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db7109-2f4c-44dd-971c-3694a84bace4 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:00.993340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:00.994095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:00.995471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:00.995867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies/c151bfa8840440cd97632b9c47d772a5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:00.997026 ncdv-master-1 devstack@keystone.service[104131]: WARNING oslo_log.versionutils [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 27 10:34:00.997808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=c151bfa8840440cd97632b9c47d772a5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:00.999302 ncdv-master-1 devstack@keystone.service[104131]: WARNING py.warnings [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:00.999302 ncdv-master-1 devstack@keystone.service[104131]: warnings.warn(msg) Sep 27 10:34:00.999302 ncdv-master-1 devstack@keystone.service[104131]: Sep 27 10:34:01.000316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.023100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:01.025734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:01.056999 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3908/7860] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:00 2022] DELETE /identity/v3/policies/c151bfa8840440cd97632b9c47d772a5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:01.061598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:01.077652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b17e13-b983-4d9d-9514-d9005364ede0 tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:01.084823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf965c7-4a0e-4ebd-b71c-d88a255bca4b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:01.091111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:01.091913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.092194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.092462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] PATH_INFO: `/v3/projects/097adec8b87c4ce5ad9d0d2cd8d5af88` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.095865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=097adec8b87c4ce5ad9d0d2cd8d5af88)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:01.099204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.100071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:01.101078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.101702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.103060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies/5057b3bb92ce465eaae2af9e1904bcf0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.104420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=5057b3bb92ce465eaae2af9e1904bcf0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:01.107429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.139491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3909/7861] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:01 2022] DELETE /identity/v3/policies/5057b3bb92ce465eaae2af9e1904bcf0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:01.144958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:01.167550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db108c-d56d-4692-8012-1bafbed2756a tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:01.173798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef31e12e91d4a59ad54d1ba987d62ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:01.174728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.175456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.176215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] PATH_INFO: `/v3/policies/58de1bc4a7bb4f1e9ff848f5a0d821e3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.177501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=58de1bc4a7bb4f1e9ff848f5a0d821e3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:01.180043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.189116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097adec8b87c4ce5ad9d0d2cd8d5af88', 'request_id': 'req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.203208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097adec8b87c4ce5ad9d0d2cd8d5af88', 'request_id': 'req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.211130 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3910/7862] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:01 2022] DELETE /identity/v3/policies/58de1bc4a7bb4f1e9ff848f5a0d821e3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:01.211997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097adec8b87c4ce5ad9d0d2cd8d5af88', 'request_id': 'req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.213014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:01.220756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:01.227615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] The token cache is being invalidate because project 097adec8b87c4ce5ad9d0d2cd8d5af88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:01.228007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.228762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.232142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3953/7863] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:01 2022] DELETE /identity/v3/projects/097adec8b87c4ce5ad9d0d2cd8d5af88 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:01.236552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:01.237988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:01.239622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166fabc-f699-4514-9254-78fd992d5e8b tempest-PoliciesTestJSON-1855520129 tempest-PoliciesTestJSON-1855520129-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:01.310319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:01.311086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.311442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.311779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] PATH_INFO: `/v3/users/54df35a7ab8543588f5a898035db3d62` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.312961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54df35a7ab8543588f5a898035db3d62)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:01.314910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.318785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5cb7f-2c40-4ba9-95cb-201aea9dc52f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:01.325466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:01.326194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.326563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.327114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] PATH_INFO: `/v3/users/77e492b6063d4c949a2f2116d51ba603` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.328011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e492b6063d4c949a2f2116d51ba603)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:01.330264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.330417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:01.381615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54df35a7ab8543588f5a898035db3d62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.382243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54df35a7ab8543588f5a898035db3d62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.390514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:01.390731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54df35a7ab8543588f5a898035db3d62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.397984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e492b6063d4c949a2f2116d51ba603'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.398812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e492b6063d4c949a2f2116d51ba603'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.404981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54df35a7ab8543588f5a898035db3d62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.408526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e492b6063d4c949a2f2116d51ba603'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.418452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e492b6063d4c949a2f2116d51ba603'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.418452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e492b6063d4c949a2f2116d51ba603'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.423330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7d124993-039f-43d5-a459-69d8d4cec927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54df35a7ab8543588f5a898035db3d62'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.433594 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3911/7864] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:01 2022] DELETE /identity/v3/users/54df35a7ab8543588f5a898035db3d62 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:01.436400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.436671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.437015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.439944 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3954/7865] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:01 2022] DELETE /identity/v3/users/77e492b6063d4c949a2f2116d51ba603 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:01.457818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:01.458957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:01.528684 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3912/7866] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:01.544034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:01.545648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:01.612701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dd3fc-3744-4c89-a7ab-137c67d6e9ab admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:01.617676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c67dd-2b32-4b5d-98eb-285687a43219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:01.618315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c67dd-2b32-4b5d-98eb-285687a43219 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.618664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c67dd-2b32-4b5d-98eb-285687a43219 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.618998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c67dd-2b32-4b5d-98eb-285687a43219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.629524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c67dd-2b32-4b5d-98eb-285687a43219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:01.631240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c67dd-2b32-4b5d-98eb-285687a43219 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.642169 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3955/7867] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:01.812191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:01.813941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:01.829817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96134ae8-f355-4a59-bbf2-2e51dd88f71b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:01.835133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:01.835798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.836264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.836589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] PATH_INFO: `/v3/projects/7ea0952f7a244043b1f5ca9b44acc9fa` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.838297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea0952f7a244043b1f5ca9b44acc9fa)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:01.840458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:01.901845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea0952f7a244043b1f5ca9b44acc9fa', 'request_id': 'req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.908801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea0952f7a244043b1f5ca9b44acc9fa', 'request_id': 'req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.918941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea0952f7a244043b1f5ca9b44acc9fa', 'request_id': 'req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.931110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] The token cache is being invalidate because project 7ea0952f7a244043b1f5ca9b44acc9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:01.931490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.932225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f3bc0ba4-8ac1-4514-89c6-a4e243b8c667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:01.934041 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3913/7868] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:01 2022] DELETE /identity/v3/projects/7ea0952f7a244043b1f5ca9b44acc9fa => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:01.939117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09221490-db34-4939-89f4-01cdd7e682b0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:01.939617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09221490-db34-4939-89f4-01cdd7e682b0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:01.939714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09221490-db34-4939-89f4-01cdd7e682b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:01.962986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-09221490-db34-4939-89f4-01cdd7e682b0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:01.965025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-09221490-db34-4939-89f4-01cdd7e682b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:02.070943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3956/7869] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:02.072850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.073400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.073400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.101008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:02.102261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:02.112102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3914/7870] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:02.116517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09221490-db34-4939-89f4-01cdd7e682b0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.128232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.130330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:02.132677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09221490-db34-4939-89f4-01cdd7e682b0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.139375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.140724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.141283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.141725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.142509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:02.142953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.153059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.162221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27203269-566b-451d-beee-bdcd31245cd4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.174011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.174696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.175022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.175296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] PATH_INFO: `/v3/projects/b1c7b7929ecf44d88f9ace0cc3415fed` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.176700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c7b7929ecf44d88f9ace0cc3415fed)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.178941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.183065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:02.190441 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3957/7871] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:02.191797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.218804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ec0a7-9d68-42de-8010-242750886c7e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.226162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.227755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.228397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.229019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.230139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.232198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.269715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c7b7929ecf44d88f9ace0cc3415fed', 'request_id': 'req-ccca4530-230a-4d5c-9e9a-f4127e1a4139'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.277281 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3958/7872] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:02 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:02.281523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c7b7929ecf44d88f9ace0cc3415fed', 'request_id': 'req-ccca4530-230a-4d5c-9e9a-f4127e1a4139'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.283463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.294097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c7b7929ecf44d88f9ace0cc3415fed', 'request_id': 'req-ccca4530-230a-4d5c-9e9a-f4127e1a4139'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.315954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] The token cache is being invalidate because project b1c7b7929ecf44d88f9ace0cc3415fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:02.316342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.318070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.324352 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3915/7873] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:02 2022] DELETE /identity/v3/projects/b1c7b7929ecf44d88f9ace0cc3415fed => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:02.326360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.328121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:02.330610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:02.385146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f46de-fe18-4cf3-9bde-e81fff423888 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.399326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.401724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.402013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.402287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.404956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.407429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.422359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4530-230a-4d5c-9e9a-f4127e1a4139 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.430007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.431103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.431673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.432188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] PATH_INFO: `/v3/users/fef31e12e91d4a59ad54d1ba987d62ae` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.434118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef31e12e91d4a59ad54d1ba987d62ae)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.437163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.444294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3959/7874] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:02 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:02.446022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.448498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:02.465162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05671-3e85-496a-bb30-25aa687d45b0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.470394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.471286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.471558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.471820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.472468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.474187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.497031 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3960/7875] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:02.499829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.514630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553581c-9aad-4246-895b-e9e665864f0f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.519145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef31e12e91d4a59ad54d1ba987d62ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.519877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef31e12e91d4a59ad54d1ba987d62ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.522620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.523254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.523596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.523876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.524531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.526568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.529698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef31e12e91d4a59ad54d1ba987d62ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.541407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef31e12e91d4a59ad54d1ba987d62ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.545629 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3961/7876] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:02.547165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.549890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:02.569113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744221-1d7d-44f6-b995-44a94af01d56 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.571242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d5d173f2-2382-42cb-897d-bca7ab605e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef31e12e91d4a59ad54d1ba987d62ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:02.580954 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3916/7877] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:02 2022] DELETE /identity/v3/users/fef31e12e91d4a59ad54d1ba987d62ae => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:02.587929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.588718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.589277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.589742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] PATH_INFO: `/v3/projects/f5e53e6ab8414fd6b3784651dfc6dad7/users/2deac36d6e0e4d1e87ea05082bdb8e23/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.613014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e53e6ab8414fd6b3784651dfc6dad7, user_id=2deac36d6e0e4d1e87ea05082bdb8e23, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.616809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.636462 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3962/7878] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:02 2022] PUT /identity/v3/projects/f5e53e6ab8414fd6b3784651dfc6dad7/users/2deac36d6e0e4d1e87ea05082bdb8e23/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:02.649261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b8dce-a3b9-4a9b-bbca-0dfaec6afb83 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.649546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b8dce-a3b9-4a9b-bbca-0dfaec6afb83 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.649851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b8dce-a3b9-4a9b-bbca-0dfaec6afb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.677748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:02.693131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9b2b8dce-a3b9-4a9b-bbca-0dfaec6afb83 None None] MFA Rules not processed for user `2deac36d6e0e4d1e87ea05082bdb8e23`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:02.694956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b2b8dce-a3b9-4a9b-bbca-0dfaec6afb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:02.770978 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3917/7879] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:02.775825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.840152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d193f28-94ea-4355-8518-ce5683439abe admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.845094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.845710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.846191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.846332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.846880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.848829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.879075 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3963/7880] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:02 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:02.883455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b8dce-a3b9-4a9b-bbca-0dfaec6afb83 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:02.895363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b8dce-a3b9-4a9b-bbca-0dfaec6afb83 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:02.958585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:02.958865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:02.959157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:02.959375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:02.960130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:02.962707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:02.988610 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3918/7881] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:02 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:02.992095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.004888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b359e-d10f-4329-8e87-0296f70dad7b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.012097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.012746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.013026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.013296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.013938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.015567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.029106 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3964/7882] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:03.034123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.044986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e1da2-eccd-41e1-ad89-f9f70daca633 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.050561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.051230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.051551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.051826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] PATH_INFO: `/v3/projects/cc1d43b1a2854f0b85a23b9b2a872df6/users/f726b326b78b4f67ba767b0554da3ccc/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.069668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1d43b1a2854f0b85a23b9b2a872df6, user_id=f726b326b78b4f67ba767b0554da3ccc, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.072667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.091674 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3919/7883] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:03 2022] PUT /identity/v3/projects/cc1d43b1a2854f0b85a23b9b2a872df6/users/f726b326b78b4f67ba767b0554da3ccc/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:03.096093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.107442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ade7b-ca5c-4d2f-84d5-a426a9988919 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.154805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:03.187604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca115f-a182-4fe1-8a3f-4e7ac7da1b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.188272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca115f-a182-4fe1-8a3f-4e7ac7da1b1e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.188549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca115f-a182-4fe1-8a3f-4e7ac7da1b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.188853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca115f-a182-4fe1-8a3f-4e7ac7da1b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.189527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca115f-a182-4fe1-8a3f-4e7ac7da1b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.191190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca115f-a182-4fe1-8a3f-4e7ac7da1b1e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.206259 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3965/7884] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:03.207550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.220077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b3a64-89a1-40d2-8f50-881f77c4bd40 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.225251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.225773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.226146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.226491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] PATH_INFO: `/v3/projects/cc1d43b1a2854f0b85a23b9b2a872df6/users/f726b326b78b4f67ba767b0554da3ccc/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.229213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1d43b1a2854f0b85a23b9b2a872df6, user_id=f726b326b78b4f67ba767b0554da3ccc, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.232127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.245549 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3920/7885] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:03 2022] PUT /identity/v3/projects/cc1d43b1a2854f0b85a23b9b2a872df6/users/f726b326b78b4f67ba767b0554da3ccc/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:03.256146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9428f6-6357-404c-adc4-5ba8975a4565 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.256354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9428f6-6357-404c-adc4-5ba8975a4565 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.256749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9428f6-6357-404c-adc4-5ba8975a4565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.288113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6b9428f6-6357-404c-adc4-5ba8975a4565 None None] MFA Rules not processed for user `f726b326b78b4f67ba767b0554da3ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:03.289499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6b9428f6-6357-404c-adc4-5ba8975a4565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:03.313283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.315322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:03.387089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:03.397418 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3966/7886] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:03.403167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9428f6-6357-404c-adc4-5ba8975a4565 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.404617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc9c09-b16f-4d47-8d87-c35905adc6f2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.410478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.411412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.411705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.412132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] PATH_INFO: `/v3/projects/a7fc0004ff7f42db8fbdd93c0fc80eac` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.413351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7fc0004ff7f42db8fbdd93c0fc80eac)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.415290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.415542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9428f6-6357-404c-adc4-5ba8975a4565 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.421641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.422209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.422488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.422823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.424093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.426398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.446806 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3967/7887] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:03 2022] POST /identity/v3/services => generated 240 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:03.451468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.473560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'request_id': 'req-a4d0f0cb-aef4-4085-bc45-8146604cdb03'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:03.476741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edfdc0-8972-4cec-b35a-5d9bbcd88b7b tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.481736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'request_id': 'req-a4d0f0cb-aef4-4085-bc45-8146604cdb03'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:03.493301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7fc0004ff7f42db8fbdd93c0fc80eac', 'request_id': 'req-a4d0f0cb-aef4-4085-bc45-8146604cdb03'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:03.508792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] The token cache is being invalidate because project a7fc0004ff7f42db8fbdd93c0fc80eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:03.509095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:03.510030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a4d0f0cb-aef4-4085-bc45-8146604cdb03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:03.512072 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3921/7888] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:03 2022] DELETE /identity/v3/projects/a7fc0004ff7f42db8fbdd93c0fc80eac => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:03.517950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e12db-dbbc-4908-b9d4-2e3b3be3b975 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.518320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e12db-dbbc-4908-b9d4-2e3b3be3b975 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.518662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e12db-dbbc-4908-b9d4-2e3b3be3b975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.540515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-783e12db-dbbc-4908-b9d4-2e3b3be3b975 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:03.541894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-783e12db-dbbc-4908-b9d4-2e3b3be3b975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:03.555157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.556217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.556839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.557346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/69c3596566c147d1a9363be560df7143` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.558212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:delete_service(service_id=69c3596566c147d1a9363be560df7143)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.560351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.588716 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3968/7889] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:03 2022] DELETE /identity/v3/services/69c3596566c147d1a9363be560df7143 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:03.594301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.597029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:03.633361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15492235-47aa-418d-8588-f1579e8c3c7e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.638202 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3922/7890] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:03.648024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a6c2-7ec9-4bc3-b2de-568c038b4ad5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.648463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a6c2-7ec9-4bc3-b2de-568c038b4ad5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.648778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.648956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a6c2-7ec9-4bc3-b2de-568c038b4ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.649437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.649799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.650078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.650748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.652479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.674077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bc63a6c2-7ec9-4bc3-b2de-568c038b4ad5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:03.675304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bc63a6c2-7ec9-4bc3-b2de-568c038b4ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:03.677254 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3969/7891] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:03 2022] POST /identity/v3/services => generated 290 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:03.678776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.689460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:03.701741 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3923/7892] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:03 2022] POST /identity/v3/auth/tokens => generated 3373 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:03.706760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a6c2-7ec9-4bc3-b2de-568c038b4ad5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.718134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a6c2-7ec9-4bc3-b2de-568c038b4ad5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.722323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc1f5-8831-47bc-b100-aa262b2abe73 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.723839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.724463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.724977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.725265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.725938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:03.726213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.727666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.727857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.728432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.728835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.729342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/2303b61aec8b4b4d960ca35cff4b3669` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.730061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:update_service(service_id=2303b61aec8b4b4d960ca35cff4b3669)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.731283 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:03.731283 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:03.731283 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:03.732150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.740992 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3924/7893] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:03.745840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.754468 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3970/7894] 10.222.0.26 () {60 vars in 1252 bytes} [Tue Sep 27 10:34:03 2022] PATCH /identity/v3/services/2303b61aec8b4b4d960ca35cff4b3669 => generated 284 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:03.759682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.760222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab4aba-1065-47fa-aedd-516fc79dc20d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.777512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.778143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.778461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.778683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.779329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.781458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.801731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85224ec6-760b-47ad-a8f6-a7c630a25d4f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.807646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.808527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.809068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.810316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/2303b61aec8b4b4d960ca35cff4b3669` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.811027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:get_service(service_id=2303b61aec8b4b4d960ca35cff4b3669)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.812746 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:03.812746 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:03.812746 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:03.815201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.822529 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3925/7895] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:03 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:03.827743 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3971/7896] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:34:03 2022] GET /identity/v3/services/2303b61aec8b4b4d960ca35cff4b3669 => generated 284 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:03.828116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.831024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.851104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:03.862415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea2cf6-8ef5-4191-8015-89f09f690211 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.927261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ee2c-f067-4106-89a6-31a08171bd7a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:03.933891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.934582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.934866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.935139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.935784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.938451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:03.974819 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3926/7897] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:03 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:03.981653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:03.982291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:03.983209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:03.983624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:03.984042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/2303b61aec8b4b4d960ca35cff4b3669` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:03.985020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:delete_service(service_id=2303b61aec8b4b4d960ca35cff4b3669)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:03.986908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.002293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f070a50-8c46-47e9-9bad-ebb9e103f94c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.025711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.028178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.028178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.028596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.033444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.044858 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3972/7898] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:03 2022] DELETE /identity/v3/services/2303b61aec8b4b4d960ca35cff4b3669 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:04.046738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.047492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.081237 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3927/7899] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:04.082378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.097811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63388b6-46f8-4cd1-85f7-29fc46d1e897 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.102465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cce8c6-a51b-4bbf-9af5-250044a4d3e9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.108779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.109445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.110454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.110729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/2303b61aec8b4b4d960ca35cff4b3669` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.111342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:get_service(service_id=2303b61aec8b4b4d960ca35cff4b3669)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.113762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Could not find service: 2303b61aec8b4b4d960ca35cff4b3669.: keystone.exception.ServiceNotFound: Could not find service: 2303b61aec8b4b4d960ca35cff4b3669. Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 2303b61aec8b4b4d960ca35cff4b3669. Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:04.139126 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:34:04.140475 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 2303b61aec8b4b4d960ca35cff4b3669. Sep 27 10:34:04.142017 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:04.143278 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3973/7900] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/services/2303b61aec8b4b4d960ca35cff4b3669 => generated 113 bytes in 97 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:04.147561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.148200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.148489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.148897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.149569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.151382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.156394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.174822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa6ff-431d-4e3f-80cb-7b9179ad4d14 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.180761 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3928/7901] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:04.185736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.194566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.195194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.195472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.195744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.196393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.198378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.207600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c6ad-ebdc-4a6d-8bcb-fbb1b32afc57 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.218906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c2dd8-cc1a-4b67-9edc-969ad1bfa70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.219494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2dd8-cc1a-4b67-9edc-969ad1bfa70f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.219794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2dd8-cc1a-4b67-9edc-969ad1bfa70f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.220068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2dd8-cc1a-4b67-9edc-969ad1bfa70f admin admin] PATH_INFO: `/v3/projects/99b5b5bd737b496ca217aa72243ae844/users/7e426f84d0a54d85bc0a80ad321fd309/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.238943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3974/7902] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:04 2022] POST /identity/v3/services => generated 273 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:04.243368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.249483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c2dd8-cc1a-4b67-9edc-969ad1bfa70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b5b5bd737b496ca217aa72243ae844, user_id=7e426f84d0a54d85bc0a80ad321fd309, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.252218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c2dd8-cc1a-4b67-9edc-969ad1bfa70f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.288030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3929/7903] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:04 2022] PUT /identity/v3/projects/99b5b5bd737b496ca217aa72243ae844/users/7e426f84d0a54d85bc0a80ad321fd309/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:04.299658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44f62-8467-4327-b826-518512cd6b4a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.299954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44f62-8467-4327-b826-518512cd6b4a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.300233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a44f62-8467-4327-b826-518512cd6b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.301904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c3ba-2490-4262-a7d2-e67b034a6b9f tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.368161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-50a44f62-8467-4327-b826-518512cd6b4a None None] MFA Rules not processed for user `7e426f84d0a54d85bc0a80ad321fd309`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:04.369653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50a44f62-8467-4327-b826-518512cd6b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:04.392139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.393002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.393277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.394120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.395104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.397624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.412278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:DELETE=2,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:04.423905 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3975/7904] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:04 2022] POST /identity/v3/services => generated 272 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:04.428924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.457811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d90e1-b67a-4a00-bd4c-63ec809f3cad tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.464303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.464957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.465339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.465613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.466263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.467968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.491850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3976/7905] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:04 2022] POST /identity/v3/services => generated 274 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:04.497168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.512932 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3930/7906] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:04 2022] POST /identity/v3/auth/tokens => generated 3744 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:04.515885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44f62-8467-4327-b826-518512cd6b4a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.531427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d4644-9f43-453a-93a0-fd8470f58f1c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.536482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.537053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.537467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.537744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.538586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.539687 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:04.539687 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:04.539687 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:04.540588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.553404 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3977/7907] 10.222.0.26 () {58 vars in 1131 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/services => generated 2974 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:04.554904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.566100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785875d2-ffbb-4c0b-a53f-a5ad7f13c8b9 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.571839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.572887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.573445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.573775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.574484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1399978680) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:04.574856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.576724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.580626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:04.583506 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3978/7908] 10.222.0.26 () {58 vars in 1222 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1399978680 => generated 416 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:04.587536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.598664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0e42-85ab-4400-a728-ae6bde796c29 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.606281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.607037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.607388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.607725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.608478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-14854131) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:04.608883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.610884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.622466 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3979/7909] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-14854131 => generated 413 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:04.623490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.634164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a44f62-8467-4327-b826-518512cd6b4a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.635082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2c38-3350-4583-b058-2eaca2078538 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.640126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.640925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.641675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.641675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.642473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-908701152) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:04.642861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.643143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.644607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.648844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.649238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.649528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.650269 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3980/7910] 10.222.0.26 () {58 vars in 1220 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-908701152 => generated 416 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:04.650585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.653300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.654779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.665274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01f384-94c4-49fa-9889-ed48f5e5d25c tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.670264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.670975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.671320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.671798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/80d17c00906f4214be27159b22672dc3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.672483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:delete_service(service_id=80d17c00906f4214be27159b22672dc3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.675064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.695143 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3931/7911] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:04 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:04.701101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.716626 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3981/7912] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:04 2022] DELETE /identity/v3/services/80d17c00906f4214be27159b22672dc3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:04.720894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.749351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9746f7-0d59-47ce-8022-cac41f62743e tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.782073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5a814-7b9f-4eb5-9833-3deeb6bc741f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.809904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.810534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.811028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.811304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.812289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.818028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.825925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.826584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.826865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.828136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/c89b1fffc226408aac250f09cc5b2787` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.828839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:delete_service(service_id=c89b1fffc226408aac250f09cc5b2787)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.831798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.849477 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3932/7913] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:04 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:04.853913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.871325 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3982/7914] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:04 2022] DELETE /identity/v3/services/c89b1fffc226408aac250f09cc5b2787 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:04.877502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.877684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154925d-3bd4-4db8-99dc-5c9267397426 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.895844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.896912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.897394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.897880 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.899275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.901222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.908559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193870-3001-4d8d-97e1-4c2b02830da5 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.913973 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3933/7915] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:04.918604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.920122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f726b326b78b4f67ba767b0554da3ccc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.921052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.921531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.921807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] PATH_INFO: `/v3/services/6b569952223e423caf5a79e78fcff7f0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.922402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorizing `identity:delete_service(service_id=6b569952223e423caf5a79e78fcff7f0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.924040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.934560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91807b-8222-4b7d-96d0-2d295fef5ab0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:04.941012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:04.942284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:04.942579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:04.942845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] PATH_INFO: `/v3/projects/dff029989c9b4b73aa179e9ce7c85901/users/942a4aa4755d4c5b839f6aea46ec4848/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:04.954796 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3983/7916] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:04 2022] DELETE /identity/v3/services/6b569952223e423caf5a79e78fcff7f0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:04.959622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.964357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:04.967172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff029989c9b4b73aa179e9ce7c85901, user_id=942a4aa4755d4c5b839f6aea46ec4848, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:04.970307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:04.988249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:04.990619 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3934/7917] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:04 2022] PUT /identity/v3/projects/dff029989c9b4b73aa179e9ce7c85901/users/942a4aa4755d4c5b839f6aea46ec4848/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:04.995126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:04.997721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c2890-21e5-42af-9a7a-f2d0adf28f94 tempest-ServicesTestJSON-1431607351 tempest-ServicesTestJSON-1431607351-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.059864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:05.075180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209d7d9-d9c9-4238-b0a9-17607fd225f2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.086592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.087575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.087862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.088275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.089272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.090734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.094297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.095218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.095553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.095892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] PATH_INFO: `/v3/users/2deac36d6e0e4d1e87ea05082bdb8e23` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.097272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2deac36d6e0e4d1e87ea05082bdb8e23)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.099692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.105616 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3935/7918] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:05.110057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.123591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227ef1e-5fe0-448f-8071-04b00e482abe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.129267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a0bec-91ed-4b75-9ab8-f000f5a99a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.129947 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a0bec-91ed-4b75-9ab8-f000f5a99a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.130330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a0bec-91ed-4b75-9ab8-f000f5a99a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.130561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a0bec-91ed-4b75-9ab8-f000f5a99a5c admin admin] PATH_INFO: `/v3/projects/dff029989c9b4b73aa179e9ce7c85901/users/942a4aa4755d4c5b839f6aea46ec4848/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.133013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a0bec-91ed-4b75-9ab8-f000f5a99a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff029989c9b4b73aa179e9ce7c85901, user_id=942a4aa4755d4c5b839f6aea46ec4848, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.136700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a0bec-91ed-4b75-9ab8-f000f5a99a5c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.152439 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3936/7919] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:05 2022] PUT /identity/v3/projects/dff029989c9b4b73aa179e9ce7c85901/users/942a4aa4755d4c5b839f6aea46ec4848/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:05.162392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2deac36d6e0e4d1e87ea05082bdb8e23'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:05.163566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ab375-ec1a-4c83-8ebf-ed1cf7da0875 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.163859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ab375-ec1a-4c83-8ebf-ed1cf7da0875 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.164132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ab375-ec1a-4c83-8ebf-ed1cf7da0875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.165327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2deac36d6e0e4d1e87ea05082bdb8e23'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:05.189553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2deac36d6e0e4d1e87ea05082bdb8e23'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:05.200381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2deac36d6e0e4d1e87ea05082bdb8e23'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:05.201154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-71f1fcd5-ac57-4c84-a6ea-27b404b09162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2deac36d6e0e4d1e87ea05082bdb8e23'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:05.222392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-490ab375-ec1a-4c83-8ebf-ed1cf7da0875 None None] MFA Rules not processed for user `942a4aa4755d4c5b839f6aea46ec4848`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:05.225519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-490ab375-ec1a-4c83-8ebf-ed1cf7da0875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:05.230243 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3984/7920] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:04 2022] DELETE /identity/v3/users/2deac36d6e0e4d1e87ea05082bdb8e23 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:05.235344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.236325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.236737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.290656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:05.293996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:05.383443 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3937/7921] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:05.390623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ab375-ec1a-4c83-8ebf-ed1cf7da0875 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.391435 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3985/7922] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:05.397834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.399984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:05.406706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ab375-ec1a-4c83-8ebf-ed1cf7da0875 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.436400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:05.518008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.518008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.518008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.518518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.519299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.521483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.534677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e9a59-91f7-4de1-85bc-ee85361d26a2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.538415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.539035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.539314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.539585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.549542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.551578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.570264 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3986/7923] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:05.588300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.595960 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3938/7924] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:05 2022] POST /identity/v3/projects => generated 328 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:05.625856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18234d6e-95ca-46b5-8208-b30b608a71b3 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.713833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.715387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.715740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.716315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.717197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.720214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.764207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.778312 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3987/7925] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:05 2022] POST /identity/v3/users => generated 416 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:05.781356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61765d1b-783f-41fd-a322-206f805a6966 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.814539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.816280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.817502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.819253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.821186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.824766 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.851433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3939/7926] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:05 2022] POST /identity/v3/roles => generated 243 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:05.858353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.873923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2211f-e61b-4d49-afb3-15031dcc64b7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.881092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.881739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.882024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.882343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.883550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:05.887570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:05.925645 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3988/7927] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:05 2022] POST /identity/v3/roles => generated 247 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:05.930960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.953006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3686c-bbbe-4f85-8eb9-371ee25cfd36 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:05.958884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:05.959665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:05.971967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:05.972344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/88cd31c7e6274b628943cada01f54d93/users/1412295be5eb44e09f980c24bf6b937f/roles/bc0a101977b94617a04c492b39447975` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:05.997738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:05.999522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:06.009369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=88cd31c7e6274b628943cada01f54d93, user_id=1412295be5eb44e09f980c24bf6b937f, role_id=bc0a101977b94617a04c492b39447975)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:06.014791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:06.048161 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3940/7928] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:05 2022] PUT /identity/v3/projects/88cd31c7e6274b628943cada01f54d93/users/1412295be5eb44e09f980c24bf6b937f/roles/bc0a101977b94617a04c492b39447975 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:06.052358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:06.063895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:06.107444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9c7b-fac1-41a8-af8d-9527492e9005 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:06.153569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6ace8-dea0-4025-b7ee-c8d75bf6748f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:06.159779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:06.160469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:06.160787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:06.161064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/88cd31c7e6274b628943cada01f54d93/users/1412295be5eb44e09f980c24bf6b937f/roles/65b68df463be416ba540716b65334ffb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:06.163845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=88cd31c7e6274b628943cada01f54d93, user_id=1412295be5eb44e09f980c24bf6b937f, role_id=65b68df463be416ba540716b65334ffb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:06.164501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:06.168247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:06.190228 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3941/7929] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:06 2022] PUT /identity/v3/projects/88cd31c7e6274b628943cada01f54d93/users/1412295be5eb44e09f980c24bf6b937f/roles/65b68df463be416ba540716b65334ffb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:06.192583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:06.194030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:06.194889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:06.195688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:06.195954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] PATH_INFO: `/v3/projects/f5e53e6ab8414fd6b3784651dfc6dad7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:06.197797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e53e6ab8414fd6b3784651dfc6dad7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:06.199949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:06.276796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b98d-c39c-415a-8ac3-3130392cc6e8 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:06.285170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e53e6ab8414fd6b3784651dfc6dad7', 'request_id': 'req-12a88c76-013c-4624-a744-1183c2de88ac'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.308145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e53e6ab8414fd6b3784651dfc6dad7', 'request_id': 'req-12a88c76-013c-4624-a744-1183c2de88ac'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.315608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e53e6ab8414fd6b3784651dfc6dad7', 'request_id': 'req-12a88c76-013c-4624-a744-1183c2de88ac'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.330132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] The token cache is being invalidate because project f5e53e6ab8414fd6b3784651dfc6dad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:06.330509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.331151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.333301 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3989/7930] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:05 2022] DELETE /identity/v3/projects/f5e53e6ab8414fd6b3784651dfc6dad7 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:06.338349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:06.342472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:06.358338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5159e-4c21-4f40-83c5-2bd9e712d5a6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:06.358887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5159e-4c21-4f40-83c5-2bd9e712d5a6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:06.359176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5159e-4c21-4f40-83c5-2bd9e712d5a6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:06.359454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5159e-4c21-4f40-83c5-2bd9e712d5a6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:06.360331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5159e-4c21-4f40-83c5-2bd9e712d5a6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:06.362942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5159e-4c21-4f40-83c5-2bd9e712d5a6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:06.417516 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3942/7931] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:06 2022] GET /identity/v3/users => generated 7510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:06.422452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884417b-7fb0-4cd7-92cc-158dffb789a0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:06.422765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884417b-7fb0-4cd7-92cc-158dffb789a0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:06.423008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884417b-7fb0-4cd7-92cc-158dffb789a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:06.467930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a88c76-013c-4624-a744-1183c2de88ac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:06.472728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:06.473377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:06.473657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:06.473931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] PATH_INFO: `/v3/users/f726b326b78b4f67ba767b0554da3ccc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:06.475093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f726b326b78b4f67ba767b0554da3ccc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:06.477172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:06.498337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1884417b-7fb0-4cd7-92cc-158dffb789a0 None None] MFA Rules not processed for user `1412295be5eb44e09f980c24bf6b937f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:06.502069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1884417b-7fb0-4cd7-92cc-158dffb789a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:06.546655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f726b326b78b4f67ba767b0554da3ccc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.547364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f726b326b78b4f67ba767b0554da3ccc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.556915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:06.560972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f726b326b78b4f67ba767b0554da3ccc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.575681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f726b326b78b4f67ba767b0554da3ccc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.576583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac7a84f6-9571-47e3-bd0c-82a54b978d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f726b326b78b4f67ba767b0554da3ccc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:06.587727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:06.618610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3990/7932] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:06 2022] DELETE /identity/v3/users/f726b326b78b4f67ba767b0554da3ccc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:06.732753 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3943/7933] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:06 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:06.733489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:06.733489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:06.733489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:06.733489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] MFA Rules not processed for user `1412295be5eb44e09f980c24bf6b937f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:06.733489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:06.788687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884417b-7fb0-4cd7-92cc-158dffb789a0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:06.788865 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3991/7934] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:06 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:06.801396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884417b-7fb0-4cd7-92cc-158dffb789a0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:06.806758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1412295be5eb44e09f980c24bf6b937f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88cd31c7e6274b628943cada01f54d93', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-556534502', 'tempest-NotDelegatedRole-1669236099'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:06.808001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:06.808282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:06.808560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:06.809291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] RBAC: Authorizing `identity:create_trust()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:06.810482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:06.930948 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3944/7935] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 10:34:06 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:06.931091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:06.932471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:07.020934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1c364-77bb-4b06-9635-8a3e2ee504a9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:07.027392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:07.028120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.028486 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.028837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.029560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:07.030678 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:07.030678 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:07.030678 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:07.033797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:07.034998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:07.037001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:07.043943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3992/7936] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 10:34:06 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:07.048012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:07.070053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9f055-83c6-4d46-9355-4b4b30ee2b37 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:07.081836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1412295be5eb44e09f980c24bf6b937f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88cd31c7e6274b628943cada01f54d93', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-556534502', 'tempest-NotDelegatedRole-1669236099'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:07.082657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.082939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.083301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] PATH_INFO: `/v3/OS-TRUST/trusts/e1e77d97ae9e49f0ac3121fb9b85e060` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.106734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] RBAC: Authorizing `identity:delete_trust(trust_id=e1e77d97ae9e49f0ac3121fb9b85e060)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:07.108362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:07.163386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'e1e77d97ae9e49f0ac3121fb9b85e060', 'request_id': 'req-0cd5c86b-308c-470b-a8bf-00f7898ee763'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.164555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'e1e77d97ae9e49f0ac3121fb9b85e060', 'request_id': 'req-0cd5c86b-308c-470b-a8bf-00f7898ee763'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.188753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'e1e77d97ae9e49f0ac3121fb9b85e060', 'request_id': 'req-0cd5c86b-308c-470b-a8bf-00f7898ee763'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.196184 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3993/7937] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:34:07 2022] DELETE /identity/v3/OS-TRUST/trusts/e1e77d97ae9e49f0ac3121fb9b85e060 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:07.201757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:07.202357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:07.245185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5c86b-308c-470b-a8bf-00f7898ee763 tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:07.254299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235aecea-e1e8-4418-ae39-93526256792d tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:07.262746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1412295be5eb44e09f980c24bf6b937f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '88cd31c7e6274b628943cada01f54d93', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-556534502', 'tempest-NotDelegatedRole-1669236099'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:07.263421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.263699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.263971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] PATH_INFO: `/v3/OS-TRUST/trusts/e1e77d97ae9e49f0ac3121fb9b85e060` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Could not find trust: e1e77d97ae9e49f0ac3121fb9b85e060.: keystone.exception.TrustNotFound: Could not find trust: e1e77d97ae9e49f0ac3121fb9b85e060. Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 27 10:34:07.306417 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: e1e77d97ae9e49f0ac3121fb9b85e060. Sep 27 10:34:07.307434 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:07.308414 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3994/7938] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:07 2022] GET /identity/v3/OS-TRUST/trusts/e1e77d97ae9e49f0ac3121fb9b85e060 => generated 111 bytes in 108 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:07.314395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:07.320324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:07.321150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:07.323771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.326339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.326615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] PATH_INFO: `/v3/projects/cc1d43b1a2854f0b85a23b9b2a872df6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.333441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1d43b1a2854f0b85a23b9b2a872df6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:07.335497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:07.358982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cac790-ad5e-422a-8dc1-3832dcc35fbc tempest-TrustsV3TestJSON-504562143 tempest-user-1924858470] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:07.376418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:07.377120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.377403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.377680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/65b68df463be416ba540716b65334ffb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.379098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=65b68df463be416ba540716b65334ffb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:07.380904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:07.440497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'request_id': 'req-237296d4-b6e5-4ab4-addb-0f25557a59e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.455753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'request_id': 'req-237296d4-b6e5-4ab4-addb-0f25557a59e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.465128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:07.476442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role 65b68df463be416ba540716b65334ffb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:07.476442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.477168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.482780 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3995/7939] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:07 2022] DELETE /identity/v3/roles/65b68df463be416ba540716b65334ffb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:07.483289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1d43b1a2854f0b85a23b9b2a872df6', 'request_id': 'req-237296d4-b6e5-4ab4-addb-0f25557a59e7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.487198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:07.489141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:07.498117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:07.518142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] The token cache is being invalidate because project cc1d43b1a2854f0b85a23b9b2a872df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:07.520911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.523456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-237296d4-b6e5-4ab4-addb-0f25557a59e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.530160 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3945/7940] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:07 2022] DELETE /identity/v3/projects/cc1d43b1a2854f0b85a23b9b2a872df6 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:07.541435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbafefe-8290-4bec-9723-ff74a3b6d2d0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.541870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbafefe-8290-4bec-9723-ff74a3b6d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.542247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbafefe-8290-4bec-9723-ff74a3b6d2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.587347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6dbafefe-8290-4bec-9723-ff74a3b6d2d0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:07.588982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6dbafefe-8290-4bec-9723-ff74a3b6d2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:07.714694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206deab9-4d2f-4585-9ac1-4fd9a9748020 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:07.728348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:07.730471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.731206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.731488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/bc0a101977b94617a04c492b39447975` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.736041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bc0a101977b94617a04c492b39447975)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:07.740675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:07.871697 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3946/7941] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:07.881171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db6d53-f835-4e6d-b113-ad438dcf8e2a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:07.881171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db6d53-f835-4e6d-b113-ad438dcf8e2a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:07.881373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role bc0a101977b94617a04c492b39447975 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:07.881482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db6d53-f835-4e6d-b113-ad438dcf8e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:07.881550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.882323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:07.889567 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3996/7942] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:07 2022] DELETE /identity/v3/roles/bc0a101977b94617a04c492b39447975 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:07.893880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:07.895831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:07.934564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e9db6d53-f835-4e6d-b113-ad438dcf8e2a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:07.936023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e9db6d53-f835-4e6d-b113-ad438dcf8e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:08.071502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529dacf-709d-4051-a680-c8f80bf2c1d7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.084402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.085645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.086128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.087184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users/1412295be5eb44e09f980c24bf6b937f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.090340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1412295be5eb44e09f980c24bf6b937f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.092384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.098145 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3947/7943] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:08.099147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db6d53-f835-4e6d-b113-ad438dcf8e2a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.118148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db6d53-f835-4e6d-b113-ad438dcf8e2a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.133449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.134300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.134887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.135696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.141532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:08.141857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.143504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.173216 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3948/7944] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:08.178013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.200398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f910d8d-9760-4ed8-abaf-b29064d549c2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.212462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.212462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.213022 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.213333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.215979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.221917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.234179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1412295be5eb44e09f980c24bf6b937f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.236192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1412295be5eb44e09f980c24bf6b937f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.262812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1412295be5eb44e09f980c24bf6b937f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.270465 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3949/7945] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:08 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:08.271404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.274070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1412295be5eb44e09f980c24bf6b937f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.275271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:08.277147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1412295be5eb44e09f980c24bf6b937f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.297958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:08.343540 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3997/7946] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:07 2022] DELETE /identity/v3/users/1412295be5eb44e09f980c24bf6b937f => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:08.347497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.350693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:08.366089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:08.397703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90988ea0-c6b2-4702-b777-0c864e8ff489 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.504814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34da67-8e66-494b-bf6f-f74cbc220110 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.512304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.513094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.513538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.514238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/88cd31c7e6274b628943cada01f54d93` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.515513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_project(project_id=88cd31c7e6274b628943cada01f54d93)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.517926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.539983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.541790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.543590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.543859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.545038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.547300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.592731 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3950/7947] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:08 2022] POST /identity/v3/users => generated 285 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:08.597143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.621439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5061b83-a53b-441f-af79-9ebdfd25c71c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.626419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88cd31c7e6274b628943cada01f54d93', 'request_id': 'req-b2102ade-329f-49a4-b6a1-8ad75c1165b9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.650343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88cd31c7e6274b628943cada01f54d93', 'request_id': 'req-b2102ade-329f-49a4-b6a1-8ad75c1165b9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.663766 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88cd31c7e6274b628943cada01f54d93', 'request_id': 'req-b2102ade-329f-49a4-b6a1-8ad75c1165b9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.690878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] The token cache is being invalidate because project 88cd31c7e6274b628943cada01f54d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:08.691224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.691929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:08.696718 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3998/7948] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:08 2022] DELETE /identity/v3/projects/88cd31c7e6274b628943cada01f54d93 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:08.705159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.710019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:08.757378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.758684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.759209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.759792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.766590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.771118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.803555 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3951/7949] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:08.804460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.806857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:08.824986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4caa79-c64f-4549-90e4-98685b449452 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.834383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.840561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.841038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.841322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.842033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.844002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.856833 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3952/7950] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:08.861142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.862604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2102ade-329f-49a4-b6a1-8ad75c1165b9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.868195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.869116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.869326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.869927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.870354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.872580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.881936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d56e1-2334-4b48-9066-b8e404147ec8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:08.895438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f2d86-c73a-42fe-a5e2-becd7142775b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:08.896431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2d86-c73a-42fe-a5e2-becd7142775b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.896879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2d86-c73a-42fe-a5e2-becd7142775b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.897242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2d86-c73a-42fe-a5e2-becd7142775b admin admin] PATH_INFO: `/v3/projects/85a2f6d03a5f4ff68d55f3cbf7b49f04/users/5c05482193ae491694ecd21aeb6f988d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:08.903052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:08.915313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 3999/7951] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:08 2022] POST /identity/v3/projects => generated 328 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:08.918171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f2d86-c73a-42fe-a5e2-becd7142775b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a2f6d03a5f4ff68d55f3cbf7b49f04, user_id=5c05482193ae491694ecd21aeb6f988d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:08.918306 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:08.921147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f2d86-c73a-42fe-a5e2-becd7142775b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:08.944173 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3953/7952] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:08 2022] PUT /identity/v3/projects/85a2f6d03a5f4ff68d55f3cbf7b49f04/users/5c05482193ae491694ecd21aeb6f988d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:08.960929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb4e02-521c-484f-801f-ca6914650d8c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:08.961210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb4e02-521c-484f-801f-ca6914650d8c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:08.961835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb4e02-521c-484f-801f-ca6914650d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.000424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-caeb4e02-521c-484f-801f-ca6914650d8c None None] MFA Rules not processed for user `5c05482193ae491694ecd21aeb6f988d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:09.001913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-caeb4e02-521c-484f-801f-ca6914650d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:09.067038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:09.091030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099295b8-1b45-4f98-9ac5-53569a1afa84 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.109856 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3954/7953] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:09.110691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb4e02-521c-484f-801f-ca6914650d8c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.207365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.208018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.208302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.208583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.209274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.211267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.223919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb4e02-521c-484f-801f-ca6914650d8c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.233882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.234641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.234996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.235346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.236099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.242102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.280367 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3955/7954] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:09 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:09.284676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.284799 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4000/7955] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:08 2022] POST /identity/v3/users => generated 416 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:09.288995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.310000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37493076-8b59-4f09-8d95-29156e3bdddb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.383402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654d04e-2346-4462-bf34-2eaf4f01224a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.389877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.390662 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.390946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.391222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.392185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.394282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.397143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.397841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.398807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.399312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.400349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.402829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.427262 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3956/7956] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:09 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:09.428412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.431628 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4001/7957] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:09 2022] POST /identity/v3/roles => generated 243 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:09.432862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.444878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d4574-f3a4-4fc6-b0b6-2a8d673e2618 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.446055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beff582-cfac-46ca-aadc-41b3128d52d9 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.451453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.452090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.452279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.452532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.452842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.453025 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.453357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.453631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.453722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.454351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.455426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.456093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.474025 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3957/7958] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:09 2022] GET /identity/v3/roles => generated 1654 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:09.479041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.486210 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4002/7959] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:09 2022] POST /identity/v3/roles => generated 246 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:09.490431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.493641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902953a6-a94e-44b3-b174-418a8685b144 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.500503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.505464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.505976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.506322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] PATH_INFO: `/v3/projects/ab703d9140c5449d9002eb91439752c6/users/12f26677216f475b801e43cc01dfb382/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.512123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4449d-50ba-4690-b1af-c6c35f0dec16 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.521851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.523253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.523690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.523979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/9f9084f4df94439aa2f5e2d61488e1ab/users/5f83f26412a04d1aae82833be73bdc6b/roles/ac4d08f4ad5340a78824a1f0c7802874` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.526464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab703d9140c5449d9002eb91439752c6, user_id=12f26677216f475b801e43cc01dfb382, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.529934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.547201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=9f9084f4df94439aa2f5e2d61488e1ab, user_id=5f83f26412a04d1aae82833be73bdc6b, role_id=ac4d08f4ad5340a78824a1f0c7802874)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.550238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.552984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:09.556335 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3958/7960] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:09 2022] PUT /identity/v3/projects/ab703d9140c5449d9002eb91439752c6/users/12f26677216f475b801e43cc01dfb382/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:09.561316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.586175 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4003/7961] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:09 2022] PUT /identity/v3/projects/9f9084f4df94439aa2f5e2d61488e1ab/users/5f83f26412a04d1aae82833be73bdc6b/roles/ac4d08f4ad5340a78824a1f0c7802874 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:09.592922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.598368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:09.657788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d911ae-33c9-4700-8e28-b9363107c796 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.747591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.749203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.749490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.749920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.750659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.752269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.770503 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3959/7962] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:09 2022] GET /identity/v3/roles => generated 1891 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:09.774320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.777329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a363-9cde-4360-91ca-ad0e8f6e224c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.789312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252989dd-027b-4ab1-b262-f2859d314b89 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:09.789912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.790874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.791345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.791754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/9f9084f4df94439aa2f5e2d61488e1ab/users/5f83f26412a04d1aae82833be73bdc6b/roles/03c800a6328c4fd9b91e47ad68ca95d2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.796607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e59ba-8fba-46b1-8013-cc93b2fa1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:09.797588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=9f9084f4df94439aa2f5e2d61488e1ab, user_id=5f83f26412a04d1aae82833be73bdc6b, role_id=03c800a6328c4fd9b91e47ad68ca95d2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.798600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e59ba-8fba-46b1-8013-cc93b2fa1e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.798978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e59ba-8fba-46b1-8013-cc93b2fa1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.799260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e59ba-8fba-46b1-8013-cc93b2fa1e75 admin admin] PATH_INFO: `/v3/projects/ab703d9140c5449d9002eb91439752c6/users/12f26677216f475b801e43cc01dfb382/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.800853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.802910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e59ba-8fba-46b1-8013-cc93b2fa1e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab703d9140c5449d9002eb91439752c6, user_id=12f26677216f475b801e43cc01dfb382, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:09.805785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e59ba-8fba-46b1-8013-cc93b2fa1e75 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:09.822418 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3960/7963] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:09 2022] PUT /identity/v3/projects/ab703d9140c5449d9002eb91439752c6/users/12f26677216f475b801e43cc01dfb382/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:09.834153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfa8fc-559e-4d96-809d-3139d537bd89 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:09.834787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfa8fc-559e-4d96-809d-3139d537bd89 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:09.835255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfa8fc-559e-4d96-809d-3139d537bd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:09.840873 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4004/7964] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:09 2022] PUT /identity/v3/projects/9f9084f4df94439aa2f5e2d61488e1ab/users/5f83f26412a04d1aae82833be73bdc6b/roles/03c800a6328c4fd9b91e47ad68ca95d2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:09.846461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:09.876734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d7dfa8fc-559e-4d96-809d-3139d537bd89 None None] MFA Rules not processed for user `12f26677216f475b801e43cc01dfb382`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:09.877975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d7dfa8fc-559e-4d96-809d-3139d537bd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:09.995737 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3961/7965] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:09.997848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfa8fc-559e-4d96-809d-3139d537bd89 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.010591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfa8fc-559e-4d96-809d-3139d537bd89 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.015495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12f26677216f475b801e43cc01dfb382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab703d9140c5449d9002eb91439752c6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.016402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.016709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.016972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.017824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.020292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.033343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4504fd7-e90e-40be-92f4-a75a4509f70f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.048628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd8055-4466-4b35-9cef-0e10c2047b73 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.049297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8055-4466-4b35-9cef-0e10c2047b73 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.049575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8055-4466-4b35-9cef-0e10c2047b73 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.049847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8055-4466-4b35-9cef-0e10c2047b73 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.050613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd8055-4466-4b35-9cef-0e10c2047b73 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.051650 ncdv-master-1 devstack@keystone.service[104132]: WARNING py.warnings [None req-f2cd8055-4466-4b35-9cef-0e10c2047b73 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 10:34:10.051650 ncdv-master-1 devstack@keystone.service[104132]: warnings.warn(msg) Sep 27 10:34:10.051650 ncdv-master-1 devstack@keystone.service[104132]: Sep 27 10:34:10.056681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd8055-4466-4b35-9cef-0e10c2047b73 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.076269 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3962/7966] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:09 2022] POST /identity/v3/projects => generated 359 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:10.077875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.095474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:10.143665 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4005/7967] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:09 2022] GET /identity/v3/users => generated 7787 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:10.146468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea06ec7-4826-4bec-b39f-89a776c954b4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.146858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea06ec7-4826-4bec-b39f-89a776c954b4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.147140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea06ec7-4826-4bec-b39f-89a776c954b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.188731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76349368-719c-431f-b0f6-d10aafb35b13 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.203353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12f26677216f475b801e43cc01dfb382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab703d9140c5449d9002eb91439752c6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.203687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4ea06ec7-4826-4bec-b39f-89a776c954b4 None None] MFA Rules not processed for user `5f83f26412a04d1aae82833be73bdc6b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:10.204013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.204299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.204578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.204911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4ea06ec7-4826-4bec-b39f-89a776c954b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:10.205546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.208415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.255333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3963/7968] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:10 2022] POST /identity/v3/users => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:10.264439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.277508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508eaa9-aa1f-4838-8071-adbd4af0c9d7 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.283469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12f26677216f475b801e43cc01dfb382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab703d9140c5449d9002eb91439752c6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.284413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.284807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.285088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.285854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:10.286164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.288334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.341305 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3964/7969] 10.222.0.26 () {58 vars in 1157 bytes} [Tue Sep 27 10:34:10 2022] GET /identity/v3/users?domain_id=default => generated 7867 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:10.346576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.362415 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4006/7970] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:10.364810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77095870-8082-4c29-8b7c-203afcc7b7b8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.365101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77095870-8082-4c29-8b7c-203afcc7b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.365373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77095870-8082-4c29-8b7c-203afcc7b7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.368861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43ef12-2184-4326-99c4-a8d3f291be19 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.380645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12f26677216f475b801e43cc01dfb382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab703d9140c5449d9002eb91439752c6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.381573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] REQUEST_METHOD: `PATCH` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.381992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.382424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] PATH_INFO: `/v3/users/b7129bc053104d7f88d06898606a5496` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.382528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:10.396489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-77095870-8082-4c29-8b7c-203afcc7b7b8 None None] MFA Rules not processed for user `5f83f26412a04d1aae82833be73bdc6b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:10.398016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-77095870-8082-4c29-8b7c-203afcc7b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:10.409915 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4007/7971] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:10.414907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77095870-8082-4c29-8b7c-203afcc7b7b8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.415826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorizing `identity:update_user(user_id=b7129bc053104d7f88d06898606a5496)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.418078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.425873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77095870-8082-4c29-8b7c-203afcc7b7b8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.430889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f83f26412a04d1aae82833be73bdc6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9084f4df94439aa2f5e2d61488e1ab', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-575960667', 'tempest-DelegatedRole-763374298'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.431983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.432430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.432871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.433681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: Authorizing `identity:create_trust()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.435143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.482917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b7129bc053104d7f88d06898606a5496'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.502350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invalidating the token cache because user b7129bc053104d7f88d06898606a5496 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:10.502658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.503507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3d869bee-2e4a-4a46-b18c-0671874d2309 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.506159 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3965/7972] 10.222.0.26 () {60 vars in 1244 bytes} [Tue Sep 27 10:34:10 2022] PATCH /identity/v3/users/b7129bc053104d7f88d06898606a5496 => generated 488 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:10.510853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b04b5-0ae5-4ce8-96d0-5af32674c022 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.511137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b04b5-0ae5-4ce8-96d0-5af32674c022 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.511413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b04b5-0ae5-4ce8-96d0-5af32674c022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.548285 ncdv-master-1 devstack@keystone.service[104131]: WARNING keystone.auth.plugins.core [None req-252b04b5-0ae5-4ce8-96d0-5af32674c022 None None] User is disabled: b7129bc053104d7f88d06898606a5496: AssertionError: User is disabled: b7129bc053104d7f88d06898606a5496 Sep 27 10:34:10.559104 ncdv-master-1 devstack@keystone.service[104131]: WARNING keystone.server.flask.application [None req-252b04b5-0ae5-4ce8-96d0-5af32674c022 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 27 10:34:10.560816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3966/7973] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:10 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 51 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 27 10:34:10.581429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.581920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.582359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.584023 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4008/7974] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 10:34:10 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:10.584913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.589000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:10.617292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] MFA Rules not processed for user `0bd8a19d0ab44011b5430f26757a0efe`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:10.618864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d752ce-349c-4822-b45b-9c001d10e1e1 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.619231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:10.636723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12f26677216f475b801e43cc01dfb382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab703d9140c5449d9002eb91439752c6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.637645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.638000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.638430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] PATH_INFO: `/v3/users/b7129bc053104d7f88d06898606a5496` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.679788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b7129bc053104d7f88d06898606a5496)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.682545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.781525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7129bc053104d7f88d06898606a5496'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.781669 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3967/7975] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:10.782578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.784024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:10.792048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7129bc053104d7f88d06898606a5496'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.804199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112378a2-e945-4abb-8050-4e54e8d3b5ee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.812969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7129bc053104d7f88d06898606a5496'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.826791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f83f26412a04d1aae82833be73bdc6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9084f4df94439aa2f5e2d61488e1ab', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-575960667', 'tempest-DelegatedRole-763374298'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:10.828034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:10.829594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:10.829873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:10.833376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:10.834759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:10.841714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7129bc053104d7f88d06898606a5496'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.843334 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7129bc053104d7f88d06898606a5496'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:10.859692 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3968/7976] 10.222.0.26 () {58 vars in 1252 bytes} [Tue Sep 27 10:34:10 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=5f83f26412a04d1aae82833be73bdc6b => generated 611 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:10.860914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.867494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:10.896710 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4009/7977] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:10 2022] DELETE /identity/v3/users/b7129bc053104d7f88d06898606a5496 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:10.901184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:10.904132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:10.918408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9a951-1ae1-48a7-a72a-357c9b57b933 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:10.965967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:11.011932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f83f26412a04d1aae82833be73bdc6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9084f4df94439aa2f5e2d61488e1ab', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-575960667', 'tempest-DelegatedRole-763374298'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:11.012918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:11.013243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:11.013798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] PATH_INFO: `/v3/OS-TRUST/trusts/13741ae5ebe747f08f34eb1b5b10126a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:11.034879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: Authorizing `identity:delete_trust(trust_id=13741ae5ebe747f08f34eb1b5b10126a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:11.036962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:11.046154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:11.075001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '13741ae5ebe747f08f34eb1b5b10126a', 'request_id': 'req-cf24d4f6-8535-4888-80af-64b54323e3e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.091772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '13741ae5ebe747f08f34eb1b5b10126a', 'request_id': 'req-cf24d4f6-8535-4888-80af-64b54323e3e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.094313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8912e-08d8-43fb-97a7-67787549362c tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:11.095773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '13741ae5ebe747f08f34eb1b5b10126a', 'request_id': 'req-cf24d4f6-8535-4888-80af-64b54323e3e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.102311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12f26677216f475b801e43cc01dfb382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab703d9140c5449d9002eb91439752c6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:11.108680 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3969/7978] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:34:10 2022] DELETE /identity/v3/OS-TRUST/trusts/13741ae5ebe747f08f34eb1b5b10126a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:11.108851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:11.108851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:11.108851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] PATH_INFO: `/v3/projects/52b056730ce74e8bac8834c20d4e25a3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:11.109553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:11.110033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorizing `identity:delete_project(project_id=52b056730ce74e8bac8834c20d4e25a3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:11.111077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:11.112149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:11.145298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24d4f6-8535-4888-80af-64b54323e3e9 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:11.155893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f83f26412a04d1aae82833be73bdc6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9084f4df94439aa2f5e2d61488e1ab', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-575960667', 'tempest-DelegatedRole-763374298'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:11.157337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:11.157487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:11.157743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] PATH_INFO: `/v3/OS-TRUST/trusts/13741ae5ebe747f08f34eb1b5b10126a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Could not find trust: 13741ae5ebe747f08f34eb1b5b10126a.: keystone.exception.TrustNotFound: Could not find trust: 13741ae5ebe747f08f34eb1b5b10126a. Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 27 10:34:11.169641 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 13741ae5ebe747f08f34eb1b5b10126a. Sep 27 10:34:11.170606 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:11.171295 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3970/7979] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:11 2022] GET /identity/v3/OS-TRUST/trusts/13741ae5ebe747f08f34eb1b5b10126a => generated 111 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:34:11.176487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:11.193261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:11.204477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b056730ce74e8bac8834c20d4e25a3', 'request_id': 'req-a363a808-b181-4804-9838-2a3abb93794d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.229999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b056730ce74e8bac8834c20d4e25a3', 'request_id': 'req-a363a808-b181-4804-9838-2a3abb93794d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.245469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b056730ce74e8bac8834c20d4e25a3', 'request_id': 'req-a363a808-b181-4804-9838-2a3abb93794d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.271477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] The token cache is being invalidate because project 52b056730ce74e8bac8834c20d4e25a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:11.271882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.273162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.280521 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4010/7980] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:10 2022] DELETE /identity/v3/projects/52b056730ce74e8bac8834c20d4e25a3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:11.282040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:11.283818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:11.363496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e3b5-8715-43fe-b1c1-b5820261ee65 tempest-TrustsV3TestJSON-405993621 tempest-user-1906088208] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:11.456429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363a808-b181-4804-9838-2a3abb93794d tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:11.479315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12f26677216f475b801e43cc01dfb382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab703d9140c5449d9002eb91439752c6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:11.481501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:11.481501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:11.481785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:11.482389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:11.484377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:11.505871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:11.506587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:11.506834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:11.507103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/03c800a6328c4fd9b91e47ad68ca95d2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:11.514019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=03c800a6328c4fd9b91e47ad68ca95d2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Could not find domain: 347a2d56ffc446ef8ae471662b514962.: keystone.exception.DomainNotFound: Could not find domain: 347a2d56ffc446ef8ae471662b514962. Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 347a2d56ffc446ef8ae471662b514962. Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:11.514593 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 10:34:11.515358 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application with Lock( Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._enter() Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.creator() Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 10:34:11.516136 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application created_value = creator( Sep 27 10:34:11.518642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 10:34:11.518642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 10:34:11.518642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 347a2d56ffc446ef8ae471662b514962. Sep 27 10:34:11.518642 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:11.518861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:11.518947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4011/7981] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:11 2022] POST /identity/v3/users => generated 112 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:11.525093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:11.537466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:11.621845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role 03c800a6328c4fd9b91e47ad68ca95d2 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:11.622208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.622812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.628374 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3971/7982] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:11 2022] DELETE /identity/v3/roles/03c800a6328c4fd9b91e47ad68ca95d2 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:11.629370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:11.632330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:11.646164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057601f-0eac-49db-8eb2-6bf74681ff10 tempest-UsersNegativeTest-1476276338 tempest-UsersNegativeTest-1476276338-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:11.659975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:11.720245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:11.803227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21933c1-a399-4d62-ba59-27e1e47fc737 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:11.813630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:11.814866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:11.815156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:11.815435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/ac4d08f4ad5340a78824a1f0c7802874` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:11.818015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=ac4d08f4ad5340a78824a1f0c7802874)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:11.820599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:11.824205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:11.824949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:11.825234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:11.825510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] PATH_INFO: `/v3/users/5c05482193ae491694ecd21aeb6f988d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:11.829433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c05482193ae491694ecd21aeb6f988d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:11.831358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:11.930302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role ac4d08f4ad5340a78824a1f0c7802874 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:11.930598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.931835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.935466 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3972/7983] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:11 2022] DELETE /identity/v3/roles/ac4d08f4ad5340a78824a1f0c7802874 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:11.940932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:11.942990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:11.965891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c05482193ae491694ecd21aeb6f988d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.966871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c05482193ae491694ecd21aeb6f988d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:11.990610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c05482193ae491694ecd21aeb6f988d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.004511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c05482193ae491694ecd21aeb6f988d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.005548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ad08a280-3a58-468c-9417-be4aa28f7e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c05482193ae491694ecd21aeb6f988d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.053106 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4012/7984] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:11 2022] DELETE /identity/v3/users/5c05482193ae491694ecd21aeb6f988d => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:12.059610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911385c8-241d-44b6-a53f-8410d170371d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:12.060167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911385c8-241d-44b6-a53f-8410d170371d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:12.060747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911385c8-241d-44b6-a53f-8410d170371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:12.100676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-911385c8-241d-44b6-a53f-8410d170371d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:12.103662 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-911385c8-241d-44b6-a53f-8410d170371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:12.113461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeacad-7b25-47a8-b212-46804116c551 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:12.217565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4013/7985] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:12.229459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:12.230251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:12.230598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:12.230981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users/5f83f26412a04d1aae82833be73bdc6b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:12.232391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_user(user_id=5f83f26412a04d1aae82833be73bdc6b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:12.232950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911385c8-241d-44b6-a53f-8410d170371d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:12.234699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:12.237320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-911385c8-241d-44b6-a53f-8410d170371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:12.270861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:12.310409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f83f26412a04d1aae82833be73bdc6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.312001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f83f26412a04d1aae82833be73bdc6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.318597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:12.323677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f83f26412a04d1aae82833be73bdc6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.329541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911385c8-241d-44b6-a53f-8410d170371d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:12.335910 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f83f26412a04d1aae82833be73bdc6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.338488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:12.339252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:12.340258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:12.341030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:12.343793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:12.356732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f83f26412a04d1aae82833be73bdc6b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.358780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:12.360562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:12.369388 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3973/7986] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:11 2022] DELETE /identity/v3/users/5f83f26412a04d1aae82833be73bdc6b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:12.373619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:12.375680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:12.377356 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4014/7987] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:12.508248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96aa45-8048-4d5f-9377-6cc2e9463f0c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:12.515398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:12.516218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:12.516370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:12.516652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/9f9084f4df94439aa2f5e2d61488e1ab` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:12.518380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9f9084f4df94439aa2f5e2d61488e1ab)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:12.521457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:12.630376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9084f4df94439aa2f5e2d61488e1ab', 'request_id': 'req-f05c83f1-4ab9-4b23-8656-20834fdbea7f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.637925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9084f4df94439aa2f5e2d61488e1ab', 'request_id': 'req-f05c83f1-4ab9-4b23-8656-20834fdbea7f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.658181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9084f4df94439aa2f5e2d61488e1ab', 'request_id': 'req-f05c83f1-4ab9-4b23-8656-20834fdbea7f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.684163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] The token cache is being invalidate because project 9f9084f4df94439aa2f5e2d61488e1ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:12.684690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.685695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:12.688324 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3974/7988] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:12 2022] DELETE /identity/v3/projects/9f9084f4df94439aa2f5e2d61488e1ab => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:12.693939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:12.713838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a68bb5-39b9-48b0-a511-fdb2b54c09a3 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:12.816258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:12.816995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:12.817215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:12.817515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:12.818121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:12.820160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:12.864151 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4015/7989] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:12 2022] POST /identity/v3/projects => generated 328 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:12.868605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:12.870783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:12.885784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:12.889416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:12.892173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c83f1-4ab9-4b23-8656-20834fdbea7f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:12.932545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:12.976192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:12.977062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:12.977505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:12.977904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:12.978813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:12.981505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.016997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3975/7990] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:12 2022] POST /identity/v3/users => generated 416 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:13.018316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:13.033162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c559d-f206-4926-ace4-441fdc643494 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.040809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.042285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.042670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.043048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.044236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.046237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.077073 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3976/7991] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:13 2022] POST /identity/v3/roles => generated 244 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:13.079992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:13.098889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8296e08-aed2-4adf-af77-e1f1b088ab42 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.102573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f046-3041-49b6-b048-9c5c6bbebfe6 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.110346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.112924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.113245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.113736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] PATH_INFO: `/v3/projects/85a2f6d03a5f4ff68d55f3cbf7b49f04` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.115833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a2f6d03a5f4ff68d55f3cbf7b49f04)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.116562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.118237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.118237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.118237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.118504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.118588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.120351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.186474 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3977/7992] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:13 2022] POST /identity/v3/roles => generated 247 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:13.187561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:13.213318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749d7c0-6aa3-489b-9d4b-4bd4ce8ba45d tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.227417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.228213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.228855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.228855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/d3797a8e9ee84e139f83dff18a3fb673/users/ae187ae7d9824deaa4ffeb43bc21c33c/roles/dd3561b254c34e0881f4ea11d5d35181` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.256557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a2f6d03a5f4ff68d55f3cbf7b49f04', 'request_id': 'req-b62975d8-6e06-423f-b92f-6490b2445724'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.256770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d3797a8e9ee84e139f83dff18a3fb673, user_id=ae187ae7d9824deaa4ffeb43bc21c33c, role_id=dd3561b254c34e0881f4ea11d5d35181)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.260785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.286052 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3978/7993] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:13 2022] PUT /identity/v3/projects/d3797a8e9ee84e139f83dff18a3fb673/users/ae187ae7d9824deaa4ffeb43bc21c33c/roles/dd3561b254c34e0881f4ea11d5d35181 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:13.290062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:13.303986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a2f6d03a5f4ff68d55f3cbf7b49f04', 'request_id': 'req-b62975d8-6e06-423f-b92f-6490b2445724'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.326571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a2f6d03a5f4ff68d55f3cbf7b49f04', 'request_id': 'req-b62975d8-6e06-423f-b92f-6490b2445724'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.392622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] The token cache is being invalidate because project 85a2f6d03a5f4ff68d55f3cbf7b49f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:13.392952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.394630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.401896 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4016/7994] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:12 2022] DELETE /identity/v3/projects/85a2f6d03a5f4ff68d55f3cbf7b49f04 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:13.403574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:13.417973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:13.425382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:13.434305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958b2d-75ca-40d9-a636-a7a3205448ab tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.440864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.452993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.454638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.455916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/d3797a8e9ee84e139f83dff18a3fb673/users/ae187ae7d9824deaa4ffeb43bc21c33c/roles/738fbf40fe8b4d2288ca77b16ddac655` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.465155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d3797a8e9ee84e139f83dff18a3fb673, user_id=ae187ae7d9824deaa4ffeb43bc21c33c, role_id=738fbf40fe8b4d2288ca77b16ddac655)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.468815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.493892 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3979/7995] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:13 2022] PUT /identity/v3/projects/d3797a8e9ee84e139f83dff18a3fb673/users/ae187ae7d9824deaa4ffeb43bc21c33c/roles/738fbf40fe8b4d2288ca77b16ddac655 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:13.495239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:13.500363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:13.536412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62975d8-6e06-423f-b92f-6490b2445724 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.581886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d99223-d828-40ed-b664-3da8b171de6b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.586429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.587226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.588866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.588866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.589640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:13.605939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:13.641695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.642982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.644155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.645143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] PATH_INFO: `/v3/users/12f26677216f475b801e43cc01dfb382` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.653015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f26677216f475b801e43cc01dfb382)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.655709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.715188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.716637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.731059 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3980/7996] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:13.732485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:13.733849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:13.756257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f26677216f475b801e43cc01dfb382'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.757034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f26677216f475b801e43cc01dfb382'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.770145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f26677216f475b801e43cc01dfb382'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.780427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f26677216f475b801e43cc01dfb382'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.781472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-8a40760a-3b33-4ada-8317-8f59009f9e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f26677216f475b801e43cc01dfb382'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:13.805602 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4017/7997] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:13 2022] DELETE /identity/v3/users/12f26677216f475b801e43cc01dfb382 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:13.869942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0809bb2-f6e9-405f-8210-22bdcf7fa6c1 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:13.943709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1e3f6-00a1-4041-8a06-6e1f2ed8ec49 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:13.944463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e3f6-00a1-4041-8a06-6e1f2ed8ec49 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.945484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e3f6-00a1-4041-8a06-6e1f2ed8ec49 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.946636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e3f6-00a1-4041-8a06-6e1f2ed8ec49 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:13.947548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1e3f6-00a1-4041-8a06-6e1f2ed8ec49 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:13.949941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1e3f6-00a1-4041-8a06-6e1f2ed8ec49 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:13.987073 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3981/7998] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:13 2022] GET /identity/v3/users => generated 7228 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:13.992311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e1991-fbc8-43e4-885c-f496447cc8bc None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:13.992598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e1991-fbc8-43e4-885c-f496447cc8bc None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:13.992894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e1991-fbc8-43e4-885c-f496447cc8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.074265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-713e1991-fbc8-43e4-885c-f496447cc8bc None None] MFA Rules not processed for user `ae187ae7d9824deaa4ffeb43bc21c33c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:14.075600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-713e1991-fbc8-43e4-885c-f496447cc8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:14.093757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:14.194637 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4018/7999] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:13 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:14.196074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.196559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.196900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.234797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] MFA Rules not processed for user `ae187ae7d9824deaa4ffeb43bc21c33c`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:14.236031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:14.251315 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3982/8000] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:14.255575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e1991-fbc8-43e4-885c-f496447cc8bc None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.267428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.270327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e1991-fbc8-43e4-885c-f496447cc8bc None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.272227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:14.279652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.279893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.280336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.282606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.283527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:create_trust()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.286504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.380686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:14.414666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9885-8b48-4d07-9e35-8a1624b31bc5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.423153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.423721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.425049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.425610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] PATH_INFO: `/v3/projects/ab703d9140c5449d9002eb91439752c6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.427144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab703d9140c5449d9002eb91439752c6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.431459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.450837 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4019/8001] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 10:34:14 2022] POST /identity/v3/OS-TRUST/trusts => generated 859 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:14.456266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.468519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbeefb-7f21-428a-885c-911424df81d2 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.479419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.480276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.480684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.481030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.514935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:get_trust(trust_id=b17ac586ef5b4c5a960b7abcc2a5669e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.517481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.527653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab703d9140c5449d9002eb91439752c6', 'request_id': 'req-980bcdac-9f17-49da-b98e-ab48db5f76ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:14.535085 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4020/8002] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:14 2022] GET /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e => generated 859 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:14.536214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.539147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab703d9140c5449d9002eb91439752c6', 'request_id': 'req-980bcdac-9f17-49da-b98e-ab48db5f76ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:14.548504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83d06-499c-48cd-beca-d0d25bbfb4e8 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.555231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab703d9140c5449d9002eb91439752c6', 'request_id': 'req-980bcdac-9f17-49da-b98e-ab48db5f76ec'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:14.579935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] The token cache is being invalidate because project ab703d9140c5449d9002eb91439752c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:14.580550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:14.582580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-980bcdac-9f17-49da-b98e-ab48db5f76ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:14.585444 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3983/8003] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:14 2022] DELETE /identity/v3/projects/ab703d9140c5449d9002eb91439752c6 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:14.593865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909b6ee-334d-4127-aab3-75eb0b1ff0ec None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.594397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909b6ee-334d-4127-aab3-75eb0b1ff0ec None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.594801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909b6ee-334d-4127-aab3-75eb0b1ff0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.638652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4909b6ee-334d-4127-aab3-75eb0b1ff0ec None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:14.640331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4909b6ee-334d-4127-aab3-75eb0b1ff0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:14.664758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.664758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.664758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.665237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.673169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=b17ac586ef5b4c5a960b7abcc2a5669e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.675598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.693440 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4021/8004] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:34:14 2022] GET /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles => generated 371 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:14.694506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.697351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:14.743957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b5a52-de6c-4ed3-b35f-d0a2c40bf9ab tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.758827 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3984/8005] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:14.759026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.759026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.759026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.759634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/dd3561b254c34e0881f4ea11d5d35181` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.762026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3d1db-b4d0-4a5f-b05a-275ce02fddcf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.762315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3d1db-b4d0-4a5f-b05a-275ce02fddcf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.762637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3d1db-b4d0-4a5f-b05a-275ce02fddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.770337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b17ac586ef5b4c5a960b7abcc2a5669e, role_id=dd3561b254c34e0881f4ea11d5d35181)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.776554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.794378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4022/8006] 10.222.0.26 () {58 vars in 1368 bytes} [Tue Sep 27 10:34:14 2022] GET /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/dd3561b254c34e0881f4ea11d5d35181 => generated 244 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:14.800581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f8d3d1db-b4d0-4a5f-b05a-275ce02fddcf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:14.803139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f8d3d1db-b4d0-4a5f-b05a-275ce02fddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:14.805897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.811945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:14.812336 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3985/8007] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:14.817701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3d1db-b4d0-4a5f-b05a-275ce02fddcf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.823213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d3231-9051-4e67-9ba8-7e3ea6af96f9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.830572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.831362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.831803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.832141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/dd3561b254c34e0881f4ea11d5d35181` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.834194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3d1db-b4d0-4a5f-b05a-275ce02fddcf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.843112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b17ac586ef5b4c5a960b7abcc2a5669e, role_id=dd3561b254c34e0881f4ea11d5d35181)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.845280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.853515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.854061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.854339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.854610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.855456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:14.855761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.859046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.863168 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4023/8008] 10.222.0.26 () {58 vars in 1369 bytes} [Tue Sep 27 10:34:14 2022] HEAD /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/dd3561b254c34e0881f4ea11d5d35181 => generated 0 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 27 10:34:14.867396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.889650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22384051-cf5c-4d95-a38b-9bbd8634cc3f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.895466 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3986/8009] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:14.897452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.907085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.909233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.909793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.910280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/738fbf40fe8b4d2288ca77b16ddac655` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.910635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ca23-fd7c-4cb2-a063-be843e60ec39 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.915498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.916121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.916395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.916975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.917652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.919882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.920243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b17ac586ef5b4c5a960b7abcc2a5669e, role_id=738fbf40fe8b4d2288ca77b16ddac655)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.922284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Could not find role: 738fbf40fe8b4d2288ca77b16ddac655.: keystone.exception.RoleNotFound: Could not find role: 738fbf40fe8b4d2288ca77b16ddac655. Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 10:34:14.945041 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 738fbf40fe8b4d2288ca77b16ddac655. Sep 27 10:34:14.945846 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:14.950238 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4024/8010] 10.222.0.26 () {58 vars in 1368 bytes} [Tue Sep 27 10:34:14 2022] GET /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/738fbf40fe8b4d2288ca77b16ddac655 => generated 110 bytes in 83 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:14.951477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.965575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb925a-f018-47ac-98da-eb30f8d67da3 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:14.972415 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3987/8011] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:14 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:14.972544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:14.972544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `HEAD` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:14.972544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:14.972804 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/738fbf40fe8b4d2288ca77b16ddac655` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:14.977658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:14.978277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:get_role_for_trust(trust_id=b17ac586ef5b4c5a960b7abcc2a5669e, role_id=738fbf40fe8b4d2288ca77b16ddac655)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:14.979870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Could not find role: 738fbf40fe8b4d2288ca77b16ddac655.: keystone.exception.RoleNotFound: Could not find role: 738fbf40fe8b4d2288ca77b16ddac655. Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 10:34:14.991113 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 738fbf40fe8b4d2288ca77b16ddac655. Sep 27 10:34:14.991996 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:14.997703 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4025/8012] 10.222.0.26 () {58 vars in 1369 bytes} [Tue Sep 27 10:34:14 2022] HEAD /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e/roles/738fbf40fe8b4d2288ca77b16ddac655 => generated 0 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 27 10:34:14.998718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.010721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e6c1-2f1d-41cb-84f8-0a5bc09320d0 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.044768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:15.082498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d059d-1975-430a-8560-f0390cae9447 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.091115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.092112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.092535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.093629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.094451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.096895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.127259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.134260 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3988/8013] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:14 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:15.134910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.135294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.135673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.138854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.151982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorizing `identity:delete_trust(trust_id=b17ac586ef5b4c5a960b7abcc2a5669e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.154554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.156369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5528d76-6999-4bc8-89d5-7f610c566e66 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.170183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.170673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.170955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.171232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.172305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.174531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.191442 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3989/8014] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:15 2022] GET /identity/v3/roles => generated 1893 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:15.195339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.213812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'b17ac586ef5b4c5a960b7abcc2a5669e', 'request_id': 'req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:15.217926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'b17ac586ef5b4c5a960b7abcc2a5669e', 'request_id': 'req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:15.221304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20a0a3-62bb-4053-abb7-0fd7f1462a92 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.233208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.234109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.234388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.234823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.235481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.237322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.248320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'b17ac586ef5b4c5a960b7abcc2a5669e', 'request_id': 'req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:15.266764 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3990/8015] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:15 2022] GET /identity/v3/roles => generated 1893 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:15.271572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.274197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:15.278276 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4026/8016] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:34:14 2022] DELETE /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:15.283933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.288617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:15.311154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b88723-4f86-4ed5-ad4b-d00fd5a8e9ce admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.320488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e295ab7-3cb3-4d84-8c20-b4ca0926e1e9 tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.325503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931cbf3-3f8e-4dad-8218-45c65be0acd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.326484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931cbf3-3f8e-4dad-8218-45c65be0acd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.327317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931cbf3-3f8e-4dad-8218-45c65be0acd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.327491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931cbf3-3f8e-4dad-8218-45c65be0acd3 admin admin] PATH_INFO: `/v3/projects/a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0/users/546765f3da1747049d7333aecd5a303f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.329682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae187ae7d9824deaa4ffeb43bc21c33c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3797a8e9ee84e139f83dff18a3fb673', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2115602599', 'tempest-NotDelegatedRole-1417837874'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.330349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.330630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.330908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] PATH_INFO: `/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Could not find trust: b17ac586ef5b4c5a960b7abcc2a5669e.: keystone.exception.TrustNotFound: Could not find trust: b17ac586ef5b4c5a960b7abcc2a5669e. Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 27 10:34:15.361274 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: b17ac586ef5b4c5a960b7abcc2a5669e. Sep 27 10:34:15.362309 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:15.373550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4027/8017] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:15 2022] GET /identity/v3/OS-TRUST/trusts/b17ac586ef5b4c5a960b7abcc2a5669e => generated 111 bytes in 89 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:15.374727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.378785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:15.380507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931cbf3-3f8e-4dad-8218-45c65be0acd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0, user_id=546765f3da1747049d7333aecd5a303f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.385339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931cbf3-3f8e-4dad-8218-45c65be0acd3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.427290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3991/8018] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:15 2022] PUT /identity/v3/projects/a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0/users/546765f3da1747049d7333aecd5a303f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:15.430751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e20190-95a2-42e2-90fb-ed41b8628a3b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.432418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e20190-95a2-42e2-90fb-ed41b8628a3b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.442329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e20190-95a2-42e2-90fb-ed41b8628a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.500171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-83e20190-95a2-42e2-90fb-ed41b8628a3b None None] MFA Rules not processed for user `546765f3da1747049d7333aecd5a303f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:15.501464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-83e20190-95a2-42e2-90fb-ed41b8628a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:15.563898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169f874-396d-4c51-abfe-71dd0a3a079f tempest-TrustsV3TestJSON-151373269 tempest-user-1614048568] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.600936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:15.654807 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3992/8019] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:15.655875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e20190-95a2-42e2-90fb-ed41b8628a3b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.668209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e20190-95a2-42e2-90fb-ed41b8628a3b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.683506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.684171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.684705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.684981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/738fbf40fe8b4d2288ca77b16ddac655` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.685894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.686420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=738fbf40fe8b4d2288ca77b16ddac655)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.686565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.686847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.687273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.687959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:create_application_credential(user_id=546765f3da1747049d7333aecd5a303f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.688276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.689416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.764232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role 738fbf40fe8b4d2288ca77b16ddac655 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:15.764853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:15.768173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:15.773186 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4028/8020] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:15 2022] DELETE /identity/v3/roles/738fbf40fe8b4d2288ca77b16ddac655 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:15.779807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.786304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:15.801281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:15.822776 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3993/8021] 10.222.0.26 () {60 vars in 1315 bytes} [Tue Sep 27 10:34:15 2022] POST /identity/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials => generated 921 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:15.827174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.829271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:15.845538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d8b01-db35-4d5e-a0dc-880a0c34580f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.889183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666bf9-7346-4bad-9701-f2d1dcd681bb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:15.894995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.895725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.895960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.896240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/dd3561b254c34e0881f4ea11d5d35181` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.898001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=dd3561b254c34e0881f4ea11d5d35181)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.901172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.949605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:15.950340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:15.950629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:15.950909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/access_rules/3a6144e540284868b1156ac13ed06435` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:15.952582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:delete_access_rule(user_id=546765f3da1747049d7333aecd5a303f, access_rule_id=3a6144e540284868b1156ac13ed06435)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:15.954238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:15.983452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role dd3561b254c34e0881f4ea11d5d35181 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:15.983742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:15.984471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:15.988718 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4029/8022] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:15 2022] DELETE /identity/v3/roles/dd3561b254c34e0881f4ea11d5d35181 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:15.992695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:15.994709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application session.delete(ref) Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application yield resource Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application next(self.gen) Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:34:16.006402 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise self.value Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application session.commit() Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.session.flush() Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 10:34:16.007498 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application compat.raise_( Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self._synchronize( Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise AssertionError( Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Sep 27 10:34:16.008465 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:16.009591 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:16.010470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Sep 27 10:34:16.010470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Sep 27 10:34:16.010470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Sep 27 10:34:16.010470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 27 10:34:16.010470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 27 10:34:16.010470 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:16.013551 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3994/8023] 10.222.0.26 () {58 vars in 1362 bytes} [Tue Sep 27 10:34:15 2022] DELETE /identity/v3/users/546765f3da1747049d7333aecd5a303f/access_rules/3a6144e540284868b1156ac13ed06435 => generated 89 bytes in 187 msecs (HTTP/1.1 403) 5 headers in 183 bytes (2 switches on core 0) Sep 27 10:34:16.017442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.019887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:16.036044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67528c-74f1-4b94-a552-ff08985518c5 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.103048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:16.125464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.126801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.128726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.129082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials/5c30c98b976d470d9f2d240d12dcd792` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.132688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838f74-e6e3-40e7-99a0-4bf67fa67bdc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.151960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.153022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.153500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.153967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users/ae187ae7d9824deaa4ffeb43bc21c33c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.155919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ae187ae7d9824deaa4ffeb43bc21c33c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.157402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:delete_application_credential(user_id=546765f3da1747049d7333aecd5a303f, application_credential_id=5c30c98b976d470d9f2d240d12dcd792)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.158920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.161620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.220353 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3995/8024] 10.222.0.26 () {58 vars in 1395 bytes} [Tue Sep 27 10:34:16 2022] DELETE /identity/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials/5c30c98b976d470d9f2d240d12dcd792 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:16.227108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.248723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41538186-cbe7-4b71-8f09-d6d882b7b446 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.264299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.266483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.267817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.267950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/access_rules` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.269502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:list_access_rules(user_id=546765f3da1747049d7333aecd5a303f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.271955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.281715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3996/8025] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:34:16 2022] GET /identity/v3/users/546765f3da1747049d7333aecd5a303f/access_rules => generated 369 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:16.285820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.287693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae187ae7d9824deaa4ffeb43bc21c33c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.288631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae187ae7d9824deaa4ffeb43bc21c33c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.305330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3001f-36a1-4152-b1a7-dcf1f7bb4436 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.311807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae187ae7d9824deaa4ffeb43bc21c33c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.312605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.313745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.314254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.314757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/access_rules/3a6144e540284868b1156ac13ed06435` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.316696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:delete_access_rule(user_id=546765f3da1747049d7333aecd5a303f, access_rule_id=3a6144e540284868b1156ac13ed06435)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.321887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.325290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae187ae7d9824deaa4ffeb43bc21c33c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.327005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae187ae7d9824deaa4ffeb43bc21c33c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.352225 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3997/8026] 10.222.0.26 () {58 vars in 1362 bytes} [Tue Sep 27 10:34:16 2022] DELETE /identity/v3/users/546765f3da1747049d7333aecd5a303f/access_rules/3a6144e540284868b1156ac13ed06435 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:16.356426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.360492 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:16.372444 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4030/8027] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:15 2022] DELETE /identity/v3/users/ae187ae7d9824deaa4ffeb43bc21c33c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:16.376300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.380471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:16.408179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e4481-2164-4807-b5cc-276e340f263c tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.474507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:16.490500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.491225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.491516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.491887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/access_rules` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.493397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5eb67-97ab-4b56-bb0b-717d5f539bff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.494316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:list_access_rules(user_id=546765f3da1747049d7333aecd5a303f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.496541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.499532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.500433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.501077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.501658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/d3797a8e9ee84e139f83dff18a3fb673` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.503609 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3998/8028] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:34:16 2022] GET /identity/v3/users/546765f3da1747049d7333aecd5a303f/access_rules => generated 158 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:16.507201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d3797a8e9ee84e139f83dff18a3fb673)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.508781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.511654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.534679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ccd53-65b0-470f-a29f-9d511d4b4d54 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.547602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.548289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.548573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.548872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.554958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:create_application_credential(user_id=546765f3da1747049d7333aecd5a303f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.557327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.635432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3797a8e9ee84e139f83dff18a3fb673', 'request_id': 'req-d74c5acb-a333-4a36-95c0-216fec9abfe7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.653221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3797a8e9ee84e139f83dff18a3fb673', 'request_id': 'req-d74c5acb-a333-4a36-95c0-216fec9abfe7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.660748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3797a8e9ee84e139f83dff18a3fb673', 'request_id': 'req-d74c5acb-a333-4a36-95c0-216fec9abfe7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.674122 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:16.681986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] The token cache is being invalidate because project d3797a8e9ee84e139f83dff18a3fb673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:16.682963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.685227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:16.687873 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4031/8029] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:16 2022] DELETE /identity/v3/projects/d3797a8e9ee84e139f83dff18a3fb673 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:16.694608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.706410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:16.745555 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 3999/8030] 10.222.0.26 () {60 vars in 1315 bytes} [Tue Sep 27 10:34:16 2022] POST /identity/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials => generated 920 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:16.757801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.759962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:16.782821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0046f-9039-4db0-9897-8cab70dc29b4 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.823145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c5acb-a333-4a36-95c0-216fec9abfe7 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.834074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.834849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.835132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.835403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.836534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.838987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.885993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:16.888532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:16.889056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:16.890090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/access_rules` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:16.892113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:list_access_rules(user_id=546765f3da1747049d7333aecd5a303f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:16.893888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:16.895512 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4032/8031] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:16 2022] POST /identity/v3/projects => generated 328 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:16.899494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.904996 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4000/8032] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:34:16 2022] GET /identity/v3/users/546765f3da1747049d7333aecd5a303f/access_rules => generated 369 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:16.910863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:16.925841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971b73a-dabf-4caf-8ef2-346ac4130034 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:16.996693 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde70033-5892-4de6-b4ca-22a65c5124b4 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.005004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.006329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.007252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.008125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.009028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.011729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.014799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.016706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.025585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.026488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.028167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:create_application_credential(user_id=546765f3da1747049d7333aecd5a303f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.029977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.048632 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4033/8033] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:16 2022] POST /identity/v3/users => generated 416 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:17.049732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.066619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011683a-6f7f-4ea6-ba10-08ba1f0d232a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.072933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.073777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.074387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.074765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.075839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.077757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.088764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:17.105508 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4034/8034] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:17 2022] POST /identity/v3/roles => generated 244 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:17.107351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.125888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bcb9f-e772-4850-92e5-717c19fe924c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.132199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.133005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.133459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.133812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.135436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.137317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.164689 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4035/8035] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:17 2022] POST /identity/v3/roles => generated 247 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:17.169039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.171572 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4001/8036] 10.222.0.26 () {60 vars in 1315 bytes} [Tue Sep 27 10:34:16 2022] POST /identity/v3/users/546765f3da1747049d7333aecd5a303f/application_credentials => generated 921 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:17.175566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.189281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8b3fa-bc6e-4609-b884-4fd52bc2c3db tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.191703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bd1a3-7f39-4bbc-8bc0-941cb59eece2 tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.198720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.199462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.199849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.200238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/ffd886063cc743018a34d5ac7df8cc4f/users/e4c8e7dd6ef8439d8668854e7b14819f/roles/cc17525b50fa453890ee2b6e980ff1d2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.201377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '546765f3da1747049d7333aecd5a303f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.202370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.202940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.203388 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f/access_rules/947c8e81ae6d4eaf87d6cf2a063ab8f0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.204915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorizing `identity:get_access_rule(user_id=546765f3da1747049d7333aecd5a303f, access_rule_id=947c8e81ae6d4eaf87d6cf2a063ab8f0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.207089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.215965 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4002/8037] 10.222.0.26 () {58 vars in 1359 bytes} [Tue Sep 27 10:34:17 2022] GET /identity/v3/users/546765f3da1747049d7333aecd5a303f/access_rules/947c8e81ae6d4eaf87d6cf2a063ab8f0 => generated 229 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:17.220717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.224162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:17.227296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ffd886063cc743018a34d5ac7df8cc4f, user_id=e4c8e7dd6ef8439d8668854e7b14819f, role_id=cc17525b50fa453890ee2b6e980ff1d2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.230692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.248453 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4036/8038] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:17 2022] PUT /identity/v3/projects/ffd886063cc743018a34d5ac7df8cc4f/users/e4c8e7dd6ef8439d8668854e7b14819f/roles/cc17525b50fa453890ee2b6e980ff1d2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:17.252413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.292353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:17.309181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30604507-cc9e-4107-9f4d-c09434d4237f tempest-AccessRulesV3Test-1095931794 tempest-AccessRulesV3Test-1095931794-project] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.348580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a1c80-8bb1-4ea6-9481-7a046a9cbf2e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.355110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.355800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.356076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.356352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/ffd886063cc743018a34d5ac7df8cc4f/users/e4c8e7dd6ef8439d8668854e7b14819f/roles/6154c5b4a6b94984876ec1e6a6357278` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.364997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=ffd886063cc743018a34d5ac7df8cc4f, user_id=e4c8e7dd6ef8439d8668854e7b14819f, role_id=6154c5b4a6b94984876ec1e6a6357278)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.370269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.395024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.396650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.397212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.397661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] PATH_INFO: `/v3/users/546765f3da1747049d7333aecd5a303f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.404027 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4037/8039] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:17 2022] PUT /identity/v3/projects/ffd886063cc743018a34d5ac7df8cc4f/users/e4c8e7dd6ef8439d8668854e7b14819f/roles/6154c5b4a6b94984876ec1e6a6357278 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:17.404719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546765f3da1747049d7333aecd5a303f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.407275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.408344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.519183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546765f3da1747049d7333aecd5a303f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:17.520581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546765f3da1747049d7333aecd5a303f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:17.538513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bd73e-2afb-424a-b1cd-9bf3c6db0d88 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.542222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546765f3da1747049d7333aecd5a303f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:17.546851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:17.549501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4fb7e-66cf-47e6-9d8a-36686c95a773 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.550430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4fb7e-66cf-47e6-9d8a-36686c95a773 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.551033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4fb7e-66cf-47e6-9d8a-36686c95a773 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.551494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4fb7e-66cf-47e6-9d8a-36686c95a773 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.552179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4fb7e-66cf-47e6-9d8a-36686c95a773 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.554250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4fb7e-66cf-47e6-9d8a-36686c95a773 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.559933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546765f3da1747049d7333aecd5a303f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:17.591065 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4038/8040] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:17 2022] GET /identity/v3/users => generated 7228 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:17.598441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f88f9-b91a-470f-9d49-e03069a0698f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.598906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f88f9-b91a-470f-9d49-e03069a0698f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.599422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f88f9-b91a-470f-9d49-e03069a0698f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.638174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-413e28a3-d79e-4965-8ea5-de7e5b3bd467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546765f3da1747049d7333aecd5a303f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:17.645476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-346f88f9-b91a-470f-9d49-e03069a0698f None None] MFA Rules not processed for user `e4c8e7dd6ef8439d8668854e7b14819f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:17.646815 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4003/8041] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:17 2022] DELETE /identity/v3/users/546765f3da1747049d7333aecd5a303f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:17.648881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-346f88f9-b91a-470f-9d49-e03069a0698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:17.651592 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.651944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.652327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.674810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:17.676156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:17.746514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4039/8042] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:17.753661 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4004/8043] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:17.758940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415fb14-85be-4e1b-b9f5-ef3fbea56020 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.759234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415fb14-85be-4e1b-b9f5-ef3fbea56020 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.759515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415fb14-85be-4e1b-b9f5-ef3fbea56020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.765482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.767128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:17.785094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a415fb14-85be-4e1b-b9f5-ef3fbea56020 None None] MFA Rules not processed for user `e4c8e7dd6ef8439d8668854e7b14819f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:17.786423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a415fb14-85be-4e1b-b9f5-ef3fbea56020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:17.797463 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4040/8044] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:17.800910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415fb14-85be-4e1b-b9f5-ef3fbea56020 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.812957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415fb14-85be-4e1b-b9f5-ef3fbea56020 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.818757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4c8e7dd6ef8439d8668854e7b14819f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffd886063cc743018a34d5ac7df8cc4f', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1701858663', 'tempest-NotDelegatedRole-1617782234'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.819701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.820057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.821056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.822445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] RBAC: Authorizing `identity:create_trust()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.823775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.854169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858876d-6903-4ce9-891a-dd348acf4b9c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:17.857869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:17.864572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:17.865396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:17.865717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:17.866544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:17.879215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:17.881421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:17.896067 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4005/8045] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Sep 27 10:34:17.933787 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise ValueError(str(e)) Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Sep 27 10:34:17.935071 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Sep 27 10:34:17.936561 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Sep 27 10:34:17.936561 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Sep 27 10:34:17.936561 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Sep 27 10:34:17.936561 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Sep 27 10:34:17.936561 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 27 10:34:17.936561 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:17.936561 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4041/8046] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 10:34:17 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 135 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 27 10:34:17.939780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:17.941406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:18.022907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:18.042767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff438dc9-9fa4-4af4-8db1-d302b6fd3eeb service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.050313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.051205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.051586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.052030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/6154c5b4a6b94984876ec1e6a6357278` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.054324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6154c5b4a6b94984876ec1e6a6357278)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.057124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.133180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role 6154c5b4a6b94984876ec1e6a6357278 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:18.133527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.134245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.136547 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4006/8047] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:17 2022] DELETE /identity/v3/roles/6154c5b4a6b94984876ec1e6a6357278 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:18.140538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.142688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:18.162453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7ed3-2fe7-4e66-9cd9-c62a88429bfc tempest-TrustsV3TestJSON-420854002 tempest-user-1919077732] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.242850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.243613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.243787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.244705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/cc17525b50fa453890ee2b6e980ff1d2` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.246245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=cc17525b50fa453890ee2b6e980ff1d2)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.247892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.333139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role cc17525b50fa453890ee2b6e980ff1d2 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:18.333449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.335018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.338165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4042/8048] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:18 2022] DELETE /identity/v3/roles/cc17525b50fa453890ee2b6e980ff1d2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:18.343873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.353537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.357252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:18.367216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eafe6b-1af5-49bc-93dc-cd6d67714aff tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.477204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.477904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.478119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.478398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users/e4c8e7dd6ef8439d8668854e7b14819f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.479728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e4c8e7dd6ef8439d8668854e7b14819f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.481731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.506513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef389d-998a-4171-ab06-d729eccece7c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.511711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.512452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.512848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.513180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] PATH_INFO: `/v3/projects/a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.514512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.516980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.549220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c8e7dd6ef8439d8668854e7b14819f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.550011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c8e7dd6ef8439d8668854e7b14819f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.564583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c8e7dd6ef8439d8668854e7b14819f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.569591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:18.580563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c8e7dd6ef8439d8668854e7b14819f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.582919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'request_id': 'req-93721f7e-5a9a-451b-8611-9f66545d053d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.595747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'request_id': 'req-93721f7e-5a9a-451b-8611-9f66545d053d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.600745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c8e7dd6ef8439d8668854e7b14819f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.609163 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4007/8049] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:18 2022] DELETE /identity/v3/users/e4c8e7dd6ef8439d8668854e7b14819f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:18.613142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0', 'request_id': 'req-93721f7e-5a9a-451b-8611-9f66545d053d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.614458 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.619094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:18.635188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] The token cache is being invalidate because project a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:18.635600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.636970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-93721f7e-5a9a-451b-8611-9f66545d053d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.638788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4043/8050] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:18 2022] DELETE /identity/v3/projects/a3cfdb747dbf4e0e9b6b8eb4ddd6c4b0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:18.644993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf71a-7d57-4f0f-ac41-fb9ac0345ded None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.645177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf71a-7d57-4f0f-ac41-fb9ac0345ded None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.645403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf71a-7d57-4f0f-ac41-fb9ac0345ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.670044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-72fbf71a-7d57-4f0f-ac41-fb9ac0345ded None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:18.671440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-72fbf71a-7d57-4f0f-ac41-fb9ac0345ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:18.682340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:18.714426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4b0f-9719-4ff3-975b-b5074594bc9e tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.724269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.725146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.725438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.725791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/ffd886063cc743018a34d5ac7df8cc4f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.727039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ffd886063cc743018a34d5ac7df8cc4f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.729079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.767925 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4044/8051] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:18.769641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449f83-8a0c-4fd9-949c-a6d4e1479f64 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.770048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449f83-8a0c-4fd9-949c-a6d4e1479f64 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.770325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449f83-8a0c-4fd9-949c-a6d4e1479f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.786168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd886063cc743018a34d5ac7df8cc4f', 'request_id': 'req-26ee5534-0560-4da8-81b5-60a817b64a2f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.793072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd886063cc743018a34d5ac7df8cc4f', 'request_id': 'req-26ee5534-0560-4da8-81b5-60a817b64a2f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.796069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-57449f83-8a0c-4fd9-949c-a6d4e1479f64 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:18.797321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57449f83-8a0c-4fd9-949c-a6d4e1479f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:18.803578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd886063cc743018a34d5ac7df8cc4f', 'request_id': 'req-26ee5534-0560-4da8-81b5-60a817b64a2f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.818704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] The token cache is being invalidate because project ffd886063cc743018a34d5ac7df8cc4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:18.819097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.820319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:18.823779 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4008/8052] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:18 2022] DELETE /identity/v3/projects/ffd886063cc743018a34d5ac7df8cc4f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:18.828949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.830544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:18.879584 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4045/8053] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:18.885295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57449f83-8a0c-4fd9-949c-a6d4e1479f64 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.898294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57449f83-8a0c-4fd9-949c-a6d4e1479f64 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.904925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.905896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.906236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.906815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.907700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:18.907841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee5534-0560-4da8-81b5-60a817b64a2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.908118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.909945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.913451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.914943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.915722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.915722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.916196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.918155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.927121 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4046/8054] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:18.928212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.942494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91792500-f27e-41d4-a791-9f5360cc7ee6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:18.949198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:18.949872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:18.950152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:18.950424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:18.951190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:18.953261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:18.965736 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4009/8055] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:18 2022] POST /identity/v3/projects => generated 327 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:18.966726 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:18.992567 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4047/8056] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:18 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:18.994607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.034543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:19.035019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:19.051999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d382-4b9d-462f-be5a-9c1ea5069c2f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.084850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c24f-cc09-47e9-bb62-797d8018170d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.098962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.099938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.100249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.100518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.101418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.103610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.135928 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4048/8057] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:18 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:19.141460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.158927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab467b0-c17b-42a8-afa9-786928c4d7eb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.159594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.160747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.161235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.161808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.162929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.163361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.163976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.164253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.164582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.165260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.166235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.166848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.184331 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4049/8058] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:19.185663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.197931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f0605-03e5-4d0b-8401-4565eeb62200 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.204570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.205246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.205521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.205866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.207190 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4010/8059] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:18 2022] POST /identity/v3/users => generated 416 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:19.210948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.213520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.213746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.225993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b4d33-d7c7-4d3c-a6bc-21788caa4b76 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.228872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4050/8060] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:19.230606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.234390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.235352 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.235641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.235924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.237049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.238303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.241733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b77b48-1b81-4221-b495-daf3ba1b26c8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.247469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f3287-c1e7-48b0-b794-e162d965e46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.248291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f3287-c1e7-48b0-b794-e162d965e46f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.248626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f3287-c1e7-48b0-b794-e162d965e46f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.249004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f3287-c1e7-48b0-b794-e162d965e46f admin admin] PATH_INFO: `/v3/projects/3e5c1efc14334e50bff5677f9cfe8bfc/users/a3e568205fa04e939219de9ef8bf5773/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.258357 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4011/8061] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:19 2022] POST /identity/v3/roles => generated 244 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:19.262717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.269456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f3287-c1e7-48b0-b794-e162d965e46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5c1efc14334e50bff5677f9cfe8bfc, user_id=a3e568205fa04e939219de9ef8bf5773, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.272393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f3287-c1e7-48b0-b794-e162d965e46f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.275302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac7e8c-e3fb-4b71-9a84-db1a7652b5d5 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.281330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.281946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.282224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.282461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.283128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.284872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.289042 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4051/8062] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:19 2022] PUT /identity/v3/projects/3e5c1efc14334e50bff5677f9cfe8bfc/users/a3e568205fa04e939219de9ef8bf5773/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:19.300252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.300537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.300858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.305655 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4012/8063] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:19 2022] POST /identity/v3/roles => generated 247 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:19.309539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.320617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0beec-76ac-4c6f-a16e-1b6228e7713b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.333022 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] MFA Rules not processed for user `a3e568205fa04e939219de9ef8bf5773`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:19.334702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:19.387848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.388560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.388882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.389152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/dedcb360f2524fd6bc7cdf0b30264c08/users/0d2366c0dfd445afaeeeb255bbf9de55/roles/f02b836304984d87a8a2d4f5c0ac1af1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.406150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=dedcb360f2524fd6bc7cdf0b30264c08, user_id=0d2366c0dfd445afaeeeb255bbf9de55, role_id=f02b836304984d87a8a2d4f5c0ac1af1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.409089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.420202 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4052/8064] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:19.426479 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4013/8065] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:19 2022] PUT /identity/v3/projects/dedcb360f2524fd6bc7cdf0b30264c08/users/0d2366c0dfd445afaeeeb255bbf9de55/roles/f02b836304984d87a8a2d4f5c0ac1af1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:19.428994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.431164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:19.431573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.489631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:19.507306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9f00b-db9f-4171-8c62-753e7b57927c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.511392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041991a-0c25-43dc-9285-34728d77c8b8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.512566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.513269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.513551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.513837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/dedcb360f2524fd6bc7cdf0b30264c08/users/0d2366c0dfd445afaeeeb255bbf9de55/roles/05e00e9cd39343c6996a4e161368909f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.516140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=dedcb360f2524fd6bc7cdf0b30264c08, user_id=0d2366c0dfd445afaeeeb255bbf9de55, role_id=05e00e9cd39343c6996a4e161368909f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.518932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.521407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:19.543791 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4014/8066] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:19 2022] PUT /identity/v3/projects/dedcb360f2524fd6bc7cdf0b30264c08/users/0d2366c0dfd445afaeeeb255bbf9de55/roles/05e00e9cd39343c6996a4e161368909f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:19.547210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.594501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbf2ba-06d4-41fe-b0ce-fa1f9da1439e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.595127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbf2ba-06d4-41fe-b0ce-fa1f9da1439e service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.595403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbf2ba-06d4-41fe-b0ce-fa1f9da1439e service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.595664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbf2ba-06d4-41fe-b0ce-fa1f9da1439e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.654940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b4f14-5670-486b-899b-8ae22aef8a23 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.663346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752d78e-5b81-4ab7-9440-0f398398477c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.663983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752d78e-5b81-4ab7-9440-0f398398477c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.664765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752d78e-5b81-4ab7-9440-0f398398477c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.665870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752d78e-5b81-4ab7-9440-0f398398477c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.666640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752d78e-5b81-4ab7-9440-0f398398477c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.668540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752d78e-5b81-4ab7-9440-0f398398477c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.685273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbf2ba-06d4-41fe-b0ce-fa1f9da1439e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.687603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbf2ba-06d4-41fe-b0ce-fa1f9da1439e service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:19.698218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4015/8067] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:19 2022] GET /identity/v3/users => generated 7509 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:19.703402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da231a-35f8-4e4b-bcc1-473971dcaa2d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.703693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da231a-35f8-4e4b-bcc1-473971dcaa2d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.704088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da231a-35f8-4e4b-bcc1-473971dcaa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.718013 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4053/8068] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:19.765852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-22da231a-35f8-4e4b-bcc1-473971dcaa2d None None] MFA Rules not processed for user `0d2366c0dfd445afaeeeb255bbf9de55`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:19.767306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-22da231a-35f8-4e4b-bcc1-473971dcaa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:19.881618 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4016/8069] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:19.892742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2103c6f-2c23-40f0-93b7-a590af723a0d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.893597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2103c6f-2c23-40f0-93b7-a590af723a0d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.894067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2103c6f-2c23-40f0-93b7-a590af723a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.926539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b2103c6f-2c23-40f0-93b7-a590af723a0d None None] MFA Rules not processed for user `0d2366c0dfd445afaeeeb255bbf9de55`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:19.928805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b2103c6f-2c23-40f0-93b7-a590af723a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:19.945277 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4054/8070] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:19.950016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da231a-35f8-4e4b-bcc1-473971dcaa2d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:19.969130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da231a-35f8-4e4b-bcc1-473971dcaa2d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:19.974302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:19.975014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:19.975792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:19.975792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:19.976531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:create_trust()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:19.978735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.081484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2103c6f-2c23-40f0-93b7-a590af723a0d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.147793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:20.166524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2103c6f-2c23-40f0-93b7-a590af723a0d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.184070 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4017/8071] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 10:34:19 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:20.189014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.190034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.190769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.191050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.191467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] PATH_INFO: `/v3/users/a3e568205fa04e939219de9ef8bf5773` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.197188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e568205fa04e939219de9ef8bf5773)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.199173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.212704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8b0c-d41c-4ab3-b3df-2295a3b9cdd3 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.229901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.230581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.231568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.231857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.245953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:get_trust(trust_id=6aa75ac0c7a34ad58bac8632efc5e8d6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.247455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.265766 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4018/8072] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:20 2022] GET /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6 => generated 834 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:20.279957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.293303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477dcc6-2988-4d41-9b7b-265a2e51050c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.298558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.299526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.299965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.300450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.300600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e568205fa04e939219de9ef8bf5773'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.301279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e568205fa04e939219de9ef8bf5773'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.308446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=6aa75ac0c7a34ad58bac8632efc5e8d6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.310160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.315454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e568205fa04e939219de9ef8bf5773'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.318802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4019/8073] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:34:20 2022] GET /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles => generated 371 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:20.323011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:20.324237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.326505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e568205fa04e939219de9ef8bf5773'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.326622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:20.327067 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2cbfefc8-f379-4c75-9f3c-24d010890442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e568205fa04e939219de9ef8bf5773'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.343968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9aab3-946f-42af-9301-35db9f57c80c tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.349855 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4055/8074] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:20 2022] DELETE /identity/v3/users/a3e568205fa04e939219de9ef8bf5773 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:20.351332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.352079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.352357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.352640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/f02b836304984d87a8a2d4f5c0ac1af1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.356251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.356547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.356853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.361748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6aa75ac0c7a34ad58bac8632efc5e8d6, role_id=f02b836304984d87a8a2d4f5c0ac1af1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.363333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.376098 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4020/8075] 10.222.0.26 () {58 vars in 1368 bytes} [Tue Sep 27 10:34:20 2022] GET /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/f02b836304984d87a8a2d4f5c0ac1af1 => generated 244 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:20.377442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.379441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:20.380688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:20.388260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2d3a9-4180-4f09-af90-7fa81dc44691 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.452751 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4056/8076] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:20.452950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.452950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.452950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.453248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/f02b836304984d87a8a2d4f5c0ac1af1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.459320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.461262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:20.462480 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6aa75ac0c7a34ad58bac8632efc5e8d6, role_id=f02b836304984d87a8a2d4f5c0ac1af1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.464564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.473171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4021/8077] 10.222.0.26 () {58 vars in 1369 bytes} [Tue Sep 27 10:34:20 2022] HEAD /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/f02b836304984d87a8a2d4f5c0ac1af1 => generated 0 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 27 10:34:20.477521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.490092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a79eb2-c9b1-406b-a1e0-825d074e2157 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.495581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.496469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.496954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.497407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/05e00e9cd39343c6996a4e161368909f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.503836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6aa75ac0c7a34ad58bac8632efc5e8d6, role_id=05e00e9cd39343c6996a4e161368909f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.505339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Could not find role: 05e00e9cd39343c6996a4e161368909f.: keystone.exception.RoleNotFound: Could not find role: 05e00e9cd39343c6996a4e161368909f. Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 10:34:20.516878 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 05e00e9cd39343c6996a4e161368909f. Sep 27 10:34:20.517765 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:20.521301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4022/8078] 10.222.0.26 () {58 vars in 1368 bytes} [Tue Sep 27 10:34:20 2022] GET /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/05e00e9cd39343c6996a4e161368909f => generated 110 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:34:20.522383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.529233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977a4b5-7283-433c-a903-6179a4f764ed None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.532709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd4388-0f2b-4e72-b5f7-8c5b8610406a tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.533514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.534331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.534611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.534886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.537189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.537887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.538169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.538449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/05e00e9cd39343c6996a4e161368909f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.545545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6aa75ac0c7a34ad58bac8632efc5e8d6, role_id=05e00e9cd39343c6996a4e161368909f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.546981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.547082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.548503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Could not find role: 05e00e9cd39343c6996a4e161368909f.: keystone.exception.RoleNotFound: Could not find role: 05e00e9cd39343c6996a4e161368909f. Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 10:34:20.557614 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 05e00e9cd39343c6996a4e161368909f. Sep 27 10:34:20.558447 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:20.559329 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4023/8079] 10.222.0.26 () {58 vars in 1369 bytes} [Tue Sep 27 10:34:20 2022] HEAD /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6/roles/05e00e9cd39343c6996a4e161368909f => generated 0 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:20.561856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4057/8080] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:20.563481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.573911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e841f-c1be-4dfd-a970-c75a1ae4921d tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.578297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.578944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.579220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.579499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.586290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorizing `identity:delete_trust(trust_id=6aa75ac0c7a34ad58bac8632efc5e8d6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.588609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.618430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '6aa75ac0c7a34ad58bac8632efc5e8d6', 'request_id': 'req-90d0f3d0-a302-4b95-9044-c020232a5df8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.636347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '6aa75ac0c7a34ad58bac8632efc5e8d6', 'request_id': 'req-90d0f3d0-a302-4b95-9044-c020232a5df8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.637035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '6aa75ac0c7a34ad58bac8632efc5e8d6', 'request_id': 'req-90d0f3d0-a302-4b95-9044-c020232a5df8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.646035 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4024/8081] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:34:20 2022] DELETE /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:20.650782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.663558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5abf5b-331f-4a24-b814-61f053150bba service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.670340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d2366c0dfd445afaeeeb255bbf9de55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dedcb360f2524fd6bc7cdf0b30264c08', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1481909205', 'tempest-DelegatedRole-1723469716'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.670998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.671326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.671607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] PATH_INFO: `/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Could not find trust: 6aa75ac0c7a34ad58bac8632efc5e8d6.: keystone.exception.TrustNotFound: Could not find trust: 6aa75ac0c7a34ad58bac8632efc5e8d6. Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 27 10:34:20.682664 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 6aa75ac0c7a34ad58bac8632efc5e8d6. Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: ERROR keystone.server.flask.application Sep 27 10:34:20.684538 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4058/8082] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:20 2022] GET /identity/v3/OS-TRUST/trusts/6aa75ac0c7a34ad58bac8632efc5e8d6 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:20.688691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.691076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:20.720408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:20.800871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0f3d0-a302-4b95-9044-c020232a5df8 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.807561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:20.808493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:20.808858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:20.809208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/05e00e9cd39343c6996a4e161368909f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:20.810835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=05e00e9cd39343c6996a4e161368909f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:20.812537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:20.863957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role 05e00e9cd39343c6996a4e161368909f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:20.864432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.865309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:20.869002 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4025/8083] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:20 2022] DELETE /identity/v3/roles/05e00e9cd39343c6996a4e161368909f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:20.872999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.875500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:20.902421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab70276-e4a7-4259-b1f1-57f0fa916514 tempest-TrustsV3TestJSON-58384194 tempest-user-1316091603] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:20.915857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:20.917490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.010510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.011381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.011706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.012039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/f02b836304984d87a8a2d4f5c0ac1af1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.013665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f02b836304984d87a8a2d4f5c0ac1af1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.015675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.042769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:21.050640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bc6ed-6ff4-48a4-8868-6b2b834a2677 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.057977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.059204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.059204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.059636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] PATH_INFO: `/v3/projects/3e5c1efc14334e50bff5677f9cfe8bfc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.061843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5c1efc14334e50bff5677f9cfe8bfc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.064819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.100222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role f02b836304984d87a8a2d4f5c0ac1af1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:21.100515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.102887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.112603 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4059/8084] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:20 2022] DELETE /identity/v3/roles/f02b836304984d87a8a2d4f5c0ac1af1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:21.115884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.118751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.146149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5c1efc14334e50bff5677f9cfe8bfc', 'request_id': 'req-7b954785-1544-46f4-a0fd-85b691ee8596'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.153893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5c1efc14334e50bff5677f9cfe8bfc', 'request_id': 'req-7b954785-1544-46f4-a0fd-85b691ee8596'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.167656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5c1efc14334e50bff5677f9cfe8bfc', 'request_id': 'req-7b954785-1544-46f4-a0fd-85b691ee8596'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.192541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] The token cache is being invalidate because project 3e5c1efc14334e50bff5677f9cfe8bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:21.192941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.196930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7b954785-1544-46f4-a0fd-85b691ee8596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.200054 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4026/8085] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:20 2022] DELETE /identity/v3/projects/3e5c1efc14334e50bff5677f9cfe8bfc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:21.205434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ef314-467d-463a-9775-d9ec1158e610 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.206446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ef314-467d-463a-9775-d9ec1158e610 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.207564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ef314-467d-463a-9775-d9ec1158e610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.234666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-939ef314-467d-463a-9775-d9ec1158e610 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:21.236475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-939ef314-467d-463a-9775-d9ec1158e610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.243757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:21.248395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ef07e-1609-46a9-8a3f-32a51f75442c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.357141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.358008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.358292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.358569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users/0d2366c0dfd445afaeeeb255bbf9de55` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.359750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0d2366c0dfd445afaeeeb255bbf9de55)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.362124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.376997 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4027/8086] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:21.385245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d58d-3a2e-43c5-94eb-558f742be9c4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.385653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d58d-3a2e-43c5-94eb-558f742be9c4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.385937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d58d-3a2e-43c5-94eb-558f742be9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.429959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0724d58d-3a2e-43c5-94eb-558f742be9c4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:21.432230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0724d58d-3a2e-43c5-94eb-558f742be9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.443921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2366c0dfd445afaeeeb255bbf9de55'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.443921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2366c0dfd445afaeeeb255bbf9de55'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.444301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4028/8087] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:21.445503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724d58d-3a2e-43c5-94eb-558f742be9c4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.454950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2366c0dfd445afaeeeb255bbf9de55'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.456751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724d58d-3a2e-43c5-94eb-558f742be9c4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.463238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.464137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.464865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.465222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.466298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:21.466930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.469204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.469323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2366c0dfd445afaeeeb255bbf9de55'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.470382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2366c0dfd445afaeeeb255bbf9de55'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.490463 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4029/8088] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:21.491615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.494255 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.501358 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4060/8089] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:21 2022] DELETE /identity/v3/users/0d2366c0dfd445afaeeeb255bbf9de55 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:21.502329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.504250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.511500 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571c1f6-d374-4563-8d41-d87a1e9d35e6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.515596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:21.586000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.586633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.586916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.587192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.587859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.590068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.595869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de9d40-f426-44c9-a5ec-ca7f2c0f859a tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.604949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.604949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.604949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.605267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/dedcb360f2524fd6bc7cdf0b30264c08` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.612958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dedcb360f2524fd6bc7cdf0b30264c08)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.615658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.636433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4030/8090] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:21 2022] POST /identity/v3/projects => generated 405 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:21.640621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.697406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dedcb360f2524fd6bc7cdf0b30264c08', 'request_id': 'req-327c885b-332e-4743-8161-1992918c38cb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.711436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dedcb360f2524fd6bc7cdf0b30264c08', 'request_id': 'req-327c885b-332e-4743-8161-1992918c38cb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.720769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dedcb360f2524fd6bc7cdf0b30264c08', 'request_id': 'req-327c885b-332e-4743-8161-1992918c38cb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.722400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b902a-6960-47f7-ac71-cce1058709fe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.737903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] The token cache is being invalidate because project dedcb360f2524fd6bc7cdf0b30264c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:21.738204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.738995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:21.741735 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4061/8091] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:21 2022] DELETE /identity/v3/projects/dedcb360f2524fd6bc7cdf0b30264c08 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:21.748706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.750358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.751923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:21.811897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.812546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.812871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.813152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.813795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.815762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.851060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c885b-332e-4743-8161-1992918c38cb tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.851970 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4031/8092] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:21 2022] POST /identity/v3/users => generated 303 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:21.854177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.856348 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:21.858062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.858904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.860033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.860033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.860709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.862718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.889007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65effa-1ba9-4616-8a17-92763223e741 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:21.905599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:21.907009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:21.907789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:21.908075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:21.908914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:21.910682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:21.914090 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4062/8093] 10.222.0.26 () {60 vars in 1152 bytes} [Tue Sep 27 10:34:21 2022] POST /identity/v3/projects => generated 329 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:21.918282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.926392 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4032/8094] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:21.931164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:21.942528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:21.945769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a525b8b-c571-4850-ba0e-0ad398072eeb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.005229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c25aa-1090-4237-b50d-41655a006616 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.015927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.016905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.017421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.017928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.018748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.021635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.035623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.036412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.036688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.037503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.038172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.040250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.052921 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4033/8095] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:22.056598 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4063/8096] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:21 2022] POST /identity/v3/users => generated 410 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:22.057580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.061288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.069289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaad731-b1f2-49c1-9ee2-92d7d1c61d6e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.074423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.075083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.076240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.076240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] PATH_INFO: `/v3/projects/823a80e1f96b49848a54c92488ff36b6/users/9af0dab0ba424338932197c368daf988/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.086446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617016a5-9b0a-4fad-b3b7-3f5dae1b9f14 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.095110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1df0d9-d542-49cf-aff2-44323b1c56ec tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.095733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1df0d9-d542-49cf-aff2-44323b1c56ec tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.096019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=823a80e1f96b49848a54c92488ff36b6, user_id=9af0dab0ba424338932197c368daf988, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.097107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1df0d9-d542-49cf-aff2-44323b1c56ec tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.097437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1df0d9-d542-49cf-aff2-44323b1c56ec tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.099906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.103026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1df0d9-d542-49cf-aff2-44323b1c56ec tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.104713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1df0d9-d542-49cf-aff2-44323b1c56ec tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.120953 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4034/8097] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:22 2022] PUT /identity/v3/projects/823a80e1f96b49848a54c92488ff36b6/users/9af0dab0ba424338932197c368daf988/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:22.129787 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4064/8098] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:22 2022] POST /identity/v3/roles => generated 243 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:22.131189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.131514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c940ec4-3ca7-4368-9e8d-2fcd2b535076 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.131799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c940ec4-3ca7-4368-9e8d-2fcd2b535076 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.132086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c940ec4-3ca7-4368-9e8d-2fcd2b535076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.163561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9c940ec4-3ca7-4368-9e8d-2fcd2b535076 None None] MFA Rules not processed for user `9af0dab0ba424338932197c368daf988`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:22.164950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c940ec4-3ca7-4368-9e8d-2fcd2b535076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:22.215450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff39147-5ccd-47ca-a28a-fe95cce5c6ca admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.219074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.219851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.220143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.220426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.221303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.222986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.249728 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4035/8099] 10.222.0.26 () {60 vars in 1143 bytes} [Tue Sep 27 10:34:22 2022] POST /identity/v3/roles => generated 246 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:22.254268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.270123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810ef39-db45-41fa-b193-2e4556f7e782 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.275950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.276720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.277146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.277435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/e11d8cb70aab4d3992e57e52e05b16cb/users/75f6b05db64049a6b797f7b08b34c14b/roles/874677b6cd5e4f99bf0e8691403c67cb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.281813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:22.284117 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4065/8100] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:22 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:22.289102 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c940ec4-3ca7-4368-9e8d-2fcd2b535076 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.301440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=e11d8cb70aab4d3992e57e52e05b16cb, user_id=75f6b05db64049a6b797f7b08b34c14b, role_id=874677b6cd5e4f99bf0e8691403c67cb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.304290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.327121 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4036/8101] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:22 2022] PUT /identity/v3/projects/e11d8cb70aab4d3992e57e52e05b16cb/users/75f6b05db64049a6b797f7b08b34c14b/roles/874677b6cd5e4f99bf0e8691403c67cb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:22.331280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.394010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c940ec4-3ca7-4368-9e8d-2fcd2b535076 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.418182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e16bc-40ea-4397-b74a-3fe6a017b20c tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.423303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.424077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.424459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.424843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/e11d8cb70aab4d3992e57e52e05b16cb/users/75f6b05db64049a6b797f7b08b34c14b/roles/76a9f8188a884639b6c9d2b363a4722a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.427364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:create_grant(project_id=e11d8cb70aab4d3992e57e52e05b16cb, user_id=75f6b05db64049a6b797f7b08b34c14b, role_id=76a9f8188a884639b6c9d2b363a4722a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.430328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.445989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:22.465991 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4037/8102] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:22 2022] PUT /identity/v3/projects/e11d8cb70aab4d3992e57e52e05b16cb/users/75f6b05db64049a6b797f7b08b34c14b/roles/76a9f8188a884639b6c9d2b363a4722a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:22.469730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.483598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.484832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.485385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.485878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.486720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.493680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.545344 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4066/8103] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:22 2022] POST /identity/v3/projects => generated 405 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:22.549152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.584392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601ff0a-ab6b-43ab-a324-e5e900c42089 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.650119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4f556-95df-488b-8b5d-c1a6c4f04888 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.665579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.666215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.666489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.666759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.667456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.669379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.718177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc323f-ca48-4117-8179-b2afff9d184b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.718668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc323f-ca48-4117-8179-b2afff9d184b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.719376 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4067/8104] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:22 2022] POST /identity/v3/users => generated 309 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:22.725079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.727800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc323f-ca48-4117-8179-b2afff9d184b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.728268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc323f-ca48-4117-8179-b2afff9d184b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.729536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc323f-ca48-4117-8179-b2afff9d184b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.731788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc323f-ca48-4117-8179-b2afff9d184b tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.748055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64446ed-584e-48a6-9f2a-b8ecd78958cd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.766868 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4038/8105] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:22 2022] GET /identity/v3/users => generated 7816 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:22.776331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208da138-6b6d-4e1d-b8c0-0058ed0cf4aa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.776331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208da138-6b6d-4e1d-b8c0-0058ed0cf4aa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.776331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208da138-6b6d-4e1d-b8c0-0058ed0cf4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.779880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.780469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.780830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.781037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.781721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.783323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.803724 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4068/8106] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:22 2022] GET /identity/v3/roles => generated 1891 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:22.806180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.817821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241bdad-f9df-42dc-89d9-1f041d637bb8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.824554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.825798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.826087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.826521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] PATH_INFO: `/v3/projects/2451815c6b4c4e63b1c899ce60023ea4/users/1b1f7671ef5c42c7b1c29588b39430bb/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.838621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-208da138-6b6d-4e1d-b8c0-0058ed0cf4aa None None] MFA Rules not processed for user `75f6b05db64049a6b797f7b08b34c14b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:22.840297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-208da138-6b6d-4e1d-b8c0-0058ed0cf4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:22.844994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2451815c6b4c4e63b1c899ce60023ea4, user_id=1b1f7671ef5c42c7b1c29588b39430bb, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.846134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:22.848550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.872037 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4069/8107] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:22 2022] PUT /identity/v3/projects/2451815c6b4c4e63b1c899ce60023ea4/users/1b1f7671ef5c42c7b1c29588b39430bb/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:22.876535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:22.930328 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4039/8108] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:22 2022] POST /identity/v3/auth/tokens => generated 3277 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:22.943272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.943563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.944033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.956717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c4bd0-7af7-4cec-81b3-d627645a2eb9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:22.963714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:22.964750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:22.965186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:22.965454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:22.966992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:22.968677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:22.978900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] MFA Rules not processed for user `75f6b05db64049a6b797f7b08b34c14b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:22.980064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:22.980794 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4070/8109] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:22 2022] GET /identity/v3/roles => generated 1891 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:22.989901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.001062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9266162-ad62-4285-af58-3cdbb69f9a99 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.026137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346e923-58ce-421f-a916-2a31182402c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.027547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346e923-58ce-421f-a916-2a31182402c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.027930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346e923-58ce-421f-a916-2a31182402c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.028330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346e923-58ce-421f-a916-2a31182402c1 admin admin] PATH_INFO: `/v3/projects/2451815c6b4c4e63b1c899ce60023ea4/users/1b1f7671ef5c42c7b1c29588b39430bb/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.030945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346e923-58ce-421f-a916-2a31182402c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2451815c6b4c4e63b1c899ce60023ea4, user_id=1b1f7671ef5c42c7b1c29588b39430bb, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.035009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346e923-58ce-421f-a916-2a31182402c1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.038329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:23.064962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4071/8110] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:22 2022] PUT /identity/v3/projects/2451815c6b4c4e63b1c899ce60023ea4/users/1b1f7671ef5c42c7b1c29588b39430bb/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:23.066795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6211a0e-cbd4-495c-9730-eff559435f45 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.067230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6211a0e-cbd4-495c-9730-eff559435f45 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.067508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6211a0e-cbd4-495c-9730-eff559435f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.120975 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4040/8111] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:22 2022] POST /identity/v3/auth/tokens => generated 3277 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:23.121968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.123378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.142757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d6211a0e-cbd4-495c-9730-eff559435f45 None None] MFA Rules not processed for user `1b1f7671ef5c42c7b1c29588b39430bb`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:23.144279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d6211a0e-cbd4-495c-9730-eff559435f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.144882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa44d8-f748-45cc-a6be-0b0fa1436d62 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.215633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.216359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.216733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.217072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] PATH_INFO: `/v3/users/178956f6c8154a06a0710b295aff3086` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.230985 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4072/8112] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:23 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:23.232870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6211a0e-cbd4-495c-9730-eff559435f45 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.237764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=178956f6c8154a06a0710b295aff3086)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.241898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.299803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:23.308793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178956f6c8154a06a0710b295aff3086'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.309936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178956f6c8154a06a0710b295aff3086'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.311864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6211a0e-cbd4-495c-9730-eff559435f45 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.317488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.318128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178956f6c8154a06a0710b295aff3086'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.318254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.318553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.318834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.319456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:create_trust()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.320753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.328298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178956f6c8154a06a0710b295aff3086'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.353895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178956f6c8154a06a0710b295aff3086'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.371523 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4041/8113] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:23 2022] DELETE /identity/v3/users/178956f6c8154a06a0710b295aff3086 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:23.373015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.374614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.447828 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4073/8114] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 10:34:23 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:23.451919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.454838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.464907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1880e2-58b5-4a09-8e9c-430367373202 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.468949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.469868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.470147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.470418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.472374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.486682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70043d-c52b-4f42-aa60-76aee0a0cb5a tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.507474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:23.540560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.542179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.551370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.552121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.552544 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4042/8115] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:23 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:23.552676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.552873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.559619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:get_trust(trust_id=6f5226e42cbb47bdada1591440b8fcf1)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.561200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.569098 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4074/8116] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:23 2022] GET /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1 => generated 834 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:23.573354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.584178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb9831-8687-4ddd-b663-45b8e9436594 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.589671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.590161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.590455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.590732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.598568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=6f5226e42cbb47bdada1591440b8fcf1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.604610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.611348 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4043/8117] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:34:23 2022] GET /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles => generated 370 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:23.611718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.622436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4bc2-92ce-479a-814a-bc4303fb04a6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.627085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.627812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.628168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.628504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/874677b6cd5e4f99bf0e8691403c67cb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.635341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6f5226e42cbb47bdada1591440b8fcf1, role_id=874677b6cd5e4f99bf0e8691403c67cb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.638757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.648996 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4075/8118] 10.222.0.26 () {58 vars in 1368 bytes} [Tue Sep 27 10:34:23 2022] GET /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/874677b6cd5e4f99bf0e8691403c67cb => generated 243 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:23.650290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.660908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373866a4-c50d-491a-8a35-bec5267db527 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.669834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.670533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.670811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.671290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/874677b6cd5e4f99bf0e8691403c67cb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.679317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6f5226e42cbb47bdada1591440b8fcf1, role_id=874677b6cd5e4f99bf0e8691403c67cb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.681225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.691279 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4044/8119] 10.222.0.26 () {58 vars in 1369 bytes} [Tue Sep 27 10:34:23 2022] HEAD /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/874677b6cd5e4f99bf0e8691403c67cb => generated 0 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 27 10:34:23.694154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.695620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.697214 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.707439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2dee5-2403-4b66-ad20-7b1bb1e414b1 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.715523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.716837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.717260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.717841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/76a9f8188a884639b6c9d2b363a4722a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.725732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6f5226e42cbb47bdada1591440b8fcf1, role_id=76a9f8188a884639b6c9d2b363a4722a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.727952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Could not find role: 76a9f8188a884639b6c9d2b363a4722a.: keystone.exception.RoleNotFound: Could not find role: 76a9f8188a884639b6c9d2b363a4722a. Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 10:34:23.745295 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 76a9f8188a884639b6c9d2b363a4722a. Sep 27 10:34:23.746193 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:23.748969 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4045/8120] 10.222.0.26 () {58 vars in 1368 bytes} [Tue Sep 27 10:34:23 2022] GET /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/76a9f8188a884639b6c9d2b363a4722a => generated 110 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:23.751197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.762962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565b7a1-fb2c-487a-afde-3da0a08774c8 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.767438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.768161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `HEAD` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.768537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.768907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/76a9f8188a884639b6c9d2b363a4722a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.777383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:get_role_for_trust(trust_id=6f5226e42cbb47bdada1591440b8fcf1, role_id=76a9f8188a884639b6c9d2b363a4722a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.778856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Could not find role: 76a9f8188a884639b6c9d2b363a4722a.: keystone.exception.RoleNotFound: Could not find role: 76a9f8188a884639b6c9d2b363a4722a. Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 10:34:23.796618 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 76a9f8188a884639b6c9d2b363a4722a. Sep 27 10:34:23.797703 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:23.797835 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4046/8121] 10.222.0.26 () {58 vars in 1369 bytes} [Tue Sep 27 10:34:23 2022] HEAD /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1/roles/76a9f8188a884639b6c9d2b363a4722a => generated 0 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 27 10:34:23.800300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68020b0-c718-4d67-9409-b367be6caeec tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.802040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.805993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.806705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.806984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.807260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] PATH_INFO: `/v3/projects/64f8fe93425d4e97b1ae5661172f4846` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.807607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:23.815373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a9d5-ea5d-4cc6-94b1-31391ef44d89 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.821696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f8fe93425d4e97b1ae5661172f4846)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.823579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.823764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.824176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.824937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.825064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.832329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorizing `identity:delete_trust(trust_id=6f5226e42cbb47bdada1591440b8fcf1)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:23.834011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:23.862653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '6f5226e42cbb47bdada1591440b8fcf1', 'request_id': 'req-f879cde7-022e-4e94-acc0-cdc5f6716fef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.871327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '6f5226e42cbb47bdada1591440b8fcf1', 'request_id': 'req-f879cde7-022e-4e94-acc0-cdc5f6716fef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.871917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '6f5226e42cbb47bdada1591440b8fcf1', 'request_id': 'req-f879cde7-022e-4e94-acc0-cdc5f6716fef'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.879838 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4047/8122] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:34:23 2022] DELETE /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:23.884323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.885776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.887598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f8fe93425d4e97b1ae5661172f4846', 'request_id': 'req-a036eceb-7899-4b45-ba82-7521432912ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.897349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f8fe93425d4e97b1ae5661172f4846', 'request_id': 'req-a036eceb-7899-4b45-ba82-7521432912ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.898960 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879cde7-022e-4e94-acc0-cdc5f6716fef tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:23.905213 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f8fe93425d4e97b1ae5661172f4846', 'request_id': 'req-a036eceb-7899-4b45-ba82-7521432912ad'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.919533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] The token cache is being invalidate because project 64f8fe93425d4e97b1ae5661172f4846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:23.919920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.920682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:23.922861 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4076/8123] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:23 2022] DELETE /identity/v3/projects/64f8fe93425d4e97b1ae5661172f4846 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:23.926624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.928061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:23.949435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:23.967488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75f6b05db64049a6b797f7b08b34c14b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e11d8cb70aab4d3992e57e52e05b16cb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-865881098', 'tempest-DelegatedRole-314965254'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:23.969001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:23.969483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:23.969974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] PATH_INFO: `/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Could not find trust: 6f5226e42cbb47bdada1591440b8fcf1.: keystone.exception.TrustNotFound: Could not find trust: 6f5226e42cbb47bdada1591440b8fcf1. Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 27 10:34:23.979904 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 6f5226e42cbb47bdada1591440b8fcf1. Sep 27 10:34:23.980880 ncdv-master-1 devstack@keystone.service[104131]: ERROR keystone.server.flask.application Sep 27 10:34:23.982058 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4048/8124] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:34:23 2022] GET /identity/v3/OS-TRUST/trusts/6f5226e42cbb47bdada1591440b8fcf1 => generated 111 bytes in 98 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 27 10:34:23.986334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:23.988370 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.011138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036eceb-7899-4b45-ba82-7521432912ad admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.025921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.027053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.027342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.027617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] PATH_INFO: `/v3/users/220cd4bdb57b402fb15667386d90dc58` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.064106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=220cd4bdb57b402fb15667386d90dc58)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.066229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.074865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974863c-01f0-4668-82c5-c1ea2ad320e6 tempest-TrustsV3TestJSON-1968621225 tempest-user-63510169] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.081554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.082723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.083007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.083335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/76a9f8188a884639b6c9d2b363a4722a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.087314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=76a9f8188a884639b6c9d2b363a4722a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.087839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.135981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220cd4bdb57b402fb15667386d90dc58'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.137608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220cd4bdb57b402fb15667386d90dc58'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.147484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220cd4bdb57b402fb15667386d90dc58'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.150783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role 76a9f8188a884639b6c9d2b363a4722a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:24.151143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.152825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.156390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220cd4bdb57b402fb15667386d90dc58'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.157389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220cd4bdb57b402fb15667386d90dc58'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.158502 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4049/8125] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:23 2022] DELETE /identity/v3/roles/76a9f8188a884639b6c9d2b363a4722a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:24.159420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:24.161677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.178032 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4077/8126] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:23 2022] DELETE /identity/v3/users/220cd4bdb57b402fb15667386d90dc58 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:24.183342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:24.185170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.198549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:24.268163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e962de9-939f-410c-a486-adf58c70cb06 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.268578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1328d-119e-469b-af00-ef7b9789ef58 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.274173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.275523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.275523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.275726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/roles/874677b6cd5e4f99bf0e8691403c67cb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.277403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_role(role_id=874677b6cd5e4f99bf0e8691403c67cb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.279757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.310528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:24.333298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invalidating the token cache because role 874677b6cd5e4f99bf0e8691403c67cb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:24.333590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.334616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.337326 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4078/8127] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:24 2022] DELETE /identity/v3/roles/874677b6cd5e4f99bf0e8691403c67cb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:24.341241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:24.342758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.352647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.353448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.353732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.354006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.355818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.423143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33bdc3-89f5-4969-88a4-5296596a870f tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.428159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.428863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.429175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.429504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/users/75f6b05db64049a6b797f7b08b34c14b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.430744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_user(user_id=75f6b05db64049a6b797f7b08b34c14b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.432717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.432826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.434415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.444888 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4050/8128] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:24 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:24.451032 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:24.453166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.488973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f6b05db64049a6b797f7b08b34c14b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.489791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f6b05db64049a6b797f7b08b34c14b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.502855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f6b05db64049a6b797f7b08b34c14b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.513824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f6b05db64049a6b797f7b08b34c14b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.514918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f6b05db64049a6b797f7b08b34c14b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.531598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:24.542212 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4079/8129] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:24 2022] DELETE /identity/v3/users/75f6b05db64049a6b797f7b08b34c14b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:24.546292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:24.547231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81db1cd-1331-4954-957c-12dcb68c70f8 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.547655 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.623355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.624121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.624619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.625024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] PATH_INFO: `/v3/projects/8b8299ae80f441c5850363b0794c28bf` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.633503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980292c-3c7b-4b02-a048-344a234ad531 tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.637641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b8299ae80f441c5850363b0794c28bf)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.638911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942a4aa4755d4c5b839f6aea46ec4848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dff029989c9b4b73aa179e9ce7c85901', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.639997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.640209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.640522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.640827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] PATH_INFO: `/v3/projects/e11d8cb70aab4d3992e57e52e05b16cb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.642554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e11d8cb70aab4d3992e57e52e05b16cb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.645178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.659283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:24.703564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b8299ae80f441c5850363b0794c28bf', 'request_id': 'req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.710431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11d8cb70aab4d3992e57e52e05b16cb', 'request_id': 'req-702e4144-4a64-4b4d-b4c5-104c44b736dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.712217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b8299ae80f441c5850363b0794c28bf', 'request_id': 'req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.719390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11d8cb70aab4d3992e57e52e05b16cb', 'request_id': 'req-702e4144-4a64-4b4d-b4c5-104c44b736dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.722922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b8299ae80f441c5850363b0794c28bf', 'request_id': 'req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.727116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11d8cb70aab4d3992e57e52e05b16cb', 'request_id': 'req-702e4144-4a64-4b4d-b4c5-104c44b736dc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.739251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] The token cache is being invalidate because project 8b8299ae80f441c5850363b0794c28bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:24.739551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.740319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.743110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4051/8130] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:24 2022] DELETE /identity/v3/projects/8b8299ae80f441c5850363b0794c28bf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:24.744291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] The token cache is being invalidate because project e11d8cb70aab4d3992e57e52e05b16cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:24.744617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.745839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-702e4144-4a64-4b4d-b4c5-104c44b736dc tempest-TrustsV3TestJSON-298437606 tempest-TrustsV3TestJSON-298437606-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.747869 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4080/8131] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:24 2022] DELETE /identity/v3/projects/e11d8cb70aab4d3992e57e52e05b16cb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:24.753827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:24.762089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:24.863355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3393c8-d2a1-4dc6-82ba-c8d184cedca5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:24.869099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:24.870444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:24.870444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:24.870611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] PATH_INFO: `/v3/users/7e426f84d0a54d85bc0a80ad321fd309` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:24.871843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e426f84d0a54d85bc0a80ad321fd309)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:24.873827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:24.904863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:24.942755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e426f84d0a54d85bc0a80ad321fd309'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.943505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e426f84d0a54d85bc0a80ad321fd309'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.956523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e426f84d0a54d85bc0a80ad321fd309'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.969865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e426f84d0a54d85bc0a80ad321fd309'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:24.991730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e426f84d0a54d85bc0a80ad321fd309'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:25.004748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4052/8132] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:24 2022] DELETE /identity/v3/users/7e426f84d0a54d85bc0a80ad321fd309 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:25.007776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:25.008573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:25.008894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:25.030290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:25.032599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:25.159807 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4081/8133] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:25.168287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:25.169810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:25.285709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3188-3ed4-462d-adb7-dc5ec48b9321 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:25.291495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:25.292135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:25.292540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:25.292867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:25.305935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:25.307920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:25.319789 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4053/8134] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:26.016188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:26.017654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:26.032170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e4534-4036-46b6-a2d5-7dadc99dc1b3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:26.038038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:26.038839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:26.039147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:26.039419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] PATH_INFO: `/v3/projects/99b5b5bd737b496ca217aa72243ae844` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:26.041251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b5b5bd737b496ca217aa72243ae844)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:26.043186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:26.106325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b5b5bd737b496ca217aa72243ae844', 'request_id': 'req-14176803-898f-4e70-b008-24ae5f15fbd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.117920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b5b5bd737b496ca217aa72243ae844', 'request_id': 'req-14176803-898f-4e70-b008-24ae5f15fbd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.127180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b5b5bd737b496ca217aa72243ae844', 'request_id': 'req-14176803-898f-4e70-b008-24ae5f15fbd7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.145454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] The token cache is being invalidate because project 99b5b5bd737b496ca217aa72243ae844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:26.145454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.145454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.146880 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4082/8135] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:26 2022] DELETE /identity/v3/projects/99b5b5bd737b496ca217aa72243ae844 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:26.150671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:26.162068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f204-1907-4d8d-a236-661af0f78e99 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:26.229367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:26.230181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:26.230642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:26.230905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] PATH_INFO: `/v3/users/942a4aa4755d4c5b839f6aea46ec4848` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:26.232063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=942a4aa4755d4c5b839f6aea46ec4848)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:26.234131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:26.297390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:26.299529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942a4aa4755d4c5b839f6aea46ec4848'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.300477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942a4aa4755d4c5b839f6aea46ec4848'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.313919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942a4aa4755d4c5b839f6aea46ec4848'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.327451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942a4aa4755d4c5b839f6aea46ec4848'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.349520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-81e00aee-7a68-49d8-bfbd-36b7bf4f7f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942a4aa4755d4c5b839f6aea46ec4848'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:26.360747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4054/8136] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:26 2022] DELETE /identity/v3/users/942a4aa4755d4c5b839f6aea46ec4848 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:27.017294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.018759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:27.033211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14176803-898f-4e70-b008-24ae5f15fbd7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.053615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:27.102845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.103802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.104079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.104351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] PATH_INFO: `/v3/projects/dff029989c9b4b73aa179e9ce7c85901` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.105725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff029989c9b4b73aa179e9ce7c85901)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.107704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.177702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff029989c9b4b73aa179e9ce7c85901', 'request_id': 'req-0600194d-2e4d-4d41-bf51-d1d314663801'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:27.189835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff029989c9b4b73aa179e9ce7c85901', 'request_id': 'req-0600194d-2e4d-4d41-bf51-d1d314663801'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:27.198016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff029989c9b4b73aa179e9ce7c85901', 'request_id': 'req-0600194d-2e4d-4d41-bf51-d1d314663801'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:27.213467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] The token cache is being invalidate because project dff029989c9b4b73aa179e9ce7c85901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:27.214367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:27.215071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-0600194d-2e4d-4d41-bf51-d1d314663801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:27.223204 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4083/8137] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:27 2022] DELETE /identity/v3/projects/dff029989c9b4b73aa179e9ce7c85901 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:27.224699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3017-e350-4731-9df3-cc8357e5e7aa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.225526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3017-e350-4731-9df3-cc8357e5e7aa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.225812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3017-e350-4731-9df3-cc8357e5e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.253121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9b1f3017-e350-4731-9df3-cc8357e5e7aa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:27.254542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b1f3017-e350-4731-9df3-cc8357e5e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:27.326394 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4055/8138] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:27.335441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b979-5d26-4cde-ab02-dbbed3a316ea None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.335441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b979-5d26-4cde-ab02-dbbed3a316ea None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.335849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b979-5d26-4cde-ab02-dbbed3a316ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.362222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-27a3b979-5d26-4cde-ab02-dbbed3a316ea None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:27.362755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-27a3b979-5d26-4cde-ab02-dbbed3a316ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:27.372521 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4084/8139] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:27.376718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f3017-e350-4731-9df3-cc8357e5e7aa None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.389080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f3017-e350-4731-9df3-cc8357e5e7aa None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.394600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.394773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.395054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.395335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.395981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:27.396513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.398229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.411598 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4056/8140] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:27.416840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b979-5d26-4cde-ab02-dbbed3a316ea None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.429032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b979-5d26-4cde-ab02-dbbed3a316ea None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.435405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.436386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.436692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.436953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.437610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.439576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.474871 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4085/8141] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:27 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:27.479429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.492374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a57b5-0de5-46bd-b798-a1888b247639 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.567924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.568617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.569228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.569228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.570052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.572061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.600687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4057/8142] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:27 2022] POST /identity/v3/users => generated 302 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:27.603247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.621992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd9afe-85cc-4aa1-9043-e0b8d3d18400 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.640183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.640851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.641131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.641403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.642183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.643878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.655229 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4086/8143] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:27.659559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.671663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634302-6113-4079-b2c8-9154e3145d59 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.677812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.678582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.678951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.679507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] PATH_INFO: `/v3/projects/470740fb9bbf4f7294fbfd8ff4c5c45d/users/3df39f6b5fc3404c96c0180547dc8443/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.697217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470740fb9bbf4f7294fbfd8ff4c5c45d, user_id=3df39f6b5fc3404c96c0180547dc8443, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.700457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.718484 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4058/8144] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:27 2022] PUT /identity/v3/projects/470740fb9bbf4f7294fbfd8ff4c5c45d/users/3df39f6b5fc3404c96c0180547dc8443/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:27.722577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.733239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132ae1f-3300-41f6-90ed-922339601913 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.798658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd285fd-d0a6-4130-9f95-ab6b0ea235c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.799273 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd285fd-d0a6-4130-9f95-ab6b0ea235c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.799548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd285fd-d0a6-4130-9f95-ab6b0ea235c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.799814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd285fd-d0a6-4130-9f95-ab6b0ea235c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.800442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd285fd-d0a6-4130-9f95-ab6b0ea235c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.802053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd285fd-d0a6-4130-9f95-ab6b0ea235c3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.816190 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4087/8145] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:27.817519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.830236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9da48c-8e95-4f96-b4a3-6cbf42335336 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:27.837717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:27.838564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.838976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.839331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] PATH_INFO: `/v3/projects/470740fb9bbf4f7294fbfd8ff4c5c45d/users/3df39f6b5fc3404c96c0180547dc8443/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.841573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470740fb9bbf4f7294fbfd8ff4c5c45d, user_id=3df39f6b5fc3404c96c0180547dc8443, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:27.844339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:27.856616 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4059/8146] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:27 2022] PUT /identity/v3/projects/470740fb9bbf4f7294fbfd8ff4c5c45d/users/3df39f6b5fc3404c96c0180547dc8443/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:27.867927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2ee9b-bef1-4825-a2ab-28f87b2c2e42 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:27.868223 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2ee9b-bef1-4825-a2ab-28f87b2c2e42 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:27.868494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2ee9b-bef1-4825-a2ab-28f87b2c2e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:27.890866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:27.900596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b6e2ee9b-bef1-4825-a2ab-28f87b2c2e42 None None] MFA Rules not processed for user `3df39f6b5fc3404c96c0180547dc8443`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:27.902528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b6e2ee9b-bef1-4825-a2ab-28f87b2c2e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:27.975555 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4088/8147] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:27.980351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:27.985523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:28.048868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af208a15-a126-48a8-b7cf-a579594a5af8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:28.053899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:28.054520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:28.054788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:28.055079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:28.055754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:28.057718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:28.088318 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4060/8148] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:27 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:28.092482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2ee9b-bef1-4825-a2ab-28f87b2c2e42 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:28.103633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2ee9b-bef1-4825-a2ab-28f87b2c2e42 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:28.162533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:28.163283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:28.163604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:28.164030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:28.165567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:28.167603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:28.191438 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4089/8149] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:28 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:28.195700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:28.207427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5c68-bc11-45c4-8dcc-258ba55e2a7b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:28.213138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:28.213936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:28.214669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:28.214669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:28.215256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:28.216888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:28.228828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4061/8150] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:28.232088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:28.243669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b2f39-08af-4c98-afb8-fb35f4a01e56 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:28.249252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92427ce1-363b-4b2d-b73b-d6af9695c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:28.249899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92427ce1-363b-4b2d-b73b-d6af9695c30e admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:28.250226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92427ce1-363b-4b2d-b73b-d6af9695c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:28.250425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92427ce1-363b-4b2d-b73b-d6af9695c30e admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:28.251065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92427ce1-363b-4b2d-b73b-d6af9695c30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:28.252709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92427ce1-363b-4b2d-b73b-d6af9695c30e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:28.264483 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4090/8151] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:28.267838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:28.282496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f0162-53bc-4fc5-a9cb-662b66ef4b45 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:28.287741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:28.289316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:28.289316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:28.289482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] PATH_INFO: `/v3/projects/3762527b51d84aedace5a138a6cbb61c/users/e7253872a4d34aafbaa3292da2d9dee4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:28.306923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3762527b51d84aedace5a138a6cbb61c, user_id=e7253872a4d34aafbaa3292da2d9dee4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:28.309644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:28.328411 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4062/8152] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:28 2022] PUT /identity/v3/projects/3762527b51d84aedace5a138a6cbb61c/users/e7253872a4d34aafbaa3292da2d9dee4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:28.339965 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6d55-b2f0-48fd-bbb6-3117d0c5b722 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:28.340335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6d55-b2f0-48fd-bbb6-3117d0c5b722 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:28.340541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6d55-b2f0-48fd-bbb6-3117d0c5b722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:28.369745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-840c6d55-b2f0-48fd-bbb6-3117d0c5b722 None None] MFA Rules not processed for user `e7253872a4d34aafbaa3292da2d9dee4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:28.370964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-840c6d55-b2f0-48fd-bbb6-3117d0c5b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:28.445904 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4091/8153] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:28.461986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:28.463771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:28.537237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90674-a154-47aa-941d-7ed43ead3ecd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:28.556803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faff796-f9a5-41e4-8422-3ccb2fec18f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:28.557508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff796-f9a5-41e4-8422-3ccb2fec18f5 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:28.558731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff796-f9a5-41e4-8422-3ccb2fec18f5 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:28.559171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff796-f9a5-41e4-8422-3ccb2fec18f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:28.685461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faff796-f9a5-41e4-8422-3ccb2fec18f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:28.686932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faff796-f9a5-41e4-8422-3ccb2fec18f5 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:28.698325 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4063/8154] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:29.769144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c6d55-b2f0-48fd-bbb6-3117d0c5b722 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:29.784768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:29.884092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c6d55-b2f0-48fd-bbb6-3117d0c5b722 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:29.896915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:29.897561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:29.897974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:29.898614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] PATH_INFO: `/v3/users/3df39f6b5fc3404c96c0180547dc8443` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:29.899434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df39f6b5fc3404c96c0180547dc8443)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:29.905073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:30.000119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df39f6b5fc3404c96c0180547dc8443'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.000890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df39f6b5fc3404c96c0180547dc8443'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.011665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df39f6b5fc3404c96c0180547dc8443'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.020309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df39f6b5fc3404c96c0180547dc8443'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.021239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df39f6b5fc3404c96c0180547dc8443'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.040826 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4092/8155] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:29 2022] DELETE /identity/v3/users/3df39f6b5fc3404c96c0180547dc8443 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:30.045445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:30.045897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:30.045982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:30.053427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:30.067499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:30.068769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:30.129998 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4064/8156] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:30.140079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:30.141783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:30.217634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4534-d482-467c-b5ae-35a64a3cdbd2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:30.221711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:30.222325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:30.222597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:30.222886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:30.236290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:30.237813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:30.248205 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4093/8157] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:30.698220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:30.699712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:30.715573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b1b4a-aca0-4591-bab7-56fc87aa3c1a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:30.724356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:30.725013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:30.725291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:30.726728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] PATH_INFO: `/v3/users/9af0dab0ba424338932197c368daf988` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:30.728439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af0dab0ba424338932197c368daf988)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:30.730337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:30.752296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:30.754772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:30.772168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d16456-7a64-4512-a498-dc3ee0cadc1c service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:30.777396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:30.778116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:30.778455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:30.778730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] PATH_INFO: `/v3/projects/470740fb9bbf4f7294fbfd8ff4c5c45d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:30.779858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=470740fb9bbf4f7294fbfd8ff4c5c45d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:30.781844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:30.801972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af0dab0ba424338932197c368daf988'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.804566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af0dab0ba424338932197c368daf988'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.808394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:30.815402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af0dab0ba424338932197c368daf988'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.830341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af0dab0ba424338932197c368daf988'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.857019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-92339503-c8b1-4c42-a1a1-0f8a6981594e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af0dab0ba424338932197c368daf988'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.858528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470740fb9bbf4f7294fbfd8ff4c5c45d', 'request_id': 'req-9132bfd9-ffa8-4666-b67f-4986e9690c79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.868394 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4065/8158] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:30 2022] DELETE /identity/v3/users/9af0dab0ba424338932197c368daf988 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:30.871694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470740fb9bbf4f7294fbfd8ff4c5c45d', 'request_id': 'req-9132bfd9-ffa8-4666-b67f-4986e9690c79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.876158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:30.876648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:30.877239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:30.879529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470740fb9bbf4f7294fbfd8ff4c5c45d', 'request_id': 'req-9132bfd9-ffa8-4666-b67f-4986e9690c79'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.895112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] The token cache is being invalidate because project 470740fb9bbf4f7294fbfd8ff4c5c45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:30.895642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.896430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:30.898856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4094/8159] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:30 2022] DELETE /identity/v3/projects/470740fb9bbf4f7294fbfd8ff4c5c45d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:30.902887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:30.904283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:30.907722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:30.916106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:30.997007 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4066/8160] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:30.998497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132bfd9-ffa8-4666-b67f-4986e9690c79 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.003357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.003989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.004261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.004528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] PATH_INFO: `/v3/users/e7253872a4d34aafbaa3292da2d9dee4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.005689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7253872a4d34aafbaa3292da2d9dee4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.006908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.007872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:31.008246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.026939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:31.063232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7253872a4d34aafbaa3292da2d9dee4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.064014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7253872a4d34aafbaa3292da2d9dee4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.074951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb66f-d83b-4515-96ae-84987af04271 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.077707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7253872a4d34aafbaa3292da2d9dee4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.079253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.079934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.080214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.080510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.082494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.086525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7253872a4d34aafbaa3292da2d9dee4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.087352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7253872a4d34aafbaa3292da2d9dee4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.096399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.098141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:31.105122 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4095/8161] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:30 2022] DELETE /identity/v3/users/e7253872a4d34aafbaa3292da2d9dee4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:31.108315 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4067/8162] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:31.323034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.324681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.363809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:31.402482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a32b5-9fcc-42ad-98b0-9cb2b03880aa admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.407076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.407715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.407993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.408265 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] PATH_INFO: `/v3/projects/823a80e1f96b49848a54c92488ff36b6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.409405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=823a80e1f96b49848a54c92488ff36b6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.411325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:31.442131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.443656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.464894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7342f6-ad06-4958-84c5-7a3b47422090 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.472577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823a80e1f96b49848a54c92488ff36b6', 'request_id': 'req-d35b3084-966e-49ad-9afa-62a4f09381d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.482636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823a80e1f96b49848a54c92488ff36b6', 'request_id': 'req-d35b3084-966e-49ad-9afa-62a4f09381d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.490356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823a80e1f96b49848a54c92488ff36b6', 'request_id': 'req-d35b3084-966e-49ad-9afa-62a4f09381d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.503157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] The token cache is being invalidate because project 823a80e1f96b49848a54c92488ff36b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:31.503591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.504250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.505820 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4096/8163] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:31 2022] DELETE /identity/v3/projects/823a80e1f96b49848a54c92488ff36b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:31.510786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.512153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.525493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.526136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.526414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.526691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] PATH_INFO: `/v3/projects/3762527b51d84aedace5a138a6cbb61c` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.527847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3762527b51d84aedace5a138a6cbb61c)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.529804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:31.578732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3762527b51d84aedace5a138a6cbb61c', 'request_id': 'req-60c9bdf9-28d3-4e99-864f-e87ac2a57372'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.581097 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3084-966e-49ad-9afa-62a4f09381d0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.586247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3762527b51d84aedace5a138a6cbb61c', 'request_id': 'req-60c9bdf9-28d3-4e99-864f-e87ac2a57372'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.595390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3762527b51d84aedace5a138a6cbb61c', 'request_id': 'req-60c9bdf9-28d3-4e99-864f-e87ac2a57372'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.608382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] The token cache is being invalidate because project 3762527b51d84aedace5a138a6cbb61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:31.608737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.609778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-60c9bdf9-28d3-4e99-864f-e87ac2a57372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.611748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4068/8164] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:31 2022] DELETE /identity/v3/projects/3762527b51d84aedace5a138a6cbb61c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:31.617057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd50e1d-1a53-42c0-bfa9-0b3d152906e2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.617220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd50e1d-1a53-42c0-bfa9-0b3d152906e2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.617520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd50e1d-1a53-42c0-bfa9-0b3d152906e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.638127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-fcd50e1d-1a53-42c0-bfa9-0b3d152906e2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:31.639340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fcd50e1d-1a53-42c0-bfa9-0b3d152906e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.645054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.645691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.645968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.646225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] PATH_INFO: `/v3/users/1b1f7671ef5c42c7b1c29588b39430bb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.647390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1f7671ef5c42c7b1c29588b39430bb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.649415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:31.653910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:31.686889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:31.704739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1f7671ef5c42c7b1c29588b39430bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.705451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1f7671ef5c42c7b1c29588b39430bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.708448 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4069/8165] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:31.714758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1f7671ef5c42c7b1c29588b39430bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.720231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b003255-088c-4f97-8a47-a79fece2577b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.720753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b003255-088c-4f97-8a47-a79fece2577b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.721076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b003255-088c-4f97-8a47-a79fece2577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.726112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1f7671ef5c42c7b1c29588b39430bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.727079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1f7671ef5c42c7b1c29588b39430bb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:31.741660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8b003255-088c-4f97-8a47-a79fece2577b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:31.743010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8b003255-088c-4f97-8a47-a79fece2577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.745350 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4097/8166] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:31 2022] DELETE /identity/v3/users/1b1f7671ef5c42c7b1c29588b39430bb => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:31.806342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4070/8167] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:31.810741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.821321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b8d4-7441-485f-ad5d-905e308fafcb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.826159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.826759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.827031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.827299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.827939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:31.828231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.829853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:31.842134 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4098/8168] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:31.846758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b003255-088c-4f97-8a47-a79fece2577b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.856861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b003255-088c-4f97-8a47-a79fece2577b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.863023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.863670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.863942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.864210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.864909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.866839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:31.895584 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4071/8169] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:31 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:31.899819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.911578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c402-c772-411d-8d1d-560194941a90 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:31.930768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:31.932601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:31.977813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:31.978531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:31.978869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:31.979205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:31.979898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:31.981941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.009709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4099/8170] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:31 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:32.013992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f7107-d1cb-4813-b90e-b691437d2158 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.014200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.018895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.019528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.019802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.020075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] PATH_INFO: `/v3/projects/2451815c6b4c4e63b1c899ce60023ea4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.021230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2451815c6b4c4e63b1c899ce60023ea4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.023228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.027360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07589-3d15-47f2-9002-05ae2c07b43a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.032052 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.032678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.032963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.033231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.033868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.035482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.046860 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4100/8171] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:32.050794 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.061580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f31d1-3fd7-4c55-bc9f-d244d80a83b8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.074320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2451815c6b4c4e63b1c899ce60023ea4', 'request_id': 'req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:32.080647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2451815c6b4c4e63b1c899ce60023ea4', 'request_id': 'req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:32.090581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2451815c6b4c4e63b1c899ce60023ea4', 'request_id': 'req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:32.097483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:32.103306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] The token cache is being invalidate because project 2451815c6b4c4e63b1c899ce60023ea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:32.103852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:32.105074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d92e42c5-6ef1-452b-8ff9-b2fc686ba143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:32.106668 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4072/8172] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:31 2022] DELETE /identity/v3/projects/2451815c6b4c4e63b1c899ce60023ea4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:32.111178 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:32.112172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343911f-f3ab-499c-9485-0717a6543a11 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.112722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343911f-f3ab-499c-9485-0717a6543a11 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.112997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343911f-f3ab-499c-9485-0717a6543a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.125962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.126577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.126853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.127124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.127760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.129472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.133076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1343911f-f3ab-499c-9485-0717a6543a11 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:32.134398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1343911f-f3ab-499c-9485-0717a6543a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:32.139584 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4101/8173] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:32.143124 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4073/8174] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:32.149705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.150982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c5d2-b185-4f41-8a59-db8b64f38385 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.151289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c5d2-b185-4f41-8a59-db8b64f38385 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.151489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:32.151578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c5d2-b185-4f41-8a59-db8b64f38385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.164268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c747f37-b360-4d4c-b7c3-296bf046bfd3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.169156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4b2f7-3d59-412f-b35f-c01bf1c54f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.169820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b2f7-3d59-412f-b35f-c01bf1c54f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.170099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b2f7-3d59-412f-b35f-c01bf1c54f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.170376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b2f7-3d59-412f-b35f-c01bf1c54f18 admin admin] PATH_INFO: `/v3/projects/8e9e80c7081d44b7981a31f647304492/users/6783addded7044429220549d7d155a5a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.171139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b014c5d2-b185-4f41-8a59-db8b64f38385 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:32.172331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b014c5d2-b185-4f41-8a59-db8b64f38385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:32.180319 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4074/8175] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:32.184724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014c5d2-b185-4f41-8a59-db8b64f38385 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.185970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4b2f7-3d59-412f-b35f-c01bf1c54f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9e80c7081d44b7981a31f647304492, user_id=6783addded7044429220549d7d155a5a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.188637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4b2f7-3d59-412f-b35f-c01bf1c54f18 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.194535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014c5d2-b185-4f41-8a59-db8b64f38385 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.199624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.200229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.200508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.201090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.201706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:32.201995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.203556 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4102/8176] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:32 2022] PUT /identity/v3/projects/8e9e80c7081d44b7981a31f647304492/users/6783addded7044429220549d7d155a5a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:32.212776 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.214661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234fa2-b657-4d5d-82cd-12ead9b82a89 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.214981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234fa2-b657-4d5d-82cd-12ead9b82a89 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.215260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234fa2-b657-4d5d-82cd-12ead9b82a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.216354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4075/8177] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:34:32.220611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.230760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e0933-8370-4c2c-a914-bd0b1310f0f6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.244252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8a234fa2-b657-4d5d-82cd-12ead9b82a89 None None] MFA Rules not processed for user `6783addded7044429220549d7d155a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:32.245517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8a234fa2-b657-4d5d-82cd-12ead9b82a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:32.289130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.289848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.290218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.290550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.291245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.293187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.311984 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4103/8178] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:32.317216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234fa2-b657-4d5d-82cd-12ead9b82a89 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.323192 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4076/8179] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/projects => generated 395 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:32.327179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.328748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234fa2-b657-4d5d-82cd-12ead9b82a89 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.387467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.388221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.388498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.388788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.389419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.391237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649bb2-97ca-4817-ba2b-10cbb036f9cf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.391344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.396349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.396991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.397266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.397548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.398324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.400233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.420314 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4104/8180] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:32.425425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.425561 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4077/8181] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:32.426644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.437651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3e395-1d58-4110-9eed-236c8a5b8f6c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.465476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:32.485002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:32.494672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cbb86-5885-4c91-aeef-57765e389dd4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.498994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.499343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.499805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.499948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.500225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.500314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.500521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.500798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.501206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.501537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.503117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.503221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.513436 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4078/8182] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:32.517596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.526816 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4105/8183] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:32.530754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.530876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a1ef9-6299-432c-9c55-b62a31b6ad31 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.536307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.537822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.538194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.538613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.539241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.540954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cc2ca-1698-460a-a4d3-0ce4f2f18c69 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.541121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.545786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.546405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.546681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.546953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.547589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.549222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.555329 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4079/8184] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:32.556427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.562248 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4106/8185] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:32.566171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.567001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b5cd2-b263-43be-b054-140cdf303856 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.572576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68cd69-a545-4d20-8f06-22cfc7971194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.573287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68cd69-a545-4d20-8f06-22cfc7971194 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.573570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68cd69-a545-4d20-8f06-22cfc7971194 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.573848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68cd69-a545-4d20-8f06-22cfc7971194 admin admin] PATH_INFO: `/v3/projects/04ea10d46ecb45dabb545d0d4fcddcb7/users/d859a7dbc66e4add8b6becd36b783136/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.576779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a1a2-e436-43b5-beb0-a83f4cf891c9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.581455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.582044 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.582316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.582588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.583238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.585518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.590039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68cd69-a545-4d20-8f06-22cfc7971194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ea10d46ecb45dabb545d0d4fcddcb7, user_id=d859a7dbc66e4add8b6becd36b783136, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.593212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68cd69-a545-4d20-8f06-22cfc7971194 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.598335 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4107/8186] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:32.599565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.611470 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c88a9-4a4f-415f-ad8a-b2c9abe19136 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.614646 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4080/8187] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:32 2022] PUT /identity/v3/projects/04ea10d46ecb45dabb545d0d4fcddcb7/users/d859a7dbc66e4add8b6becd36b783136/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:32.625536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce4d2f-eb7f-4ff7-807b-31254b01b586 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.625903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce4d2f-eb7f-4ff7-807b-31254b01b586 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.626184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce4d2f-eb7f-4ff7-807b-31254b01b586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.655141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cbce4d2f-eb7f-4ff7-807b-31254b01b586 None None] MFA Rules not processed for user `d859a7dbc66e4add8b6becd36b783136`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:32.656365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cbce4d2f-eb7f-4ff7-807b-31254b01b586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:32.674507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b878f37-bdb5-47af-a02c-af9c12a60cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.675202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b878f37-bdb5-47af-a02c-af9c12a60cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.675478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b878f37-bdb5-47af-a02c-af9c12a60cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.675720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b878f37-bdb5-47af-a02c-af9c12a60cb5 admin admin] PATH_INFO: `/v3/projects/bcb760741ce44cd8902c556318c58222/users/6d61094cf9c34981ac23102c3ba7be9f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.690588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b878f37-bdb5-47af-a02c-af9c12a60cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb760741ce44cd8902c556318c58222, user_id=6d61094cf9c34981ac23102c3ba7be9f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.693297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b878f37-bdb5-47af-a02c-af9c12a60cb5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.709839 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4108/8188] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:32 2022] PUT /identity/v3/projects/bcb760741ce44cd8902c556318c58222/users/6d61094cf9c34981ac23102c3ba7be9f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:32.725675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.726330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.726423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.735525 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4081/8189] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:32.736531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce4d2f-eb7f-4ff7-807b-31254b01b586 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.747002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce4d2f-eb7f-4ff7-807b-31254b01b586 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.755883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] MFA Rules not processed for user `6d61094cf9c34981ac23102c3ba7be9f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:32.757845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:32.804238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.804886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.805165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.805437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.806163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.808067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.824647 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4109/8190] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:32.837649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.839050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:32.839726 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4082/8191] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/projects => generated 399 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:32.843899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.858509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:32.871254 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:32.904757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9d087-afdf-4167-8fa2-b82b2e3b9060 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.910184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e3d74-eddf-4a4e-b503-3f29bb1f4cee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.915089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.915704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.915979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.916249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.916901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.918820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.945771 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4083/8192] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:32 2022] POST /identity/v3/users => generated 306 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:32.949685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.958778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.959984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.960275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.960568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.962802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397830ef-ec82-46e3-af34-0c8dc8587f84 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:32.967803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:32.968413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:32.968769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:32.969051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:32.969691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:32.971496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:32.981865 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4084/8193] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:32.986046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:32.996043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4720ba-ba53-4384-9bff-b83d06964a5a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:33.000645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:33.001386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:33.001666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:33.001941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] PATH_INFO: `/v3/projects/f24de9d8343746489acbb8bda0e54dcf/users/8489e8ba64a54a20a5e4ebbe028a8779/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:33.016582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f24de9d8343746489acbb8bda0e54dcf, user_id=8489e8ba64a54a20a5e4ebbe028a8779, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:33.019325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:33.019754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:33.021239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:33.031473 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4110/8194] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:33.033547 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4085/8195] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:32 2022] PUT /identity/v3/projects/f24de9d8343746489acbb8bda0e54dcf/users/8489e8ba64a54a20a5e4ebbe028a8779/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:33.037491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:33.049250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd869e-bb53-4b75-9659-e0a9db3a1770 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:33.115031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:33.115329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:33.115614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:33.115939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:33.117170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:33.118217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:33.128610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4111/8196] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:33.133124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:33.143511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5713608-5e5e-43b2-a22e-f142ca2e3ac1 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:33.148791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74c8d7-1e2d-45f4-972e-0d498d59e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:33.149541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74c8d7-1e2d-45f4-972e-0d498d59e609 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:33.149841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74c8d7-1e2d-45f4-972e-0d498d59e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:33.150119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74c8d7-1e2d-45f4-972e-0d498d59e609 admin admin] PATH_INFO: `/v3/projects/f24de9d8343746489acbb8bda0e54dcf/users/8489e8ba64a54a20a5e4ebbe028a8779/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:33.152209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74c8d7-1e2d-45f4-972e-0d498d59e609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f24de9d8343746489acbb8bda0e54dcf, user_id=8489e8ba64a54a20a5e4ebbe028a8779, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:33.155020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74c8d7-1e2d-45f4-972e-0d498d59e609 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:33.166398 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4086/8197] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:33 2022] PUT /identity/v3/projects/f24de9d8343746489acbb8bda0e54dcf/users/8489e8ba64a54a20a5e4ebbe028a8779/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:33.168162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:33.177533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:33.177803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:33.178071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:33.197319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:33.208374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] MFA Rules not processed for user `8489e8ba64a54a20a5e4ebbe028a8779`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:33.210291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:33.232603 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f1dce-a439-48ec-809c-ed2fccec5bf6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:33.284105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:33.284747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:33.285026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:33.285302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:33.285432 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4087/8198] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:33 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:33.295937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:33.297395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:33.348936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:33.350598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:33.355786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:33.360083 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4112/8199] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:33.362864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62395310-7c04-42d2-9a5f-9bee9276296e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:33.368402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:33.368402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:33.368708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:33.369405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:33.433034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:33.434578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:33.444324 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4088/8200] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:33 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:33.776009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:33.777522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:33.870924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbedb0-62e7-4f79-b404-8c388fb940e5 service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:33.876467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882598b0-b897-4528-86a2-d65196486748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:33.877172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882598b0-b897-4528-86a2-d65196486748 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:33.877531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882598b0-b897-4528-86a2-d65196486748 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:33.877852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882598b0-b897-4528-86a2-d65196486748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:33.886400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882598b0-b897-4528-86a2-d65196486748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:33.887986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882598b0-b897-4528-86a2-d65196486748 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:33.900586 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4113/8201] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:33 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:34.715276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:34.785661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83628a2d-7bad-4c7e-abb6-ac5cb33ec0d8 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:34.791165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:34.791811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:34.792110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:34.792389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] PATH_INFO: `/v3/users/6783addded7044429220549d7d155a5a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:34.793554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6783addded7044429220549d7d155a5a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:34.795488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:34.851424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6783addded7044429220549d7d155a5a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:34.852305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6783addded7044429220549d7d155a5a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:34.859739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6783addded7044429220549d7d155a5a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:34.870534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6783addded7044429220549d7d155a5a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:34.884041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6783addded7044429220549d7d155a5a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:34.890570 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4089/8202] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:34 2022] DELETE /identity/v3/users/6783addded7044429220549d7d155a5a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:34.895246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:34.895745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:34.895835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:34.915357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:34.916515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:34.932758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:34.985921 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4114/8203] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:34.997271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:34.999099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:35.027150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:35.067306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce6fcd-dda7-4645-a89f-5391eb50cf69 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:35.071645 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:35.072361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:35.072361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:35.072717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:35.082987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:35.084878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:35.094437 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4090/8204] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:35.513545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:35.515023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:35.528342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa507b-0c1d-4daa-8fde-1f0a297540fb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:35.535061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:35.535527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:35.535815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:35.536110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] PATH_INFO: `/v3/projects/8e9e80c7081d44b7981a31f647304492` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:35.537600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9e80c7081d44b7981a31f647304492)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:35.541009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:35.596765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9e80c7081d44b7981a31f647304492', 'request_id': 'req-2327e58c-f241-422e-8c03-5f5c2e8fb564'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.609322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9e80c7081d44b7981a31f647304492', 'request_id': 'req-2327e58c-f241-422e-8c03-5f5c2e8fb564'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.615636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9e80c7081d44b7981a31f647304492', 'request_id': 'req-2327e58c-f241-422e-8c03-5f5c2e8fb564'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.628112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] The token cache is being invalidate because project 8e9e80c7081d44b7981a31f647304492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:35.628417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.629089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.633758 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4115/8205] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:35 2022] DELETE /identity/v3/projects/8e9e80c7081d44b7981a31f647304492 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:35.634968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:35.644916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1988f1-fa2f-4c20-8ba3-a6dd6678af85 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:35.711920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:35.712601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:35.713313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:35.713454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] PATH_INFO: `/v3/users/6d61094cf9c34981ac23102c3ba7be9f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:35.714682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d61094cf9c34981ac23102c3ba7be9f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:35.716635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:35.772927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d61094cf9c34981ac23102c3ba7be9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.773641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d61094cf9c34981ac23102c3ba7be9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.780840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d61094cf9c34981ac23102c3ba7be9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.791924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d61094cf9c34981ac23102c3ba7be9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.806566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-5cbcb265-2293-49db-b292-1cd5d8f72f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d61094cf9c34981ac23102c3ba7be9f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:35.812739 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4091/8206] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:35 2022] DELETE /identity/v3/users/6d61094cf9c34981ac23102c3ba7be9f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:36.109756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.111377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:36.135829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327e58c-f241-422e-8c03-5f5c2e8fb564 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.203188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.203838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.204111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.204769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] PATH_INFO: `/v3/projects/bcb760741ce44cd8902c556318c58222` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.205814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb760741ce44cd8902c556318c58222)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.207936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:36.260790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:36.264810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb760741ce44cd8902c556318c58222', 'request_id': 'req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:36.276384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb760741ce44cd8902c556318c58222', 'request_id': 'req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:36.282773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb760741ce44cd8902c556318c58222', 'request_id': 'req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:36.295472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] The token cache is being invalidate because project bcb760741ce44cd8902c556318c58222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:36.295727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:36.296463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ddf8e8e7-786c-44d0-b87b-2137de2a9e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:36.298263 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4116/8207] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:36 2022] DELETE /identity/v3/projects/bcb760741ce44cd8902c556318c58222 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:36.305033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a10cb-1936-4afc-a57d-40b2b82fa9ff None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.305427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a10cb-1936-4afc-a57d-40b2b82fa9ff None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.305826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a10cb-1936-4afc-a57d-40b2b82fa9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.327232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-872a10cb-1936-4afc-a57d-40b2b82fa9ff None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:36.328461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-872a10cb-1936-4afc-a57d-40b2b82fa9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:36.371339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:36.392951 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4092/8208] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:36.401942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4bf9-47f5-480f-ad61-4e47fcad19c3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.402226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4bf9-47f5-480f-ad61-4e47fcad19c3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.402499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4bf9-47f5-480f-ad61-4e47fcad19c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.423254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a92c4bf9-47f5-480f-ad61-4e47fcad19c3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:36.424467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a92c4bf9-47f5-480f-ad61-4e47fcad19c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:36.433509 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4117/8209] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:36.437877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a10cb-1936-4afc-a57d-40b2b82fa9ff None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.448772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a10cb-1936-4afc-a57d-40b2b82fa9ff None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.454196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.454811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.455094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.455368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.456020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:36.456311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.457958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:36.471029 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4093/8210] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:36.476912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4bf9-47f5-480f-ad61-4e47fcad19c3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.487277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4bf9-47f5-480f-ad61-4e47fcad19c3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.492859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.493533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.493819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.494089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.494728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.496645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:36.532266 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4118/8211] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:36 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:36.536385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.546834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc0642-c5eb-4a6f-ad56-b605b2bf712a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.607597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.608268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.608555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.608863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.609517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.611498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:36.634211 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4094/8212] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:36 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:36.638475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.650196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae381a6e-5647-44ef-a273-d33162e57e4d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.657365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.657979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.658546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.659989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.661579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.663906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:36.676429 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4119/8213] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:36.681310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.692068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b47e-e028-44e3-bb1b-a04dcf333fd0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.698978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad217230-0e63-40bd-9b35-287eaceb2f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.699859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217230-0e63-40bd-9b35-287eaceb2f1a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.700270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217230-0e63-40bd-9b35-287eaceb2f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.700564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217230-0e63-40bd-9b35-287eaceb2f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.701289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217230-0e63-40bd-9b35-287eaceb2f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.702909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217230-0e63-40bd-9b35-287eaceb2f1a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:36.719427 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4095/8214] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:36.723787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.734815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6af94e-d43d-4c5a-ae08-ce6ce6be2276 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.739555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.740221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.740635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.740920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] PATH_INFO: `/v3/projects/e40e561fbe634c9c92fc501c3c8e3590/users/96bfb390daeb440287300b926c8edc22/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.756976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40e561fbe634c9c92fc501c3c8e3590, user_id=96bfb390daeb440287300b926c8edc22, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.759653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:36.774523 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4120/8215] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:36 2022] PUT /identity/v3/projects/e40e561fbe634c9c92fc501c3c8e3590/users/96bfb390daeb440287300b926c8edc22/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:36.787144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09717243-c528-4918-b579-76d1b4595d54 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.787443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09717243-c528-4918-b579-76d1b4595d54 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.787714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09717243-c528-4918-b579-76d1b4595d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.819249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-09717243-c528-4918-b579-76d1b4595d54 None None] MFA Rules not processed for user `96bfb390daeb440287300b926c8edc22`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:36.820383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-09717243-c528-4918-b579-76d1b4595d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:36.897491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4096/8216] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:36.899663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:36.975572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfac19f-a76e-4bbb-9d4c-746412084701 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:36.980579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:36.981225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:36.981480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:36.981764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:36.982439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:36.984337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:37.013452 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4121/8217] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:36 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:37.017927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09717243-c528-4918-b579-76d1b4595d54 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:37.029376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09717243-c528-4918-b579-76d1b4595d54 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:37.072679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:37.093102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:37.093740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:37.094008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:37.094284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:37.094935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:37.096997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:37.123646 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4097/8218] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:37 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:37.125114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:37.135984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f646f-a52d-43d2-af9d-0226bf82ffeb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:37.140813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:37.141425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:37.141725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:37.141991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:37.142650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:37.144286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:37.156023 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4122/8219] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:37.160379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:37.174057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10d77e-fa4e-464d-8eb2-767184beb83c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:37.179064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:37.179739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:37.180020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:37.180299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] PATH_INFO: `/v3/projects/76b530686343447ca12e1baa8032b9c5/users/b8855f21badf4d669b6072b904347fc4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:37.195649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b530686343447ca12e1baa8032b9c5, user_id=b8855f21badf4d669b6072b904347fc4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:37.198358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:37.219571 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4098/8220] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:37 2022] PUT /identity/v3/projects/76b530686343447ca12e1baa8032b9c5/users/b8855f21badf4d669b6072b904347fc4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:37.224870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:37.235683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384cbc2-d2cd-45e4-b3c3-9d83c7fd1bbe admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:37.280149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:37.300981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03613a7-9604-4f55-8290-9c59e1212ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:37.301753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03613a7-9604-4f55-8290-9c59e1212ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:37.301979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03613a7-9604-4f55-8290-9c59e1212ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:37.302254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03613a7-9604-4f55-8290-9c59e1212ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:37.302944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03613a7-9604-4f55-8290-9c59e1212ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:37.304545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03613a7-9604-4f55-8290-9c59e1212ed6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:37.317818 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4123/8221] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:37.319416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:37.330517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116c546-4bbe-41a0-8bc8-b4c6741ad96b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:37.336002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:37.336912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:37.336912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:37.337192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] PATH_INFO: `/v3/projects/76b530686343447ca12e1baa8032b9c5/users/b8855f21badf4d669b6072b904347fc4/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:37.339440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b530686343447ca12e1baa8032b9c5, user_id=b8855f21badf4d669b6072b904347fc4, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:37.342656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:37.354855 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4099/8222] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:37 2022] PUT /identity/v3/projects/76b530686343447ca12e1baa8032b9c5/users/b8855f21badf4d669b6072b904347fc4/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:37.365345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:37.366355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:37.366690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:37.397888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] MFA Rules not processed for user `b8855f21badf4d669b6072b904347fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:37.399127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:37.471962 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4124/8223] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:37.483335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:37.485274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:37.551374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc11d9-480e-4a61-bbd8-9c939760a49c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:37.555665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:37.556392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:37.557595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:37.557955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:37.570030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:37.571616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:37.582105 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4100/8224] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:38.938489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:38.940233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.085243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cfee9-faff-4ecf-8ece-d413d86277f9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:39.091127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e014d3-c066-45ce-b446-b2c45669317b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:39.091762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e014d3-c066-45ce-b446-b2c45669317b service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.092037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e014d3-c066-45ce-b446-b2c45669317b service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.092356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e014d3-c066-45ce-b446-b2c45669317b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.094366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3e014d3-c066-45ce-b446-b2c45669317b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.111327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e014d3-c066-45ce-b446-b2c45669317b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:39.113159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e014d3-c066-45ce-b446-b2c45669317b service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:39.132914 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4125/8225] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:39.372875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:39.374491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.448746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e63bb-e8f4-4d1c-b979-96c859970166 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:39.453585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:39.454312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.454577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.454856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] PATH_INFO: `/v3/users/d859a7dbc66e4add8b6becd36b783136` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.455975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d859a7dbc66e4add8b6becd36b783136)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:39.457924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:39.502708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:39.512188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd859a7dbc66e4add8b6becd36b783136'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.512979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd859a7dbc66e4add8b6becd36b783136'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.520090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd859a7dbc66e4add8b6becd36b783136'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.530206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd859a7dbc66e4add8b6becd36b783136'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.546055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd859a7dbc66e4add8b6becd36b783136'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.553749 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4101/8226] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:39 2022] DELETE /identity/v3/users/d859a7dbc66e4add8b6becd36b783136 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:39.558534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.558817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.559060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.575622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:39.577401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.579860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:39.581155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.617758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:39.648904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b4a8a-713f-4231-8e20-eb2138bb5aaf admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:39.650059 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4126/8227] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:39.654294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:39.654973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.655248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.655515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] PATH_INFO: `/v3/users/96bfb390daeb440287300b926c8edc22` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.656649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96bfb390daeb440287300b926c8edc22)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:39.658985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:39.659375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:39.660859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.715685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96bfb390daeb440287300b926c8edc22'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.716602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96bfb390daeb440287300b926c8edc22'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.724753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff16c5-51c8-4dc9-bafe-d33678d0bd06 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:39.726594 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96bfb390daeb440287300b926c8edc22'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.728537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:39.729326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.729597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.729867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.731803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.735483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96bfb390daeb440287300b926c8edc22'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.745339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:39.746819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:39.749081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a90aa114-7b33-4f36-99f4-994deed76af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96bfb390daeb440287300b926c8edc22'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:39.755316 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4102/8228] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:39 2022] DELETE /identity/v3/users/96bfb390daeb440287300b926c8edc22 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:39.759080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4127/8229] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:39.760484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.760806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.761086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.786007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:39.787362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.820640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:39.855691 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4103/8230] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:39.865733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:39.867636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.881870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473278-a185-40e8-a185-5fcbe55b7ffb service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:39.935347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:39.936965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:39.938240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:39.939107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.939637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.939763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.950603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7b965-cc34-4431-bf15-73821bc3cf50 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:39.955094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:39.956617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:39.957416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:39.957620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:39.957975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:39.958779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] PATH_INFO: `/v3/projects/04ea10d46ecb45dabb545d0d4fcddcb7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:39.960248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ea10d46ecb45dabb545d0d4fcddcb7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:39.962738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:39.978307 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4128/8231] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:40.014797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ea10d46ecb45dabb545d0d4fcddcb7', 'request_id': 'req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.021441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ea10d46ecb45dabb545d0d4fcddcb7', 'request_id': 'req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.030937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ea10d46ecb45dabb545d0d4fcddcb7', 'request_id': 'req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.042526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] The token cache is being invalidate because project 04ea10d46ecb45dabb545d0d4fcddcb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:40.042813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.043473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.045069 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4104/8232] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:39 2022] DELETE /identity/v3/projects/04ea10d46ecb45dabb545d0d4fcddcb7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:40.049412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.061123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8509c5-0850-4709-b05f-0fb225516865 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:40.121078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:40.121866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.122171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.122440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] PATH_INFO: `/v3/users/8489e8ba64a54a20a5e4ebbe028a8779` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.123592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8489e8ba64a54a20a5e4ebbe028a8779)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:40.125739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:40.153920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:40.186218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8489e8ba64a54a20a5e4ebbe028a8779'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.186986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8489e8ba64a54a20a5e4ebbe028a8779'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.196157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8489e8ba64a54a20a5e4ebbe028a8779'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.204903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8489e8ba64a54a20a5e4ebbe028a8779'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.205586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8489e8ba64a54a20a5e4ebbe028a8779'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.223634 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4129/8233] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:40 2022] DELETE /identity/v3/users/8489e8ba64a54a20a5e4ebbe028a8779 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:40.273670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.275794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:40.291770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453b0fb-e987-4c16-bc0c-fa1f64b15d30 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:40.353336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:40.353969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.354242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.354511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] PATH_INFO: `/v3/projects/e40e561fbe634c9c92fc501c3c8e3590` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.355635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40e561fbe634c9c92fc501c3c8e3590)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:40.357567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:40.410811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40e561fbe634c9c92fc501c3c8e3590', 'request_id': 'req-3db48cd8-bb40-4956-a9fc-710f84208480'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.413779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.417050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:40.419769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40e561fbe634c9c92fc501c3c8e3590', 'request_id': 'req-3db48cd8-bb40-4956-a9fc-710f84208480'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.425902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:40.432636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f3975-0a9a-4e91-8c07-1a09976ae7d6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:40.434564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40e561fbe634c9c92fc501c3c8e3590', 'request_id': 'req-3db48cd8-bb40-4956-a9fc-710f84208480'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.451272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] The token cache is being invalidate because project e40e561fbe634c9c92fc501c3c8e3590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:40.451721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.452525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.454239 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4105/8234] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:40 2022] DELETE /identity/v3/projects/e40e561fbe634c9c92fc501c3c8e3590 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:40.458795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.460976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:40.499092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:40.499649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.499924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.500190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] PATH_INFO: `/v3/projects/f24de9d8343746489acbb8bda0e54dcf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.501362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f24de9d8343746489acbb8bda0e54dcf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:40.503408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:40.525596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db48cd8-bb40-4956-a9fc-710f84208480 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:40.530857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:40.531521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.531791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.532099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] PATH_INFO: `/v3/users/b8855f21badf4d669b6072b904347fc4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.533276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8855f21badf4d669b6072b904347fc4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:40.535208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:40.553121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f24de9d8343746489acbb8bda0e54dcf', 'request_id': 'req-ecc07a67-be82-41b8-bbe7-58acdd3be872'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.563163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f24de9d8343746489acbb8bda0e54dcf', 'request_id': 'req-ecc07a67-be82-41b8-bbe7-58acdd3be872'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.569792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f24de9d8343746489acbb8bda0e54dcf', 'request_id': 'req-ecc07a67-be82-41b8-bbe7-58acdd3be872'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.581878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] The token cache is being invalidate because project f24de9d8343746489acbb8bda0e54dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:40.582167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.582841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ecc07a67-be82-41b8-bbe7-58acdd3be872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.584934 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4130/8235] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:40 2022] DELETE /identity/v3/projects/f24de9d8343746489acbb8bda0e54dcf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:40.590120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de4318-0028-46c9-b4ae-adffaea7e4aa None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.590390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de4318-0028-46c9-b4ae-adffaea7e4aa None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.590658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de4318-0028-46c9-b4ae-adffaea7e4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.594081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8855f21badf4d669b6072b904347fc4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.594820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8855f21badf4d669b6072b904347fc4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.602589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8855f21badf4d669b6072b904347fc4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.612564 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-26de4318-0028-46c9-b4ae-adffaea7e4aa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:40.613834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-26de4318-0028-46c9-b4ae-adffaea7e4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:40.616586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8855f21badf4d669b6072b904347fc4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.630116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6dfa0644-9581-4406-a2cc-b2c491a1489a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8855f21badf4d669b6072b904347fc4'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:40.636211 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4106/8236] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:40 2022] DELETE /identity/v3/users/b8855f21badf4d669b6072b904347fc4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:40.642971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:40.684039 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4131/8237] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:40.692068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70c845-69af-4f7a-9bd5-1fb2fb6270c1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.692354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70c845-69af-4f7a-9bd5-1fb2fb6270c1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.692630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70c845-69af-4f7a-9bd5-1fb2fb6270c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.714525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-aa70c845-69af-4f7a-9bd5-1fb2fb6270c1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:40.715756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aa70c845-69af-4f7a-9bd5-1fb2fb6270c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:40.780580 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4107/8238] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:40.784907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de4318-0028-46c9-b4ae-adffaea7e4aa None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.798411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de4318-0028-46c9-b4ae-adffaea7e4aa None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:40.803465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:40.804108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.804932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.804932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.805331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:40.805590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:40.807224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:40.819832 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4132/8239] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:40.824083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70c845-69af-4f7a-9bd5-1fb2fb6270c1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.833851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70c845-69af-4f7a-9bd5-1fb2fb6270c1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:40.838780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:40.839390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.839659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.839925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.840555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:40.842508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:40.847139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:40.879421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4108/8240] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:40 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:40.880817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.891606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397029d-2e68-4fe6-b1bf-02bfa6284e6c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:40.937675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:40.939565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:40.964024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:40.964936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:40.965333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:40.966066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:40.966938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:40.969471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:40.997294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4133/8241] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:40 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:41.001409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.011239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e9b3-dd1f-456a-a840-49a9669e0963 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.015995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.016590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.016910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.017165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.017797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.019419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.023228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5632b74-f831-462f-8bd3-25ccfe718a78 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.027923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.028595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.028908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.029222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] PATH_INFO: `/v3/projects/76b530686343447ca12e1baa8032b9c5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.030393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b530686343447ca12e1baa8032b9c5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.030768 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4134/8242] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:41.032466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.034661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.045494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99f228-aeb7-4706-8468-4e389957b5e4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.050200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.050813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.051089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.051360 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.051996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.054494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.065081 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4135/8243] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:41.069568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.079931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115b9fd-bba0-4058-96f1-b6bdb1c31529 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.088277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b530686343447ca12e1baa8032b9c5', 'request_id': 'req-3446ddd8-7c0e-48b4-9860-16979ff0a376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:41.094696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b530686343447ca12e1baa8032b9c5', 'request_id': 'req-3446ddd8-7c0e-48b4-9860-16979ff0a376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:41.105351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b530686343447ca12e1baa8032b9c5', 'request_id': 'req-3446ddd8-7c0e-48b4-9860-16979ff0a376'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:41.116969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] The token cache is being invalidate because project 76b530686343447ca12e1baa8032b9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:41.117377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:41.118113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-3446ddd8-7c0e-48b4-9860-16979ff0a376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:41.119916 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4109/8244] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:40 2022] DELETE /identity/v3/projects/76b530686343447ca12e1baa8032b9c5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:41.124555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae34da-6f36-473f-8d2e-077b6a126290 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.124873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae34da-6f36-473f-8d2e-077b6a126290 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.125145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae34da-6f36-473f-8d2e-077b6a126290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.140196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e9006-54a9-477b-abcf-a4be89c32e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.140893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e9006-54a9-477b-abcf-a4be89c32e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.141166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e9006-54a9-477b-abcf-a4be89c32e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.141432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e9006-54a9-477b-abcf-a4be89c32e64 admin admin] PATH_INFO: `/v3/projects/4e6bddee3dff4231b15208f6ccba4245/users/8fb604570ba64af29a4cb03a4541133d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.144551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-aeae34da-6f36-473f-8d2e-077b6a126290 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:41.145805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aeae34da-6f36-473f-8d2e-077b6a126290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:41.154534 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4110/8245] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:41.156075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e9006-54a9-477b-abcf-a4be89c32e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6bddee3dff4231b15208f6ccba4245, user_id=8fb604570ba64af29a4cb03a4541133d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.159176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e9006-54a9-477b-abcf-a4be89c32e64 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.162846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4406d6a-0924-421b-b428-e11ef833c13d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.163134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4406d6a-0924-421b-b428-e11ef833c13d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.163408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4406d6a-0924-421b-b428-e11ef833c13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.173269 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4136/8246] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:41 2022] PUT /identity/v3/projects/4e6bddee3dff4231b15208f6ccba4245/users/8fb604570ba64af29a4cb03a4541133d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:34:41.183670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.183948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.184212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.185300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f4406d6a-0924-421b-b428-e11ef833c13d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:41.186555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f4406d6a-0924-421b-b428-e11ef833c13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:41.198316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:41.216527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] MFA Rules not processed for user `8fb604570ba64af29a4cb03a4541133d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:41.218017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:41.260915 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4111/8247] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:41.266014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4406d6a-0924-421b-b428-e11ef833c13d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.276360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4406d6a-0924-421b-b428-e11ef833c13d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.281661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.282275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.282561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.283160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.283822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:41.284131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.286079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.288684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:41.289472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4137/8248] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:41.300441 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4112/8249] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:41.301945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.304247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:41.305983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.315938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc18f3-b489-4f30-975f-cefa1781874e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.320793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.321396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.321673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.321946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.322566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.324457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.355093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4113/8250] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:41 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:41.359553 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.367659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d11d49-b1d6-402b-b504-d5de50261ea2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.419580 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.420195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.420473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.420792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.421901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62658b86-e72b-4eaf-8adc-7a175025fc75 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.426722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.427405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.428211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.428487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.429155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.431118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.457627 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4114/8251] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:41 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:41.459790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.469607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7448cb-1bf8-4e81-8199-7669e89b0428 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.475407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.476011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.476287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.476552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.477228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.478814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.485733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.487164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.488894 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4115/8252] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:41.493374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.497561 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4138/8253] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:34:41.503511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7175bee-055f-4744-b150-0bde60b14793 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.508823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed36da-1013-4308-8e74-0170a04628f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.509428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed36da-1013-4308-8e74-0170a04628f7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.509709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed36da-1013-4308-8e74-0170a04628f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.509981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed36da-1013-4308-8e74-0170a04628f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.510625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed36da-1013-4308-8e74-0170a04628f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.512278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed36da-1013-4308-8e74-0170a04628f7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.523865 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4116/8254] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:41.528405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.539847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273a5d1-483a-4016-80b7-fd5f9be3a357 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.544896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.545559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.545863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.546134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] PATH_INFO: `/v3/projects/68a594ac511e4aa09bb3db613f4e7083/users/19ebf9b56f7b4131a18cab75d54aced7/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.548558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:41.564381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a594ac511e4aa09bb3db613f4e7083, user_id=19ebf9b56f7b4131a18cab75d54aced7, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.567366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.585190 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4139/8255] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:41 2022] PUT /identity/v3/projects/68a594ac511e4aa09bb3db613f4e7083/users/19ebf9b56f7b4131a18cab75d54aced7/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:41.597548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77c2c-43ce-4e34-a896-0543834d1b5e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.598074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77c2c-43ce-4e34-a896-0543834d1b5e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.598353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77c2c-43ce-4e34-a896-0543834d1b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.636309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-abd77c2c-43ce-4e34-a896-0543834d1b5e None None] MFA Rules not processed for user `19ebf9b56f7b4131a18cab75d54aced7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:41.637614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-abd77c2c-43ce-4e34-a896-0543834d1b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:41.712260 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4117/8256] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:41.722617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:41.724747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:41.791397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf431c18-a644-4556-a1b6-32d9c47dcbcc admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:41.795365 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9a77a-6040-4c27-ba07-cd8e9c7ab555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:41.796092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a77a-6040-4c27-ba07-cd8e9c7ab555 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:41.796353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a77a-6040-4c27-ba07-cd8e9c7ab555 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:41.796631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a77a-6040-4c27-ba07-cd8e9c7ab555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:41.805939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9a77a-6040-4c27-ba07-cd8e9c7ab555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:41.807383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9a77a-6040-4c27-ba07-cd8e9c7ab555 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:41.816822 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4140/8257] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:48.269509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd77c2c-43ce-4e34-a896-0543834d1b5e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:48.308627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:48.341144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd77c2c-43ce-4e34-a896-0543834d1b5e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:48.345921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:48.346568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:48.346848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:48.347126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] PATH_INFO: `/v3/users/19ebf9b56f7b4131a18cab75d54aced7` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:48.348330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ebf9b56f7b4131a18cab75d54aced7)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:48.350466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:48.398633 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ebf9b56f7b4131a18cab75d54aced7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:48.399490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ebf9b56f7b4131a18cab75d54aced7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:48.406786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ebf9b56f7b4131a18cab75d54aced7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:48.416174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ebf9b56f7b4131a18cab75d54aced7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:48.428951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ebf9b56f7b4131a18cab75d54aced7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:48.435198 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4118/8258] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:48 2022] DELETE /identity/v3/users/19ebf9b56f7b4131a18cab75d54aced7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:48.439779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:48.440248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:48.440362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:48.462251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:48.463545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:48.525451 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4141/8259] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:48.536304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:48.537612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:48.629562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd648a40-ac21-402c-b4d6-b11ee2705d17 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:48.642835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a900e5-1ed5-45f0-bcd7-05a0a6d41144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:48.643549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a900e5-1ed5-45f0-bcd7-05a0a6d41144 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:48.643893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a900e5-1ed5-45f0-bcd7-05a0a6d41144 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:48.644236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a900e5-1ed5-45f0-bcd7-05a0a6d41144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:48.655415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a900e5-1ed5-45f0-bcd7-05a0a6d41144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:48.656901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a900e5-1ed5-45f0-bcd7-05a0a6d41144 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:48.667506 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4119/8260] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:49.040541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.042078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:49.056124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02019d3f-40c6-4665-92ae-bed2d1754996 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.060879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.061668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.061945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.062215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] PATH_INFO: `/v3/projects/68a594ac511e4aa09bb3db613f4e7083` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.063349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a594ac511e4aa09bb3db613f4e7083)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.065285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.126017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a594ac511e4aa09bb3db613f4e7083', 'request_id': 'req-73e66c2b-3a84-437a-a231-782c78df599c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:49.136268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a594ac511e4aa09bb3db613f4e7083', 'request_id': 'req-73e66c2b-3a84-437a-a231-782c78df599c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:49.144197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a594ac511e4aa09bb3db613f4e7083', 'request_id': 'req-73e66c2b-3a84-437a-a231-782c78df599c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:49.162189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] The token cache is being invalidate because project 68a594ac511e4aa09bb3db613f4e7083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:49.164941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:49.164941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-73e66c2b-3a84-437a-a231-782c78df599c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:49.168089 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4142/8261] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:49 2022] DELETE /identity/v3/projects/68a594ac511e4aa09bb3db613f4e7083 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:49.171399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958f84-d4ba-4924-ad6c-1063b515402f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.172216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958f84-d4ba-4924-ad6c-1063b515402f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.172642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958f84-d4ba-4924-ad6c-1063b515402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.195761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-df958f84-d4ba-4924-ad6c-1063b515402f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:49.197523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-df958f84-d4ba-4924-ad6c-1063b515402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:49.254060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:49.268766 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4120/8262] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:34:49.276972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b38f31-69a0-41a6-b29a-a5b37ac8e762 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.277253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b38f31-69a0-41a6-b29a-a5b37ac8e762 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.277528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b38f31-69a0-41a6-b29a-a5b37ac8e762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.298369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-78b38f31-69a0-41a6-b29a-a5b37ac8e762 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:49.299546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-78b38f31-69a0-41a6-b29a-a5b37ac8e762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:49.308817 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4143/8263] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:49.313232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958f84-d4ba-4924-ad6c-1063b515402f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.323556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958f84-d4ba-4924-ad6c-1063b515402f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.328575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.329281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.329583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.329877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.330563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:49.330866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.334024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.350594 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4121/8264] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:49.352137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b38f31-69a0-41a6-b29a-a5b37ac8e762 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.356791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:49.363166 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b38f31-69a0-41a6-b29a-a5b37ac8e762 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.371245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.372070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.372361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.372640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.373328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.375670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.407179 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4144/8265] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:49 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:49.411597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.422520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44b0d-31a0-4b09-9149-5fe7480e833c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.491190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.491819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.492101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.492376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.493298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.496996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.527829 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4122/8266] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:49 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:49.529101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.540842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a89fa-fb25-4df0-af2d-49fad072ed2d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.545779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.546389 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.546663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.546929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.547560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.549297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.566999 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4145/8267] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:49.568475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.579906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe102f-732e-432f-9859-0eb9de941eee admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.587676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4db2df-a723-4fec-a2cb-3b8c230bcd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.589395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4db2df-a723-4fec-a2cb-3b8c230bcd99 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.589672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4db2df-a723-4fec-a2cb-3b8c230bcd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.589951 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4db2df-a723-4fec-a2cb-3b8c230bcd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.590660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4db2df-a723-4fec-a2cb-3b8c230bcd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.592389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4db2df-a723-4fec-a2cb-3b8c230bcd99 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.610907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4123/8268] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:49.615465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.626588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8dec7-b9ab-4832-9132-99c0a956c1f8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.635045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.635837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.636187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.636520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] PATH_INFO: `/v3/projects/9b358f17d02a462ba3d18a96adcbe35b/users/30f09132ad1f466e999edad15f20036b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.654812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b358f17d02a462ba3d18a96adcbe35b, user_id=30f09132ad1f466e999edad15f20036b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.657608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.678900 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4146/8269] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:49 2022] PUT /identity/v3/projects/9b358f17d02a462ba3d18a96adcbe35b/users/30f09132ad1f466e999edad15f20036b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:49.689973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.690252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.690525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.735395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] MFA Rules not processed for user `30f09132ad1f466e999edad15f20036b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:49.736694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:49.823317 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4124/8270] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:49.836879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:49.839639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:49.911700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c103-fd3f-4a1f-9a81-41defa032c86 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:49.915751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb500ed-9b68-4df6-a1c5-9a28d6fa3485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:49.916407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb500ed-9b68-4df6-a1c5-9a28d6fa3485 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:49.916798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb500ed-9b68-4df6-a1c5-9a28d6fa3485 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:49.917074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb500ed-9b68-4df6-a1c5-9a28d6fa3485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:49.926921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb500ed-9b68-4df6-a1c5-9a28d6fa3485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:49.928478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb500ed-9b68-4df6-a1c5-9a28d6fa3485 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:49.938725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4147/8271] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:54.578508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:54.580316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:54.655331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:54.675030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502309c-a52e-4315-88d0-a55ca334c592 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:54.680060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:54.680731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:54.681009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:54.681399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] PATH_INFO: `/v3/users/8fb604570ba64af29a4cb03a4541133d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:54.682814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb604570ba64af29a4cb03a4541133d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:54.684784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:54.735244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb604570ba64af29a4cb03a4541133d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:54.735890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb604570ba64af29a4cb03a4541133d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:54.743509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb604570ba64af29a4cb03a4541133d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:54.758149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb604570ba64af29a4cb03a4541133d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:54.771705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb604570ba64af29a4cb03a4541133d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:54.777618 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4125/8272] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:34:54 2022] DELETE /identity/v3/users/8fb604570ba64af29a4cb03a4541133d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:54.782363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:54.782891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:54.782891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:54.803142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:54.804300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:54.885371 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4148/8273] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:54.893058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:54.896233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:54.965711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6effd2-83af-4b1c-8d55-71491e50c764 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:54.969475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c6f2-90da-4bca-8376-22a633f89671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:54.970361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c6f2-90da-4bca-8376-22a633f89671 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:54.971239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c6f2-90da-4bca-8376-22a633f89671 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:54.971518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c6f2-90da-4bca-8376-22a633f89671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:54.983412 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c6f2-90da-4bca-8376-22a633f89671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:54.984945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c6f2-90da-4bca-8376-22a633f89671 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:54.997318 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4126/8274] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:34:55.276153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:55.277852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:55.291263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b8ddb-78a1-4e3b-85b2-c1713f2ae754 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:55.300702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:55.301595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.301994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.302257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] PATH_INFO: `/v3/projects/4e6bddee3dff4231b15208f6ccba4245` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.303506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6bddee3dff4231b15208f6ccba4245)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:55.305447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:55.364614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:55.368682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6bddee3dff4231b15208f6ccba4245', 'request_id': 'req-64bfb1b0-55cc-4208-831b-f7678361066c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:55.380092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6bddee3dff4231b15208f6ccba4245', 'request_id': 'req-64bfb1b0-55cc-4208-831b-f7678361066c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:55.386798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6bddee3dff4231b15208f6ccba4245', 'request_id': 'req-64bfb1b0-55cc-4208-831b-f7678361066c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:55.399404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] The token cache is being invalidate because project 4e6bddee3dff4231b15208f6ccba4245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:34:55.399724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:55.400472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-64bfb1b0-55cc-4208-831b-f7678361066c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:34:55.402120 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4149/8275] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:34:55 2022] DELETE /identity/v3/projects/4e6bddee3dff4231b15208f6ccba4245 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:55.407179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eff275-934a-4e1a-99ef-6155ba1141a4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.407467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eff275-934a-4e1a-99ef-6155ba1141a4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.407800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eff275-934a-4e1a-99ef-6155ba1141a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.431444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-35eff275-934a-4e1a-99ef-6155ba1141a4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:55.432410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-35eff275-934a-4e1a-99ef-6155ba1141a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:55.670309 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4127/8276] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:55.679477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6f60-9509-44b4-8915-36400b5effd2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.679764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6f60-9509-44b4-8915-36400b5effd2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.680041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6f60-9509-44b4-8915-36400b5effd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.701240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-09ec6f60-9509-44b4-8915-36400b5effd2 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:55.702590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-09ec6f60-9509-44b4-8915-36400b5effd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:55.715616 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4150/8277] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:55.720031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eff275-934a-4e1a-99ef-6155ba1141a4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:55.732342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eff275-934a-4e1a-99ef-6155ba1141a4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:55.738423 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:55.739145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.739422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.739702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.746155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:34:55.746489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:55.748948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:55.756768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:55.763896 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4128/8278] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:34:55.768276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6f60-9509-44b4-8915-36400b5effd2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:55.778210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec6f60-9509-44b4-8915-36400b5effd2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:55.783135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:55.783744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.784004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.784276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.785036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:55.786971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:55.820009 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4151/8279] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:34:55 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:34:55.821228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:55.832282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10881b-44f3-4a23-81ce-97026b154a54 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:55.890703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:55.891377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.891651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.891926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.892570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:55.894612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:55.917646 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4129/8280] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:34:55 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:34:55.921820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:55.944384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235a59d-f503-4286-8f89-bd7666c6b439 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:55.950158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:55.950767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.951046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.951315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.951987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:55.953656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:55.965104 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4152/8281] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:34:55.969583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:55.980591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61d8b7-62a1-404c-8657-0b95d42f1b16 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:55.985612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f2e6c-3f4d-479e-b1bf-b2f48e84d91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:55.986217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f2e6c-3f4d-479e-b1bf-b2f48e84d91a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:55.986493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f2e6c-3f4d-479e-b1bf-b2f48e84d91a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:55.986765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f2e6c-3f4d-479e-b1bf-b2f48e84d91a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:55.987413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f2e6c-3f4d-479e-b1bf-b2f48e84d91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:55.989076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f2e6c-3f4d-479e-b1bf-b2f48e84d91a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:55.999694 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4130/8282] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:34:56.004424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:56.014546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a1ef7-c60f-4fab-aec7-904258983036 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:56.019332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:56.020195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:56.020480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:56.020796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] PATH_INFO: `/v3/projects/38c8c2b14abb435e95a12ecb732c2ccb/users/cd6e82e6a1154178825f698201529704/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:56.037933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c8c2b14abb435e95a12ecb732c2ccb, user_id=cd6e82e6a1154178825f698201529704, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:56.040627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:56.055339 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4153/8283] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:34:56 2022] PUT /identity/v3/projects/38c8c2b14abb435e95a12ecb732c2ccb/users/cd6e82e6a1154178825f698201529704/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:34:56.066234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:56.066522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:56.066797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:56.096641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] MFA Rules not processed for user `cd6e82e6a1154178825f698201529704`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:34:56.097876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:56.161669 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4131/8284] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:34:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:34:56.171886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:34:56.173623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:34:56.233050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766871-0a4b-413c-af63-353bf075e520 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:34:56.236817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee85816-bb33-4208-af8e-10961313eb90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:34:56.237473 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85816-bb33-4208-af8e-10961313eb90 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:34:56.237749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85816-bb33-4208-af8e-10961313eb90 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:34:56.238099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee85816-bb33-4208-af8e-10961313eb90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:34:56.246697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee85816-bb33-4208-af8e-10961313eb90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:34:56.248220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee85816-bb33-4208-af8e-10961313eb90 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:34:56.257558 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4154/8285] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:34:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:02.386162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:02.387944 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:02.463020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eb361-4196-402f-a002-8f8e864ba2be None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:02.467889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:02.468736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:02.469020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:02.469291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] PATH_INFO: `/v3/users/30f09132ad1f466e999edad15f20036b` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:02.470503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f09132ad1f466e999edad15f20036b)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:02.472581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:02.481058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:02.538703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f09132ad1f466e999edad15f20036b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:02.539475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f09132ad1f466e999edad15f20036b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:02.551713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f09132ad1f466e999edad15f20036b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:02.563225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f09132ad1f466e999edad15f20036b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:02.580393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f09132ad1f466e999edad15f20036b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:02.590084 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4132/8286] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:35:02 2022] DELETE /identity/v3/users/30f09132ad1f466e999edad15f20036b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:02.594744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:02.595354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:02.595354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:02.634537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:02.635781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:02.710835 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4155/8287] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:02.720848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:02.726709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:02.811196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a735-998a-483d-af27-f9821b690665 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:02.816597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2058e05-ab1b-4c0b-a1ad-b35378391056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:02.817280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2058e05-ab1b-4c0b-a1ad-b35378391056 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:02.817555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2058e05-ab1b-4c0b-a1ad-b35378391056 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:02.817845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2058e05-ab1b-4c0b-a1ad-b35378391056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:02.829728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2058e05-ab1b-4c0b-a1ad-b35378391056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:02.832453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2058e05-ab1b-4c0b-a1ad-b35378391056 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:02.843369 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4133/8288] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:03.126645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.128724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:03.148982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cf9a3-4c7c-4969-bf54-644351ea366b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.153958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.154977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.155513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.156363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] PATH_INFO: `/v3/projects/9b358f17d02a462ba3d18a96adcbe35b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.159326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b358f17d02a462ba3d18a96adcbe35b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.161996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.169053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:03.213819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b358f17d02a462ba3d18a96adcbe35b', 'request_id': 'req-172ec547-566f-473c-902b-b923de485bb5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:03.224275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b358f17d02a462ba3d18a96adcbe35b', 'request_id': 'req-172ec547-566f-473c-902b-b923de485bb5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:03.230796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b358f17d02a462ba3d18a96adcbe35b', 'request_id': 'req-172ec547-566f-473c-902b-b923de485bb5'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:03.242391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] The token cache is being invalidate because project 9b358f17d02a462ba3d18a96adcbe35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:35:03.242691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:03.243648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-172ec547-566f-473c-902b-b923de485bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:03.245161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4156/8289] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:35:03 2022] DELETE /identity/v3/projects/9b358f17d02a462ba3d18a96adcbe35b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:03.249818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ee7ea-472a-4cc4-a34d-3ac62ec4f89f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.250162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ee7ea-472a-4cc4-a34d-3ac62ec4f89f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.250504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ee7ea-472a-4cc4-a34d-3ac62ec4f89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.272891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a85ee7ea-472a-4cc4-a34d-3ac62ec4f89f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:03.274145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a85ee7ea-472a-4cc4-a34d-3ac62ec4f89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:03.350559 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4134/8290] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:03.358807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93b3a7-7fc9-421f-ae18-a1e893f7f815 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.359119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93b3a7-7fc9-421f-ae18-a1e893f7f815 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.359404 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93b3a7-7fc9-421f-ae18-a1e893f7f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.383275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3a93b3a7-7fc9-421f-ae18-a1e893f7f815 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:03.384501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3a93b3a7-7fc9-421f-ae18-a1e893f7f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:03.393556 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4157/8291] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:03.397413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ee7ea-472a-4cc4-a34d-3ac62ec4f89f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.414489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ee7ea-472a-4cc4-a34d-3ac62ec4f89f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.421829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.422651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.423016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.423379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.423942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:35:03.424806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.425961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.443042 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4135/8292] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:35:03.445238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93b3a7-7fc9-421f-ae18-a1e893f7f815 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.457128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93b3a7-7fc9-421f-ae18-a1e893f7f815 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.463110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.464130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.464604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.465145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.465968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.468425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.505078 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4158/8293] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:35:03 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:35:03.505884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.516638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef4357-1561-4a73-821e-46e0322f063c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.534046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:03.591230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.591896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.592177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.592452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.593219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.595281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.624456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4136/8294] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:35:03 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:35:03.625986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.637181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16d961-4dd5-41a9-afaf-1e06719738f5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.642261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.642896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.643305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.643613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.644272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.645964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.657256 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4159/8295] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:35:03.661464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.671996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b256d58-c3dd-4f43-ad56-1086f9c5970c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.677406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cf26c-b09d-4126-a1b5-daefbfa30d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.678026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cf26c-b09d-4126-a1b5-daefbfa30d1a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.678274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cf26c-b09d-4126-a1b5-daefbfa30d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.678551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cf26c-b09d-4126-a1b5-daefbfa30d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.679233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cf26c-b09d-4126-a1b5-daefbfa30d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.681010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cf26c-b09d-4126-a1b5-daefbfa30d1a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.692517 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4137/8296] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:35:03.696996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.709548 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5625a-3167-4cab-afb3-ca9a91b30c3b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.715493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.716111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.716376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.716868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] PATH_INFO: `/v3/projects/8fd01ef96cf4484aafa19d3c06846485/users/1d88394f6bba434c9312a3d0e08a02b0/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.734035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd01ef96cf4484aafa19d3c06846485, user_id=1d88394f6bba434c9312a3d0e08a02b0, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.736734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.752320 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4160/8297] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:35:03 2022] PUT /identity/v3/projects/8fd01ef96cf4484aafa19d3c06846485/users/1d88394f6bba434c9312a3d0e08a02b0/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:03.763504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1657b-8889-4e5f-abcf-af6fcabb7afa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.763781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1657b-8889-4e5f-abcf-af6fcabb7afa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.764054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1657b-8889-4e5f-abcf-af6fcabb7afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.792836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-62e1657b-8889-4e5f-abcf-af6fcabb7afa None None] MFA Rules not processed for user `1d88394f6bba434c9312a3d0e08a02b0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:03.794129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62e1657b-8889-4e5f-abcf-af6fcabb7afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:03.864978 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4138/8298] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:03.875991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:03.877991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:03.945154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0741f-7164-4db2-b918-af8735632c8a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:03.949688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f987e-1c76-417c-a7da-0694cfd1b316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:03.950433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f987e-1c76-417c-a7da-0694cfd1b316 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:03.951599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f987e-1c76-417c-a7da-0694cfd1b316 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:03.951830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f987e-1c76-417c-a7da-0694cfd1b316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:03.963186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f987e-1c76-417c-a7da-0694cfd1b316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:03.964969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f987e-1c76-417c-a7da-0694cfd1b316 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:03.977655 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4161/8299] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:13.831872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1657b-8889-4e5f-abcf-af6fcabb7afa None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:13.912227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1657b-8889-4e5f-abcf-af6fcabb7afa None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:13.920886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:13.921563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:13.921841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:13.922722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] PATH_INFO: `/v3/users/1d88394f6bba434c9312a3d0e08a02b0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:13.923485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d88394f6bba434c9312a3d0e08a02b0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:13.926946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:13.968475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:13.971513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:13.999869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d88394f6bba434c9312a3d0e08a02b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.000544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d88394f6bba434c9312a3d0e08a02b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.008451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d88394f6bba434c9312a3d0e08a02b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.018888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d88394f6bba434c9312a3d0e08a02b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.032911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d88394f6bba434c9312a3d0e08a02b0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.040421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4139/8300] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:35:13 2022] DELETE /identity/v3/users/1d88394f6bba434c9312a3d0e08a02b0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:14.045394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:14.045677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:14.045957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:14.067847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:14.069384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:14.143572 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4162/8301] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:14.151991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:14.153871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:14.228926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a192d0-2acf-4d4e-b074-cbc164fa9ca5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:14.233843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b03c7-f319-45e7-b6f3-3f76919ba387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:14.234167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b03c7-f319-45e7-b6f3-3f76919ba387 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:14.234541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b03c7-f319-45e7-b6f3-3f76919ba387 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:14.234907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b03c7-f319-45e7-b6f3-3f76919ba387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:14.245531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b03c7-f319-45e7-b6f3-3f76919ba387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:14.247178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b03c7-f319-45e7-b6f3-3f76919ba387 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:14.258313 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4140/8302] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:14.564204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:14.566861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:14.599041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91763c20-49cc-43ff-94a5-70f73b77dfa6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:14.621907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:14.622784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:14.623071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:14.623348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] PATH_INFO: `/v3/projects/8fd01ef96cf4484aafa19d3c06846485` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:14.625006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd01ef96cf4484aafa19d3c06846485)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:14.627229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:14.694813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd01ef96cf4484aafa19d3c06846485', 'request_id': 'req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.708377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd01ef96cf4484aafa19d3c06846485', 'request_id': 'req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.717771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd01ef96cf4484aafa19d3c06846485', 'request_id': 'req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.730979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] The token cache is being invalidate because project 8fd01ef96cf4484aafa19d3c06846485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:35:14.731406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.732143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9aeba95e-a1fb-4e0f-b8d8-86eadbc70d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:14.736745 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4163/8303] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:35:14 2022] DELETE /identity/v3/projects/8fd01ef96cf4484aafa19d3c06846485 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:14.740280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fbd6-7d48-4850-8445-b367a47579e4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:14.740615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fbd6-7d48-4850-8445-b367a47579e4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:14.740943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fbd6-7d48-4850-8445-b367a47579e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:14.766487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-de52fbd6-7d48-4850-8445-b367a47579e4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:14.767799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-de52fbd6-7d48-4850-8445-b367a47579e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:14.838503 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4141/8304] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:14.846238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b804d10-a080-44d3-b4f0-5421129c1705 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:14.846522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b804d10-a080-44d3-b4f0-5421129c1705 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:14.846796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b804d10-a080-44d3-b4f0-5421129c1705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:14.874444 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4b804d10-a080-44d3-b4f0-5421129c1705 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:14.875984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4b804d10-a080-44d3-b4f0-5421129c1705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:14.887411 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4164/8305] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:14.891963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52fbd6-7d48-4850-8445-b367a47579e4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:14.903007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52fbd6-7d48-4850-8445-b367a47579e4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:14.908989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:14.909719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:14.910064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:14.910405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:14.911142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:35:14.911513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:14.913310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:14.928460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4142/8306] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:35:14.934055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b804d10-a080-44d3-b4f0-5421129c1705 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:14.946220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b804d10-a080-44d3-b4f0-5421129c1705 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:14.957500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:14.958180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:14.958418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:14.958685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:14.960133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:14.963109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:14.994863 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4165/8307] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:35:14 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:35:14.999501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:15.011224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef1190-515a-498c-985a-998c28527fa2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:15.046557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:15.087091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:15.088681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:15.088681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:15.088681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:15.089248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:15.091481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:15.124591 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4143/8308] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:35:14 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:35:15.126293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:15.144622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e7cba-98e9-4406-9b90-d3d1ade5f8f9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:15.152216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:15.152999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:15.153518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:15.153896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:15.154635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:15.156362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:15.170753 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4166/8309] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:35:15.175448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:15.191517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c28d1-c7ef-4468-8b91-2f556601a311 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:15.196811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa8fa7-ef62-430f-aca2-64d7ae6abf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:15.197687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa8fa7-ef62-430f-aca2-64d7ae6abf60 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:15.198146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa8fa7-ef62-430f-aca2-64d7ae6abf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:15.198607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa8fa7-ef62-430f-aca2-64d7ae6abf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:15.199452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa8fa7-ef62-430f-aca2-64d7ae6abf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:15.201440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa8fa7-ef62-430f-aca2-64d7ae6abf60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:15.216952 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4144/8310] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:35:15.222386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:15.234239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892eead-c8df-4f84-86a5-d3ea2464a071 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:15.239861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:15.240616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:15.241141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:15.241481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] PATH_INFO: `/v3/projects/93d0ff16192d4f319da6ff81d6d57ae7/users/679234e2323d4bb2b50e5991e315bf85/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:15.258721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d0ff16192d4f319da6ff81d6d57ae7, user_id=679234e2323d4bb2b50e5991e315bf85, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:15.261543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:15.277471 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4167/8311] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:35:15 2022] PUT /identity/v3/projects/93d0ff16192d4f319da6ff81d6d57ae7/users/679234e2323d4bb2b50e5991e315bf85/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:15.288709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:15.289145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:15.290114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:15.326843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] MFA Rules not processed for user `679234e2323d4bb2b50e5991e315bf85`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:15.328076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:15.410202 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4145/8312] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:15.422443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:15.425800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:15.453026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:15.518109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9a3db-147f-4366-8ea2-94394f495d91 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:15.523559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:15.526727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:15.527386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:15.527763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:15.541245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:15.542699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:15.554636 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4168/8313] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:25.406021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:25.549927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=17 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:41.591961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:41.593619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:41.742538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71cc91-0d98-43a5-acf2-22158ae67993 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:41.749902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:41.750543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:41.750967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:41.751413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:41.753445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:41.771433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:41.773433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:41.792036 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4146/8314] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:44.394460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:44.412127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d31e9-836a-4fc7-9c31-22ab95a46be7 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:44.423916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c922954-9473-42bd-ba05-1fe3402ccb51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:44.424855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c922954-9473-42bd-ba05-1fe3402ccb51 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:44.425284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c922954-9473-42bd-ba05-1fe3402ccb51 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:44.425951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c922954-9473-42bd-ba05-1fe3402ccb51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:44.429469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7c922954-9473-42bd-ba05-1fe3402ccb51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:44.449451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c922954-9473-42bd-ba05-1fe3402ccb51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:44.454642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c922954-9473-42bd-ba05-1fe3402ccb51 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:44.471013 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4169/8315] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:51.781075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=32 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:53.028242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:53.029789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:53.203338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1749838-8832-4a26-b93b-cdf1baa221d7 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:53.219712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:53.228678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:53.228678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:53.228678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] PATH_INFO: `/v3/users/cd6e82e6a1154178825f698201529704` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:53.228678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd6e82e6a1154178825f698201529704)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:53.228678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:53.342294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd6e82e6a1154178825f698201529704'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:53.344646 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd6e82e6a1154178825f698201529704'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:53.357097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd6e82e6a1154178825f698201529704'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:53.381058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd6e82e6a1154178825f698201529704'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:53.418740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd6e82e6a1154178825f698201529704'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:53.433633 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4147/8316] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:35:53 2022] DELETE /identity/v3/users/cd6e82e6a1154178825f698201529704 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:53.438565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:53.439500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:53.439992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:53.472308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:53.473609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:53.588935 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4170/8317] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:53.616074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:53.618674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:53.726893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe24db-0575-4172-9a47-b5c07ce4215f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:53.731961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae121343-d249-4d37-9582-8c8d3183303a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:53.732971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae121343-d249-4d37-9582-8c8d3183303a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:53.733467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae121343-d249-4d37-9582-8c8d3183303a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:53.733971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae121343-d249-4d37-9582-8c8d3183303a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:53.750988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae121343-d249-4d37-9582-8c8d3183303a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:53.753023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae121343-d249-4d37-9582-8c8d3183303a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:53.764818 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4148/8318] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:35:54.000300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.003369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:54.016919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccc8c5-5bbb-42be-aaa5-5f1e46b31098 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.025529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.026660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.026660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.031899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] PATH_INFO: `/v3/projects/38c8c2b14abb435e95a12ecb732c2ccb` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.037071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c8c2b14abb435e95a12ecb732c2ccb)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.039993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.092232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c8c2b14abb435e95a12ecb732c2ccb', 'request_id': 'req-347a8629-5832-4e87-ac2f-4dd0cac3eca0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:54.102423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c8c2b14abb435e95a12ecb732c2ccb', 'request_id': 'req-347a8629-5832-4e87-ac2f-4dd0cac3eca0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:54.108991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c8c2b14abb435e95a12ecb732c2ccb', 'request_id': 'req-347a8629-5832-4e87-ac2f-4dd0cac3eca0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:54.120719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] The token cache is being invalidate because project 38c8c2b14abb435e95a12ecb732c2ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:35:54.121224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:54.121944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-347a8629-5832-4e87-ac2f-4dd0cac3eca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:35:54.127499 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4171/8319] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:35:53 2022] DELETE /identity/v3/projects/38c8c2b14abb435e95a12ecb732c2ccb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:54.130351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73993ca2-e2dc-4b12-b338-066d7c30276c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.130855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73993ca2-e2dc-4b12-b338-066d7c30276c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.130983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73993ca2-e2dc-4b12-b338-066d7c30276c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.152011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-73993ca2-e2dc-4b12-b338-066d7c30276c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:54.153541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-73993ca2-e2dc-4b12-b338-066d7c30276c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:54.174164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:54.231150 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4149/8320] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:54.236681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03230d-cca3-4472-aa2f-fa60157ccddd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.237422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03230d-cca3-4472-aa2f-fa60157ccddd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.237422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03230d-cca3-4472-aa2f-fa60157ccddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.258184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fe03230d-cca3-4472-aa2f-fa60157ccddd None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:54.259399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fe03230d-cca3-4472-aa2f-fa60157ccddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:54.268050 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4172/8321] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:54.272351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73993ca2-e2dc-4b12-b338-066d7c30276c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.282474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73993ca2-e2dc-4b12-b338-066d7c30276c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.287028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.287635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.287908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.288182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.288855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:35:54.289153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.290746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.303307 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4150/8322] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:35:54.307798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03230d-cca3-4472-aa2f-fa60157ccddd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.317467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03230d-cca3-4472-aa2f-fa60157ccddd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.322583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.323259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.323532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.323795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.324427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.326913 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.356447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4173/8323] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:35:54 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:35:54.360704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.371344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241a194-83ed-48c7-b228-f7c22296d193 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.431897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.432528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.432856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.433131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.433785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.435800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.466582 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4151/8324] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:35:54 2022] POST /identity/v3/users => generated 280 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:35:54.468272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.480347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dabf1-94b5-42d9-a29c-8730b316f13c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.486538 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.487232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.487510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.487795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.488491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.490235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.504347 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4174/8325] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:35:54.505654 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.516706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe6516-c914-46d7-b7a6-3d2648eea84d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.522682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae08b1-a61f-4f69-ae19-be79cba16146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.523355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae08b1-a61f-4f69-ae19-be79cba16146 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.523636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae08b1-a61f-4f69-ae19-be79cba16146 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.523927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae08b1-a61f-4f69-ae19-be79cba16146 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.524569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae08b1-a61f-4f69-ae19-be79cba16146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.526230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae08b1-a61f-4f69-ae19-be79cba16146 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.537900 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4152/8326] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:35:54.542254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.552990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d267a3-861f-4e91-ada2-1e9179b2841b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.558005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.558673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.558954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.559438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] PATH_INFO: `/v3/projects/8ff9168912304a0baf394fe13f3dcb14/users/34a85525c845484280f7c50d88cb5719/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.574929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff9168912304a0baf394fe13f3dcb14, user_id=34a85525c845484280f7c50d88cb5719, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.578683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.593833 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4175/8327] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:35:54 2022] PUT /identity/v3/projects/8ff9168912304a0baf394fe13f3dcb14/users/34a85525c845484280f7c50d88cb5719/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:35:54.605230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2cfe-dbd4-4031-8da1-738b42afd62d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.608601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2cfe-dbd4-4031-8da1-738b42afd62d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.608601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2cfe-dbd4-4031-8da1-738b42afd62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.639779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-926c2cfe-dbd4-4031-8da1-738b42afd62d None None] MFA Rules not processed for user `34a85525c845484280f7c50d88cb5719`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:35:54.641042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-926c2cfe-dbd4-4031-8da1-738b42afd62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:54.710433 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4153/8328] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:35:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:35:54.721568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:35:54.723261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:35:54.733536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:54.799207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c7178-5680-49aa-a68e-3dfad0e5561f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:35:54.803517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:35:54.804132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:35:54.804406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:35:54.804692 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:35:54.814489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:35:54.815936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:35:54.825688 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4176/8329] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:35:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:36:04.706822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:04.821249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=23 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:25.415475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7fcfe-eafb-4fdd-9971-2fbc02d7da89 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:25.416277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7fcfe-eafb-4fdd-9971-2fbc02d7da89 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:25.416277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7fcfe-eafb-4fdd-9971-2fbc02d7da89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:25.462721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-83b7fcfe-eafb-4fdd-9971-2fbc02d7da89 None None] MFA Rules not processed for user `0bd8a19d0ab44011b5430f26757a0efe`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:25.464345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-83b7fcfe-eafb-4fdd-9971-2fbc02d7da89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:25.557501 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4154/8330] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:36:26.170235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:26.171943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:26.190739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8325c3-44b8-4613-a94f-719713e8a1ba service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:26.203467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:26.204467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:26.205077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:26.205492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:26.207549 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:26.224132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:26.226026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:26.240476 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4177/8331] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:36:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:36:35.543694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=25 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:36.232119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:38.833872 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7fcfe-eafb-4fdd-9971-2fbc02d7da89 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:38.861424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7fcfe-eafb-4fdd-9971-2fbc02d7da89 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:38.868259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357575b7-06cc-44df-997c-be085dd22079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:38.871462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357575b7-06cc-44df-997c-be085dd22079 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:38.871962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357575b7-06cc-44df-997c-be085dd22079 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:38.872396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357575b7-06cc-44df-997c-be085dd22079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:38.874550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-357575b7-06cc-44df-997c-be085dd22079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:39.030885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357575b7-06cc-44df-997c-be085dd22079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:39.032847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357575b7-06cc-44df-997c-be085dd22079 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:39.067301 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4155/8332] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:36:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:36:43.173192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:43.174691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:43.277453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b4746-1b0b-4cc3-9c6a-e07da245e9e7 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:43.282748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:43.283739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:43.287117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:43.288015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] PATH_INFO: `/v3/users/679234e2323d4bb2b50e5991e315bf85` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:43.291587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=679234e2323d4bb2b50e5991e315bf85)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:43.299944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:43.378560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '679234e2323d4bb2b50e5991e315bf85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:43.379484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '679234e2323d4bb2b50e5991e315bf85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:43.402878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '679234e2323d4bb2b50e5991e315bf85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:43.411308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '679234e2323d4bb2b50e5991e315bf85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:43.412007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '679234e2323d4bb2b50e5991e315bf85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:43.430913 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4178/8333] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:36:43 2022] DELETE /identity/v3/users/679234e2323d4bb2b50e5991e315bf85 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:43.435534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:43.436019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:43.436119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:43.456204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:43.457515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:43.534825 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4156/8334] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:36:43.545837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:43.547623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:43.589551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:43.594581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:43.639344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8453685-eb32-4b11-8835-3c43ce73fd38 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:43.643743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:43.644361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:43.645241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:43.645522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:43.656085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:43.657612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:43.666563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc8fd9-238d-4ed4-ad0b-66929d59d40d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:43.667989 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4179/8335] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:36:43.671097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b5e4f-9854-4885-9052-0a41ebaf097b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:43.671718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b5e4f-9854-4885-9052-0a41ebaf097b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:43.671996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b5e4f-9854-4885-9052-0a41ebaf097b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:43.672270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b5e4f-9854-4885-9052-0a41ebaf097b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:43.674099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e14b5e4f-9854-4885-9052-0a41ebaf097b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:43.698853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:43.761389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b5e4f-9854-4885-9052-0a41ebaf097b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:43.763113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b5e4f-9854-4885-9052-0a41ebaf097b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:43.778867 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4157/8336] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:36:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:36:43.918687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:43.920570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:43.944085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8a543-ed8a-4bab-8d73-df95305c279f service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:43.950343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:43.951157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:43.951537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:43.952015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] PATH_INFO: `/v3/projects/93d0ff16192d4f319da6ff81d6d57ae7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:43.953622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d0ff16192d4f319da6ff81d6d57ae7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:43.955668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:43.988546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:44.040586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d0ff16192d4f319da6ff81d6d57ae7', 'request_id': 'req-ed262361-84f3-4e6b-b57e-3262696ca509'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:44.050515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d0ff16192d4f319da6ff81d6d57ae7', 'request_id': 'req-ed262361-84f3-4e6b-b57e-3262696ca509'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:44.060837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d0ff16192d4f319da6ff81d6d57ae7', 'request_id': 'req-ed262361-84f3-4e6b-b57e-3262696ca509'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:44.087184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] The token cache is being invalidate because project 93d0ff16192d4f319da6ff81d6d57ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:36:44.087946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:44.088711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ed262361-84f3-4e6b-b57e-3262696ca509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:44.090759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4180/8337] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:36:43 2022] DELETE /identity/v3/projects/93d0ff16192d4f319da6ff81d6d57ae7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:44.095941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb5b9-fcc4-4144-8158-f07d71c78485 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.096210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb5b9-fcc4-4144-8158-f07d71c78485 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.096489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb5b9-fcc4-4144-8158-f07d71c78485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.124641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-de1eb5b9-fcc4-4144-8158-f07d71c78485 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:44.127078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-de1eb5b9-fcc4-4144-8158-f07d71c78485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:44.222150 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4158/8338] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:36:44.231864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabc21c-19c8-44dd-9eab-1fc361d4f074 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.232426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabc21c-19c8-44dd-9eab-1fc361d4f074 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.232587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabc21c-19c8-44dd-9eab-1fc361d4f074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.254551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7aabc21c-19c8-44dd-9eab-1fc361d4f074 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:44.256019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7aabc21c-19c8-44dd-9eab-1fc361d4f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:44.271226 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4181/8339] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:36:44.272387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eb5b9-fcc4-4144-8158-f07d71c78485 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:44.286056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eb5b9-fcc4-4144-8158-f07d71c78485 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:44.294865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:44.295702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.296164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.296578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.297524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:36:44.298131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:44.301450 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:44.316701 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4159/8340] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:36:44.323136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabc21c-19c8-44dd-9eab-1fc361d4f074 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:44.337144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabc21c-19c8-44dd-9eab-1fc361d4f074 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:44.343297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:44.344115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.344554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.344993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.345762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:44.348152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:44.385723 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4182/8341] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:36:44 2022] POST /identity/v3/projects => generated 359 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:36:44.386973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:44.398623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee1ba9-06cc-46f4-be07-db33c3c63efe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:44.467637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:44.468311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.468758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.469448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.470231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:44.473636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:44.501885 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4160/8342] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:36:44 2022] POST /identity/v3/users => generated 280 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:36:44.506262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:44.517664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118ab26-f068-4f8b-b679-e8a050702170 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:44.522445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:44.523482 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.523946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.524207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.524901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:44.527231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:44.539694 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4183/8343] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:36:44.544628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:44.556328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee888d89-2c42-406a-9b0f-defdb401977a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:44.561598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18679b6-e421-4d82-95ac-fd2ecbd1899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:44.562224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18679b6-e421-4d82-95ac-fd2ecbd1899b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.562655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18679b6-e421-4d82-95ac-fd2ecbd1899b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.562936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18679b6-e421-4d82-95ac-fd2ecbd1899b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.563630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18679b6-e421-4d82-95ac-fd2ecbd1899b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:44.565302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18679b6-e421-4d82-95ac-fd2ecbd1899b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:44.578456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4161/8344] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:36:44.583039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:44.600454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c450a72-522f-4234-a599-ab87a2c4b0f8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:44.608959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:44.610115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.610491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.610849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] PATH_INFO: `/v3/projects/774946268eb04a0dae648a3bb5b6dcfe/users/ec1d69c563b3444d9f192bb1e2415825/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.631708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774946268eb04a0dae648a3bb5b6dcfe, user_id=ec1d69c563b3444d9f192bb1e2415825, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:44.635041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:44.694005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4184/8345] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:36:44 2022] PUT /identity/v3/projects/774946268eb04a0dae648a3bb5b6dcfe/users/ec1d69c563b3444d9f192bb1e2415825/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:44.695712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fb027-aebe-4a4c-be4e-bfb40ff49bae None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.696385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fb027-aebe-4a4c-be4e-bfb40ff49bae None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.697754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fb027-aebe-4a4c-be4e-bfb40ff49bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:44.747068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a34fb027-aebe-4a4c-be4e-bfb40ff49bae None None] MFA Rules not processed for user `ec1d69c563b3444d9f192bb1e2415825`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:44.748810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a34fb027-aebe-4a4c-be4e-bfb40ff49bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:44.753566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:44.895405 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4162/8346] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:36:44.908133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:44.910776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:44.993581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8e985-8bbf-41d9-a412-38121fb9cc87 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:44.997591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4426239-fa8c-459a-8e8c-0fdb1f1c2898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:44.998159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4426239-fa8c-459a-8e8c-0fdb1f1c2898 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:44.998437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4426239-fa8c-459a-8e8c-0fdb1f1c2898 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:44.998873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4426239-fa8c-459a-8e8c-0fdb1f1c2898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:45.009851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4426239-fa8c-459a-8e8c-0fdb1f1c2898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:45.011791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4426239-fa8c-459a-8e8c-0fdb1f1c2898 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:45.022811 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4185/8347] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:36:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:36:54.891564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=25 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:55.018033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:56.905516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fb027-aebe-4a4c-be4e-bfb40ff49bae None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:57.064103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fb027-aebe-4a4c-be4e-bfb40ff49bae None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:57.071473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:57.072731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:57.073796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:57.074600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] PATH_INFO: `/v3/users/ec1d69c563b3444d9f192bb1e2415825` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:57.076333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1d69c563b3444d9f192bb1e2415825)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:57.078678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:57.155837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1d69c563b3444d9f192bb1e2415825'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.157221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1d69c563b3444d9f192bb1e2415825'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.169921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1d69c563b3444d9f192bb1e2415825'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.190211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1d69c563b3444d9f192bb1e2415825'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.216026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1d69c563b3444d9f192bb1e2415825'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.229160 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4163/8348] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:36:56 2022] DELETE /identity/v3/users/ec1d69c563b3444d9f192bb1e2415825 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:57.230835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:57.231346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:57.231449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:57.274872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:57.277299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:57.374160 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4186/8349] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:36:57.384315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:57.385891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:57.488210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124345d-4620-480d-8c42-e58cdf986989 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:57.492494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdd647-29cb-4fd9-b258-f52eade8dc86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:57.493304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdd647-29cb-4fd9-b258-f52eade8dc86 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:57.493675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdd647-29cb-4fd9-b258-f52eade8dc86 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:57.494045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdd647-29cb-4fd9-b258-f52eade8dc86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:57.506400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdd647-29cb-4fd9-b258-f52eade8dc86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:57.508162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdd647-29cb-4fd9-b258-f52eade8dc86 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:57.545550 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4164/8350] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:36:57.782853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:57.789779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:57.807147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481106d-5e20-4c67-bba9-259a2850f37d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:57.815449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:57.815449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:57.815449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:57.815449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] PATH_INFO: `/v3/projects/774946268eb04a0dae648a3bb5b6dcfe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:57.820278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774946268eb04a0dae648a3bb5b6dcfe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:57.822654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:57.913535 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774946268eb04a0dae648a3bb5b6dcfe', 'request_id': 'req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.930575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774946268eb04a0dae648a3bb5b6dcfe', 'request_id': 'req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.937420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774946268eb04a0dae648a3bb5b6dcfe', 'request_id': 'req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.952161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] The token cache is being invalidate because project 774946268eb04a0dae648a3bb5b6dcfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:36:57.953006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.954208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ac29524c-30d6-48a1-b21d-8a0cdf1eca66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:36:57.956108 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4187/8351] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:36:57 2022] DELETE /identity/v3/projects/774946268eb04a0dae648a3bb5b6dcfe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:57.961753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa070bd-6bc5-4d47-86b3-d3a902b42d0d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:57.962226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa070bd-6bc5-4d47-86b3-d3a902b42d0d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:57.962305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa070bd-6bc5-4d47-86b3-d3a902b42d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:57.996821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9aa070bd-6bc5-4d47-86b3-d3a902b42d0d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:57.998311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9aa070bd-6bc5-4d47-86b3-d3a902b42d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:58.031391 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:58.097256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4165/8352] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:36:58.106943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9f72d-30ad-4b9c-ad47-9f5957885590 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.107238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9f72d-30ad-4b9c-ad47-9f5957885590 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.107517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9f72d-30ad-4b9c-ad47-9f5957885590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.155955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-88f9f72d-30ad-4b9c-ad47-9f5957885590 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:58.157259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-88f9f72d-30ad-4b9c-ad47-9f5957885590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:58.186732 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4188/8353] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:36:58.187854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa070bd-6bc5-4d47-86b3-d3a902b42d0d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.202413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa070bd-6bc5-4d47-86b3-d3a902b42d0d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.216204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:58.217855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.218835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.219451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.221876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:36:58.224327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:58.228104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:58.251436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9f72d-30ad-4b9c-ad47-9f5957885590 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.258480 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4166/8354] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:36:58.262475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9f72d-30ad-4b9c-ad47-9f5957885590 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.269068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:58.269852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.270278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.270622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.271680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:58.275505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:58.319605 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4189/8355] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:36:58 2022] POST /identity/v3/projects => generated 405 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:36:58.321841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.335892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66a2a3-f02b-424f-ab5e-27edd1d3d0d4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.423365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:58.423958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.424626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.424813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.426735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:58.428914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:58.458993 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4167/8356] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:36:58 2022] POST /identity/v3/users => generated 303 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:36:58.463175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.480193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92690-864a-41e0-8149-2eacf386acbd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.485585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:58.486287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.486489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.486758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.487407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:58.489120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:58.503525 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4190/8357] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:36:58.505016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.516719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08f1b1-7823-4f98-a187-a36ecfd13c96 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.522632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c05afc-bf8c-4dab-973b-304413f41ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:58.523281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c05afc-bf8c-4dab-973b-304413f41ccd admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.523610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c05afc-bf8c-4dab-973b-304413f41ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.523889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c05afc-bf8c-4dab-973b-304413f41ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.524540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c05afc-bf8c-4dab-973b-304413f41ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:58.526190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c05afc-bf8c-4dab-973b-304413f41ccd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:58.544622 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4168/8358] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:36:58.545522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.558256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ca1fe-c965-4922-b718-4e4582a0a609 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.564355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:58.565073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.565354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.565631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] PATH_INFO: `/v3/projects/f802a7b1b07e40c1bb34a5f382a05605/users/2f44d871ab1849a7989d91233f4fc16f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.584587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f802a7b1b07e40c1bb34a5f382a05605, user_id=2f44d871ab1849a7989d91233f4fc16f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:58.587573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:58.622972 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4191/8359] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:36:58 2022] PUT /identity/v3/projects/f802a7b1b07e40c1bb34a5f382a05605/users/2f44d871ab1849a7989d91233f4fc16f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:58.628911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2672e-92ff-4cfc-8e92-8b5cda4ceecc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.629240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2672e-92ff-4cfc-8e92-8b5cda4ceecc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.631615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2672e-92ff-4cfc-8e92-8b5cda4ceecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.675320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-adb2672e-92ff-4cfc-8e92-8b5cda4ceecc None None] MFA Rules not processed for user `2f44d871ab1849a7989d91233f4fc16f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:58.676629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-adb2672e-92ff-4cfc-8e92-8b5cda4ceecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:58.783606 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4169/8360] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:36:58.788873 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.813587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:58.867737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946152b-2f6e-4bda-a0ee-91c068ecf59a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.873019 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:58.873892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:58.874230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:58.874570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:58.875423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:58.877521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:58.933376 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4192/8361] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:36:58 2022] POST /identity/v3/projects => generated 405 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:36:58.937218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2672e-92ff-4cfc-8e92-8b5cda4ceecc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:58.952608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2672e-92ff-4cfc-8e92-8b5cda4ceecc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:58.976080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:59.044587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:59.045315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.045622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.045929 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.046547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:59.048555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:59.086263 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4170/8362] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:36:58 2022] POST /identity/v3/users => generated 309 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:36:59.090674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:59.103199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d96c6-b30b-4c93-8ac2-43615c288e02 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:59.108878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:59.109734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.110020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.110295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.110960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:59.112701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:59.128912 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4193/8363] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:36:59.130241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:59.142908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e5856-d589-4832-91e8-d1135773803f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:59.149146 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:59.150380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.150819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.151431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] PATH_INFO: `/v3/projects/48f587d24d3541d3b02c35219a7f11f4/users/fa33ec926f824162887ca0fd78298aaf/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.173867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f587d24d3541d3b02c35219a7f11f4, user_id=fa33ec926f824162887ca0fd78298aaf, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:59.177555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:59.196710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4171/8364] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:36:59 2022] PUT /identity/v3/projects/48f587d24d3541d3b02c35219a7f11f4/users/fa33ec926f824162887ca0fd78298aaf/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:59.201133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:59.217583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651dbfe-1a45-4e5c-a658-812f303a8b0a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:59.324069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a77be1-fa5a-4a62-83bc-558e23755765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:59.324836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a77be1-fa5a-4a62-83bc-558e23755765 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.325153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a77be1-fa5a-4a62-83bc-558e23755765 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.325428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a77be1-fa5a-4a62-83bc-558e23755765 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.326077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a77be1-fa5a-4a62-83bc-558e23755765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:59.327757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a77be1-fa5a-4a62-83bc-558e23755765 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:59.342007 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4194/8365] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:36:59.345629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:59.360410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6302403-896c-4270-87dc-a878cfef8abe admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:59.370793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:59.371894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.372298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.373314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] PATH_INFO: `/v3/projects/48f587d24d3541d3b02c35219a7f11f4/users/fa33ec926f824162887ca0fd78298aaf/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.375648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f587d24d3541d3b02c35219a7f11f4, user_id=fa33ec926f824162887ca0fd78298aaf, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:59.380989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:59.407931 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4172/8366] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:36:59 2022] PUT /identity/v3/projects/48f587d24d3541d3b02c35219a7f11f4/users/fa33ec926f824162887ca0fd78298aaf/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:36:59.410158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c17fd2-c34f-4638-9726-b60e5d589459 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.411324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c17fd2-c34f-4638-9726-b60e5d589459 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.411760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c17fd2-c34f-4638-9726-b60e5d589459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.455111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b2c17fd2-c34f-4638-9726-b60e5d589459 None None] MFA Rules not processed for user `fa33ec926f824162887ca0fd78298aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:36:59.457590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b2c17fd2-c34f-4638-9726-b60e5d589459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:59.562268 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4195/8367] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:36:59.580105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:59.582519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:36:59.681476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f39244-0d1d-4658-bf1b-bf91df52677e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:59.685608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340432-6deb-44a0-8c5c-198992fbdb0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:59.686688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340432-6deb-44a0-8c5c-198992fbdb0d service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.686688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340432-6deb-44a0-8c5c-198992fbdb0d service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.686925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340432-6deb-44a0-8c5c-198992fbdb0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.698121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340432-6deb-44a0-8c5c-198992fbdb0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:59.700757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340432-6deb-44a0-8c5c-198992fbdb0d service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:36:59.711770 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4173/8368] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:36:59 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:36:59.874855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c17fd2-c34f-4638-9726-b60e5d589459 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:36:59.895211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:59.949599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c17fd2-c34f-4638-9726-b60e5d589459 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:36:59.954297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:36:59.954860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:36:59.955300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:36:59.955425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] PATH_INFO: `/v3/users/2f44d871ab1849a7989d91233f4fc16f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:36:59.956716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f44d871ab1849a7989d91233f4fc16f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:36:59.958960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:00.042525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f44d871ab1849a7989d91233f4fc16f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.043628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f44d871ab1849a7989d91233f4fc16f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.054727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f44d871ab1849a7989d91233f4fc16f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.068595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f44d871ab1849a7989d91233f4fc16f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.069720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f44d871ab1849a7989d91233f4fc16f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.100932 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4196/8369] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:36:59 2022] DELETE /identity/v3/users/2f44d871ab1849a7989d91233f4fc16f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:00.103928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:00.104446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:00.104705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:00.131511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:00.133127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:00.203831 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4174/8370] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:00.214261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:00.216494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:00.298534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f500e93-a745-4c66-bc8b-092c5022a7ca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:00.303488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:00.304216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:00.304488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:00.304834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:00.318796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:00.321077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:00.351611 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4197/8371] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:00.686547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:00.694707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:00.713380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:00.717433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1103ab-4319-498b-93b7-1d65a757a23c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:00.727029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:00.727980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:00.728417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:00.729735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] PATH_INFO: `/v3/projects/f802a7b1b07e40c1bb34a5f382a05605` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:00.732687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f802a7b1b07e40c1bb34a5f382a05605)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:00.735067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:00.804763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f802a7b1b07e40c1bb34a5f382a05605', 'request_id': 'req-44becda1-ce1e-41af-b38e-37bdae225dfb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.814158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f802a7b1b07e40c1bb34a5f382a05605', 'request_id': 'req-44becda1-ce1e-41af-b38e-37bdae225dfb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.825758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f802a7b1b07e40c1bb34a5f382a05605', 'request_id': 'req-44becda1-ce1e-41af-b38e-37bdae225dfb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.839250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] The token cache is being invalidate because project f802a7b1b07e40c1bb34a5f382a05605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:00.839703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.840507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:00.847342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4175/8372] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:00 2022] DELETE /identity/v3/projects/f802a7b1b07e40c1bb34a5f382a05605 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:00.848632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:00.860519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104aba-57ae-4395-a62d-ef665b0b4928 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:00.906992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:00.941192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:00.943572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:00.944063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:00.944640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] PATH_INFO: `/v3/users/fa33ec926f824162887ca0fd78298aaf` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:00.946883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa33ec926f824162887ca0fd78298aaf)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:00.949125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:01.012700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa33ec926f824162887ca0fd78298aaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.013649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa33ec926f824162887ca0fd78298aaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.040980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa33ec926f824162887ca0fd78298aaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.059192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa33ec926f824162887ca0fd78298aaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.060013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ba7f982c-9760-4512-b9f3-1f9d5ab3d753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa33ec926f824162887ca0fd78298aaf'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.080348 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4198/8373] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:00 2022] DELETE /identity/v3/users/fa33ec926f824162887ca0fd78298aaf => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:01.589994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:01.593150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:01.622289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44becda1-ce1e-41af-b38e-37bdae225dfb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:01.720277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:01.726990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:01.727505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:01.727995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] PATH_INFO: `/v3/projects/48f587d24d3541d3b02c35219a7f11f4` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:01.734049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=48f587d24d3541d3b02c35219a7f11f4)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:01.736533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:01.838821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48f587d24d3541d3b02c35219a7f11f4', 'request_id': 'req-a1201904-acee-4c6e-9f48-909f7eea70cb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.845446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48f587d24d3541d3b02c35219a7f11f4', 'request_id': 'req-a1201904-acee-4c6e-9f48-909f7eea70cb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.855013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48f587d24d3541d3b02c35219a7f11f4', 'request_id': 'req-a1201904-acee-4c6e-9f48-909f7eea70cb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.874791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] The token cache is being invalidate because project 48f587d24d3541d3b02c35219a7f11f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:01.875019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.875832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-a1201904-acee-4c6e-9f48-909f7eea70cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:01.877749 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4176/8374] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:01 2022] DELETE /identity/v3/projects/48f587d24d3541d3b02c35219a7f11f4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:01.882591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c9f3-fb55-4412-87dc-f4a9c1fd7dbb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:01.883597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c9f3-fb55-4412-87dc-f4a9c1fd7dbb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:01.884111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81c9f3-fb55-4412-87dc-f4a9c1fd7dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:01.909422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1c81c9f3-fb55-4412-87dc-f4a9c1fd7dbb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:01.911047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1c81c9f3-fb55-4412-87dc-f4a9c1fd7dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:01.993346 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4199/8375] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:02.000893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7840fd-a097-4fde-a274-b2f18fa6612f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.001204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7840fd-a097-4fde-a274-b2f18fa6612f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.001856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7840fd-a097-4fde-a274-b2f18fa6612f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.025187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5e7840fd-a097-4fde-a274-b2f18fa6612f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:02.026485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e7840fd-a097-4fde-a274-b2f18fa6612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:02.040539 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4177/8376] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:02.045430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81c9f3-fb55-4412-87dc-f4a9c1fd7dbb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:02.056796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81c9f3-fb55-4412-87dc-f4a9c1fd7dbb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:02.068435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:02.069119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.069395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.069664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.070358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:02.071007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:02.072651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:02.087358 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4200/8377] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:02.091884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7840fd-a097-4fde-a274-b2f18fa6612f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:02.102503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7840fd-a097-4fde-a274-b2f18fa6612f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:02.107506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:02.108142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.108430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.108722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.109367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:02.111331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:02.121890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:02.154063 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4178/8378] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:02 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:02.155748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:02.177425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c005bd-0da4-433e-a9e3-d083aa3a41d2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:02.284068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:02.307626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:02.313212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.313938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.314341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.315746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:02.325406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:02.360479 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4201/8379] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:02 2022] POST /identity/v3/users => generated 302 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:02.361508 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:02.387241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f806858-d2c5-4b28-97e9-d154f19d28ba admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:02.404157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:02.405100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.405586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.405931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.406930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:02.408707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:02.423861 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4179/8380] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:02.428156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:02.441095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6df96b-6c1c-4c5e-b44f-446a74ca9a9b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:02.455532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d006-625c-4f72-b025-b9a3ac44a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:02.457090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d006-625c-4f72-b025-b9a3ac44a92c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.460161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d006-625c-4f72-b025-b9a3ac44a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.460747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d006-625c-4f72-b025-b9a3ac44a92c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.461586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d006-625c-4f72-b025-b9a3ac44a92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:02.463453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d006-625c-4f72-b025-b9a3ac44a92c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:02.488247 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4202/8381] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:02.493094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:02.507324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d768d-cf8d-484f-ac8d-af382544d462 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:02.512429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:02.513495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.513870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.514234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] PATH_INFO: `/v3/projects/6c2a96b091834528865ee27772b15a98/users/1bb5556e35bb4ab197d675f5663fd46f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.545849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2a96b091834528865ee27772b15a98, user_id=1bb5556e35bb4ab197d675f5663fd46f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:02.549658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:02.572869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4180/8382] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:02 2022] PUT /identity/v3/projects/6c2a96b091834528865ee27772b15a98/users/1bb5556e35bb4ab197d675f5663fd46f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:02.578417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743678f1-a700-4d93-a72d-8372639c0bb4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.578713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743678f1-a700-4d93-a72d-8372639c0bb4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.578991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743678f1-a700-4d93-a72d-8372639c0bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.624907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-743678f1-a700-4d93-a72d-8372639c0bb4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:02.626376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-743678f1-a700-4d93-a72d-8372639c0bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:02.724588 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4203/8383] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:02.735074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:02.737158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:02.843767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e3303-0078-4a74-875b-4faf3ffe1811 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:02.848200 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:02.849184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:02.849764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:02.850342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:02.873387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:02.876844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:02.912583 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4181/8384] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:03.768078 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782312af-b42d-4458-889d-c6b918965008 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:03.768565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782312af-b42d-4458-889d-c6b918965008 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:03.768727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782312af-b42d-4458-889d-c6b918965008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:03.808146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-782312af-b42d-4458-889d-c6b918965008 None None] MFA Rules not processed for user `1bb5556e35bb4ab197d675f5663fd46f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:03.810585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-782312af-b42d-4458-889d-c6b918965008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:03.917497 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4204/8385] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:03.918723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:03.930220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656a60-6fd5-463e-8597-0e79ce7c3048 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:03.938187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:03.940219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:03.940648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:03.941118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:03.942086 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:03.945205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:03.993647 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4182/8386] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:03 2022] POST /identity/v3/projects => generated 403 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:03.997789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782312af-b42d-4458-889d-c6b918965008 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:04.011970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782312af-b42d-4458-889d-c6b918965008 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:04.079092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:04.094063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:04.095263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:04.096064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:04.096064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:04.096821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:04.099539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:04.126699 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4205/8387] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:03 2022] POST /identity/v3/users => generated 308 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:04.130675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:04.141367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b876d6-b09b-4be3-8835-f36b769ba656 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:04.146179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:04.146807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:04.147139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:04.147461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:04.148108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:04.149875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:04.164833 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4183/8388] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:04.166456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:04.178845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b08a-bd47-40c4-b62a-4903ca889c15 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:04.184168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:04.187425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:04.188150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:04.188150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] PATH_INFO: `/v3/projects/9b1c5d0b529948b39bfe22fce0e92810/users/4c605ceff6e24ff0b89bd7fdf1f6447d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:04.207192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1c5d0b529948b39bfe22fce0e92810, user_id=4c605ceff6e24ff0b89bd7fdf1f6447d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:04.211135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:04.226910 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4206/8389] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:04 2022] PUT /identity/v3/projects/9b1c5d0b529948b39bfe22fce0e92810/users/4c605ceff6e24ff0b89bd7fdf1f6447d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:04.231003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:04.241393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4518f-08ab-4b83-96b8-0f8df9fa56b5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:04.322371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b58d29-34fe-4e1e-abc8-0cb8ad01afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:04.324998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b58d29-34fe-4e1e-abc8-0cb8ad01afd5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:04.325861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b58d29-34fe-4e1e-abc8-0cb8ad01afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:04.326550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b58d29-34fe-4e1e-abc8-0cb8ad01afd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:04.327831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b58d29-34fe-4e1e-abc8-0cb8ad01afd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:04.329076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b58d29-34fe-4e1e-abc8-0cb8ad01afd5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:04.341066 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4184/8390] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:04.345081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:04.356843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373be32-daf5-439f-9b86-93916f650bb5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:04.362462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:04.363137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:04.363411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:04.363699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] PATH_INFO: `/v3/projects/9b1c5d0b529948b39bfe22fce0e92810/users/4c605ceff6e24ff0b89bd7fdf1f6447d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:04.367942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1c5d0b529948b39bfe22fce0e92810, user_id=4c605ceff6e24ff0b89bd7fdf1f6447d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:04.370825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:04.386375 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4207/8391] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:04 2022] PUT /identity/v3/projects/9b1c5d0b529948b39bfe22fce0e92810/users/4c605ceff6e24ff0b89bd7fdf1f6447d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:05.234248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5e7b-d799-49c4-a0e3-ffd2bf4a5881 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:05.234741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5e7b-d799-49c4-a0e3-ffd2bf4a5881 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:05.234846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5e7b-d799-49c4-a0e3-ffd2bf4a5881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:05.301040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-66dc5e7b-d799-49c4-a0e3-ffd2bf4a5881 None None] MFA Rules not processed for user `4c605ceff6e24ff0b89bd7fdf1f6447d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:05.302839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-66dc5e7b-d799-49c4-a0e3-ffd2bf4a5881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:05.315263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:05.456841 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4185/8392] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:05 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:06.976417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:07.079738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a553b-9a12-419f-abd0-05d0a0afda27 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:07.087772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:07.088980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:07.089639 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:07.090116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] PATH_INFO: `/v3/users/1bb5556e35bb4ab197d675f5663fd46f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:07.091681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb5556e35bb4ab197d675f5663fd46f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:07.095279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:07.160706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb5556e35bb4ab197d675f5663fd46f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.161698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb5556e35bb4ab197d675f5663fd46f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.170623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb5556e35bb4ab197d675f5663fd46f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.177942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb5556e35bb4ab197d675f5663fd46f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.178598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb5556e35bb4ab197d675f5663fd46f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.197942 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4208/8393] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:06 2022] DELETE /identity/v3/users/1bb5556e35bb4ab197d675f5663fd46f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:07.600363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc5e7b-d799-49c4-a0e3-ffd2bf4a5881 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:07.613148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc5e7b-d799-49c4-a0e3-ffd2bf4a5881 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:07.687555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:07.688100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:07.688377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:07.688682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] PATH_INFO: `/v3/projects/6c2a96b091834528865ee27772b15a98` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:07.689879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2a96b091834528865ee27772b15a98)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:07.691791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:07.742898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2a96b091834528865ee27772b15a98', 'request_id': 'req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.749288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2a96b091834528865ee27772b15a98', 'request_id': 'req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.758250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2a96b091834528865ee27772b15a98', 'request_id': 'req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.769876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] The token cache is being invalidate because project 6c2a96b091834528865ee27772b15a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:07.770193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.771045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.772683 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4186/8394] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:07 2022] DELETE /identity/v3/projects/6c2a96b091834528865ee27772b15a98 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:07.777426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:07.779133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:07.792773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a0741-73f6-4ae9-9c2f-721d7f9845c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:07.814798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:07.857134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:07.857837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:07.858113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:07.858381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] PATH_INFO: `/v3/users/4c605ceff6e24ff0b89bd7fdf1f6447d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:07.859506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c605ceff6e24ff0b89bd7fdf1f6447d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:07.861571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:07.912146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c605ceff6e24ff0b89bd7fdf1f6447d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.912843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c605ceff6e24ff0b89bd7fdf1f6447d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.923432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c605ceff6e24ff0b89bd7fdf1f6447d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.932218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c605ceff6e24ff0b89bd7fdf1f6447d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.933039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f118eb10-2866-47b4-a5f7-d5a61dfa3133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c605ceff6e24ff0b89bd7fdf1f6447d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:07.953580 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4209/8395] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:07 2022] DELETE /identity/v3/users/4c605ceff6e24ff0b89bd7fdf1f6447d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:08.472046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:08.474827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:08.497361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83272f0a-faaa-4ce1-b727-2dbf02dd9d50 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:08.539386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:08.569176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:08.569836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:08.570460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:08.570827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] PATH_INFO: `/v3/projects/9b1c5d0b529948b39bfe22fce0e92810` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:08.571867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1c5d0b529948b39bfe22fce0e92810)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:08.574010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:08.640027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1c5d0b529948b39bfe22fce0e92810', 'request_id': 'req-97200cba-1fe7-4169-b95f-89d438e97b94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:08.647358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1c5d0b529948b39bfe22fce0e92810', 'request_id': 'req-97200cba-1fe7-4169-b95f-89d438e97b94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:08.660001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1c5d0b529948b39bfe22fce0e92810', 'request_id': 'req-97200cba-1fe7-4169-b95f-89d438e97b94'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:08.678859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] The token cache is being invalidate because project 9b1c5d0b529948b39bfe22fce0e92810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:08.679161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:08.680245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-97200cba-1fe7-4169-b95f-89d438e97b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:08.681737 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4187/8396] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:08 2022] DELETE /identity/v3/projects/9b1c5d0b529948b39bfe22fce0e92810 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:08.690624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eed631-8c9a-4be2-9e2d-900c231db24e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:08.691103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eed631-8c9a-4be2-9e2d-900c231db24e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:08.691181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eed631-8c9a-4be2-9e2d-900c231db24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:08.727640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c5eed631-8c9a-4be2-9e2d-900c231db24e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:08.729274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c5eed631-8c9a-4be2-9e2d-900c231db24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:08.825541 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4210/8397] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:08.827881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f27e6-406d-4aa7-83b6-4eaed1601ab3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:08.827881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f27e6-406d-4aa7-83b6-4eaed1601ab3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:08.827881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f27e6-406d-4aa7-83b6-4eaed1601ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:08.858711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e56f27e6-406d-4aa7-83b6-4eaed1601ab3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:08.860110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e56f27e6-406d-4aa7-83b6-4eaed1601ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:08.877995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eed631-8c9a-4be2-9e2d-900c231db24e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:08.878976 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4188/8398] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:08.890834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eed631-8c9a-4be2-9e2d-900c231db24e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:08.898767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:08.900387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:08.900387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:08.900604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:08.901542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:08.902305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:08.905792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:08.930494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f27e6-406d-4aa7-83b6-4eaed1601ab3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:08.937220 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4211/8399] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:08.941365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f27e6-406d-4aa7-83b6-4eaed1601ab3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:08.947357 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:08.948049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:08.948400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:08.948775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:08.949626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:08.951577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:08.990694 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4189/8400] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:08 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:08.995061 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.009983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81a507-02bc-48ef-84da-09674af5a244 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.066487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:09.137247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.139063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.139966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.140352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.141343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.143711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.175782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4212/8401] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:08 2022] POST /identity/v3/users => generated 282 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:09.180233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.195311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f828211-8405-4b78-bc94-5dcf1b56ee07 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.205209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.205851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.206135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.206410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.209832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.213059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.236932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.237280 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4190/8402] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:09.252893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15b2cc-a747-4819-a010-e2c8c5b428b5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.261302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8eeb3a-ca8f-4435-b7aa-93668935bfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.262161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8eeb3a-ca8f-4435-b7aa-93668935bfb5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.262516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8eeb3a-ca8f-4435-b7aa-93668935bfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.262868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8eeb3a-ca8f-4435-b7aa-93668935bfb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.263598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8eeb3a-ca8f-4435-b7aa-93668935bfb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.265340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8eeb3a-ca8f-4435-b7aa-93668935bfb5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.279943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4213/8403] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:09.283898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.300211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9b7f5-9d35-424c-a163-61acfe41f476 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.308158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.311626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.312561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.313691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] PATH_INFO: `/v3/projects/bd101b0537ef4b4188a021f66d1082fe/users/2770aa729e964bd5a84406b8cb5006ae/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.336874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd101b0537ef4b4188a021f66d1082fe, user_id=2770aa729e964bd5a84406b8cb5006ae, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.344263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.373354 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4191/8404] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:09 2022] PUT /identity/v3/projects/bd101b0537ef4b4188a021f66d1082fe/users/2770aa729e964bd5a84406b8cb5006ae/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:09.375235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c883e7-60eb-42a2-953c-9bc525afb5d4 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.375525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c883e7-60eb-42a2-953c-9bc525afb5d4 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.375802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c883e7-60eb-42a2-953c-9bc525afb5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.409532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-05c883e7-60eb-42a2-953c-9bc525afb5d4 None None] MFA Rules not processed for user `2770aa729e964bd5a84406b8cb5006ae`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:09.410739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-05c883e7-60eb-42a2-953c-9bc525afb5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:09.507979 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4214/8405] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:09.509769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.593523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3be7-6a22-429a-852c-2023942f5360 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.602283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.602932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.603221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.603497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.604159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.606163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.612540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:09.645327 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4192/8406] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:09 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:09.650159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c883e7-60eb-42a2-953c-9bc525afb5d4 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.662320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c883e7-60eb-42a2-953c-9bc525afb5d4 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.725134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.725894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.726274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.726601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.727378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.729419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.753922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4215/8407] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:09 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:09.758418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.770237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4879c6-7a6c-4cb3-8433-dbb0b360a1dd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.775262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.776009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.776288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.776578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.777303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.778996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.790431 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4193/8408] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:09.794756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.806920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb28a2-9dfc-4dd0-9960-958a6a37f28d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.812540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.813262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.813539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.813813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] PATH_INFO: `/v3/projects/3a42ddd29392456b8683385b8a0e392a/users/44e2fe949f394467860ae123b2a267f5/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.831149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a42ddd29392456b8683385b8a0e392a, user_id=44e2fe949f394467860ae123b2a267f5, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.834197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.854068 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4216/8409] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:09 2022] PUT /identity/v3/projects/3a42ddd29392456b8683385b8a0e392a/users/44e2fe949f394467860ae123b2a267f5/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:09.855183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.866406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236c45-c8ab-4d88-a329-601be4fce4ef admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.934634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f34f7-3290-42a0-a247-0593b00db9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.935292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f34f7-3290-42a0-a247-0593b00db9ce admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.935576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f34f7-3290-42a0-a247-0593b00db9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.935848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f34f7-3290-42a0-a247-0593b00db9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.936565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f34f7-3290-42a0-a247-0593b00db9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.938246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f34f7-3290-42a0-a247-0593b00db9ce admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:09.950908 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4194/8410] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:09.955528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:09.964876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:09.981070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d88be-88e4-4147-ac6d-974cdf139a4e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:09.988261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:09.989256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:09.989814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:09.989814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] PATH_INFO: `/v3/projects/3a42ddd29392456b8683385b8a0e392a/users/44e2fe949f394467860ae123b2a267f5/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:09.991922 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a42ddd29392456b8683385b8a0e392a, user_id=44e2fe949f394467860ae123b2a267f5, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:09.995005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:10.011465 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4217/8411] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:09 2022] PUT /identity/v3/projects/3a42ddd29392456b8683385b8a0e392a/users/44e2fe949f394467860ae123b2a267f5/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:10.023021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bc7bf-6367-463a-bbb7-e827b7aa1035 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:10.023402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bc7bf-6367-463a-bbb7-e827b7aa1035 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:10.023687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bc7bf-6367-463a-bbb7-e827b7aa1035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:10.069063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c85bc7bf-6367-463a-bbb7-e827b7aa1035 None None] MFA Rules not processed for user `44e2fe949f394467860ae123b2a267f5`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:10.071160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c85bc7bf-6367-463a-bbb7-e827b7aa1035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:10.154182 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4195/8412] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:10.167367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:10.168931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:10.235992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c082-136b-4928-8d29-3ff7cf15141b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:10.239854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035e6e7-e214-45e7-84fa-74754aa3fa54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:10.240454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035e6e7-e214-45e7-84fa-74754aa3fa54 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:10.240799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035e6e7-e214-45e7-84fa-74754aa3fa54 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:10.241042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035e6e7-e214-45e7-84fa-74754aa3fa54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:10.251010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035e6e7-e214-45e7-84fa-74754aa3fa54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:10.252453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035e6e7-e214-45e7-84fa-74754aa3fa54 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:10.266246 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4218/8413] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:10.752238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bc7bf-6367-463a-bbb7-e827b7aa1035 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:10.814994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:10.819745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bc7bf-6367-463a-bbb7-e827b7aa1035 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:10.826394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:10.827062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:10.827342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:10.827625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] PATH_INFO: `/v3/users/2770aa729e964bd5a84406b8cb5006ae` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:10.828847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2770aa729e964bd5a84406b8cb5006ae)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:10.830827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:10.886454 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2770aa729e964bd5a84406b8cb5006ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:10.887115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2770aa729e964bd5a84406b8cb5006ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:10.895142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2770aa729e964bd5a84406b8cb5006ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:10.905078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2770aa729e964bd5a84406b8cb5006ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:10.919000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2770aa729e964bd5a84406b8cb5006ae'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:10.925912 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4196/8414] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:10 2022] DELETE /identity/v3/users/2770aa729e964bd5a84406b8cb5006ae => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:10.930832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:10.931349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:10.931547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:10.954158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:10.955448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:11.017700 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4219/8415] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:11.033809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:11.035404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:11.102506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626247b-27d8-4d00-955e-5b81292aab66 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:11.106189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:11.106907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:11.107270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:11.107611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:11.117706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:11.119234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:11.129733 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4197/8416] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:11.420474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:11.422545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:11.442450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37e61d-bce2-4eb3-8093-5178ae57e9ff None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:11.448984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:11.449657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:11.449943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:11.450235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] PATH_INFO: `/v3/projects/bd101b0537ef4b4188a021f66d1082fe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:11.451392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd101b0537ef4b4188a021f66d1082fe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:11.453434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:11.517406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd101b0537ef4b4188a021f66d1082fe', 'request_id': 'req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.527561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd101b0537ef4b4188a021f66d1082fe', 'request_id': 'req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.533762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd101b0537ef4b4188a021f66d1082fe', 'request_id': 'req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.545672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] The token cache is being invalidate because project bd101b0537ef4b4188a021f66d1082fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:11.545963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.546809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.548453 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4220/8417] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:11 2022] DELETE /identity/v3/projects/bd101b0537ef4b4188a021f66d1082fe => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:11.552474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:11.562470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2ecf-0641-41be-a75f-a3c130e5c2c1 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:11.623643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:11.624284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:11.624557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:11.624864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] PATH_INFO: `/v3/users/44e2fe949f394467860ae123b2a267f5` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:11.625979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e2fe949f394467860ae123b2a267f5)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:11.627895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:11.650418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:11.683499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e2fe949f394467860ae123b2a267f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.684164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e2fe949f394467860ae123b2a267f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.691263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e2fe949f394467860ae123b2a267f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.700132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e2fe949f394467860ae123b2a267f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.712603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e69b0d18-bea0-4543-b905-ad1f79058b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e2fe949f394467860ae123b2a267f5'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:11.718611 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4198/8418] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:11 2022] DELETE /identity/v3/users/44e2fe949f394467860ae123b2a267f5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:12.136177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:12.137820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:12.159827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb593b-55d4-4d33-a0b2-b75ea04179d0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:12.185291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:12.241893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:12.242544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.252712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.253123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] PATH_INFO: `/v3/projects/3a42ddd29392456b8683385b8a0e392a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.260991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a42ddd29392456b8683385b8a0e392a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:12.263299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:12.370481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a42ddd29392456b8683385b8a0e392a', 'request_id': 'req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:12.380813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a42ddd29392456b8683385b8a0e392a', 'request_id': 'req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:12.387931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a42ddd29392456b8683385b8a0e392a', 'request_id': 'req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:12.403249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] The token cache is being invalidate because project 3a42ddd29392456b8683385b8a0e392a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:12.403674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:12.404435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d3bdd733-c96a-4da6-ba07-55a3f74a0c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:12.406437 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4221/8419] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:12 2022] DELETE /identity/v3/projects/3a42ddd29392456b8683385b8a0e392a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:12.412154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5c371-c417-4ebb-a631-0fef98b5e126 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.413134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5c371-c417-4ebb-a631-0fef98b5e126 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.413417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5c371-c417-4ebb-a631-0fef98b5e126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.434015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-33f5c371-c417-4ebb-a631-0fef98b5e126 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:12.435271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-33f5c371-c417-4ebb-a631-0fef98b5e126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:12.512337 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4199/8420] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:12.514393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf7b6a-b99f-4d47-b723-3d58c2936452 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.514785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf7b6a-b99f-4d47-b723-3d58c2936452 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.515156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf7b6a-b99f-4d47-b723-3d58c2936452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.538612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c8bf7b6a-b99f-4d47-b723-3d58c2936452 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:12.540720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c8bf7b6a-b99f-4d47-b723-3d58c2936452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:12.555141 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4222/8421] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:12.558640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5c371-c417-4ebb-a631-0fef98b5e126 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:12.571438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5c371-c417-4ebb-a631-0fef98b5e126 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:12.576968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:12.577705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.578083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.578455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.579209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:12.579608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:12.581496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:12.600813 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4200/8422] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:12.602797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf7b6a-b99f-4d47-b723-3d58c2936452 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:12.616909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf7b6a-b99f-4d47-b723-3d58c2936452 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:12.622600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:12.623333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.623823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.624504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.625092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:12.627403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:12.663767 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4223/8423] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:12 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:12.668267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:12.679916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e95e0-7f3e-4e4a-aa9d-e819b71617fb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:12.779212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:12.779916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.780282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.780607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.781318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:12.783324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:12.820825 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4201/8424] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:12 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:12.825217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:12.842919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa975ede-8d95-4dce-8058-62fa4bd6dcb7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:12.848309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:12.848979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.849256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.849524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.850168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:12.851822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:12.872530 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4224/8425] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:12.873319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:12.884326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5ef14-8367-4dfa-9960-5732cd2822b6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:12.893185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d78d83-f0c1-48f6-9031-0f976cb3292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:12.894066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78d83-f0c1-48f6-9031-0f976cb3292a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.894419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78d83-f0c1-48f6-9031-0f976cb3292a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.894762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78d83-f0c1-48f6-9031-0f976cb3292a admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.895751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d78d83-f0c1-48f6-9031-0f976cb3292a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:12.897563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d78d83-f0c1-48f6-9031-0f976cb3292a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:12.912724 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4202/8426] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:12.914744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:12.925727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ecffd-99b1-464d-abbb-10a1b1d3540b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:12.930968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:12.931710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.932051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.932386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] PATH_INFO: `/v3/projects/16cb4161df8d40f1bb5c26dd5dc56128/users/e2ff9f24f59545719211216adf0bffcc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.949414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cb4161df8d40f1bb5c26dd5dc56128, user_id=e2ff9f24f59545719211216adf0bffcc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:12.952176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:12.970074 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4225/8427] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:12 2022] PUT /identity/v3/projects/16cb4161df8d40f1bb5c26dd5dc56128/users/e2ff9f24f59545719211216adf0bffcc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:12.981660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c47c52-fb92-4571-aa84-8bf7fd03b381 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:12.981969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c47c52-fb92-4571-aa84-8bf7fd03b381 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:12.982130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c47c52-fb92-4571-aa84-8bf7fd03b381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:12.990522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:13.013895 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c5c47c52-fb92-4571-aa84-8bf7fd03b381 None None] MFA Rules not processed for user `e2ff9f24f59545719211216adf0bffcc`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:13.015112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c5c47c52-fb92-4571-aa84-8bf7fd03b381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:13.086769 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4203/8428] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:13.091202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:13.152170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:13.170866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229452-9ab5-4290-a8db-4abc98a0ac6b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:13.175530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:13.176222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.176494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.176858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.177554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:13.179465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:13.208313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4226/8429] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:13 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:13.212339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c47c52-fb92-4571-aa84-8bf7fd03b381 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:13.223402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c47c52-fb92-4571-aa84-8bf7fd03b381 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:13.281095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:13.281695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.281975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.282242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.282871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:13.284800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:13.308566 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4204/8430] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:13 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:13.312822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:13.328138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a50b41-b712-42f6-b79f-dc084bddbb91 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:13.338402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:13.339208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.339491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.339792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.340433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:13.342127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:13.354623 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4227/8431] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:13.359046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:13.369620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f5906-aeac-4543-9111-a132c5239e75 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:13.374527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12620634-3423-4587-a938-f8c26336435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:13.375193 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12620634-3423-4587-a938-f8c26336435f admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.375472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12620634-3423-4587-a938-f8c26336435f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.375743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12620634-3423-4587-a938-f8c26336435f admin admin] PATH_INFO: `/v3/projects/f0e343874994409c9699ead6814f23fe/users/728d275c18004ba494f879b37df2d905/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.392585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12620634-3423-4587-a938-f8c26336435f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e343874994409c9699ead6814f23fe, user_id=728d275c18004ba494f879b37df2d905, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:13.395539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12620634-3423-4587-a938-f8c26336435f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:13.410542 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4205/8432] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:13 2022] PUT /identity/v3/projects/f0e343874994409c9699ead6814f23fe/users/728d275c18004ba494f879b37df2d905/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:13.414939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:13.425767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060445a-c972-499a-b158-576da52a6ea5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:13.489768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d54985-16ab-4176-a458-666e180a8a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:13.490392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d54985-16ab-4176-a458-666e180a8a6c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.490672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d54985-16ab-4176-a458-666e180a8a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.490978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d54985-16ab-4176-a458-666e180a8a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.491619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d54985-16ab-4176-a458-666e180a8a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:13.493250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d54985-16ab-4176-a458-666e180a8a6c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:13.504381 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4228/8433] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:13.509358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12620634-3423-4587-a938-f8c26336435f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:13.520049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12620634-3423-4587-a938-f8c26336435f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:13.525534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:13.526204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.526570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.526854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] PATH_INFO: `/v3/projects/f0e343874994409c9699ead6814f23fe/users/728d275c18004ba494f879b37df2d905/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.528976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e343874994409c9699ead6814f23fe, user_id=728d275c18004ba494f879b37df2d905, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:13.531665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:13.543067 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4206/8434] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:13 2022] PUT /identity/v3/projects/f0e343874994409c9699ead6814f23fe/users/728d275c18004ba494f879b37df2d905/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:13.553491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05ba8-52df-4b00-ba32-b724647a9c28 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.553772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05ba8-52df-4b00-ba32-b724647a9c28 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.554038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05ba8-52df-4b00-ba32-b724647a9c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.583243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d6c05ba8-52df-4b00-ba32-b724647a9c28 None None] MFA Rules not processed for user `728d275c18004ba494f879b37df2d905`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:13.584503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d6c05ba8-52df-4b00-ba32-b724647a9c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:13.665556 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4229/8435] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:13.676813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:13.678808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:13.749416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:13.761112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ae191-d971-4c5e-bfa5-09418f9d7586 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:13.765906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b48a9-89fb-453d-9252-5dbf04f8d796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:13.766636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b48a9-89fb-453d-9252-5dbf04f8d796 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:13.766977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b48a9-89fb-453d-9252-5dbf04f8d796 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:13.767292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b48a9-89fb-453d-9252-5dbf04f8d796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:13.777857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b48a9-89fb-453d-9252-5dbf04f8d796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:13.779660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b48a9-89fb-453d-9252-5dbf04f8d796 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:13.793125 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4207/8436] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:13.949926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c05ba8-52df-4b00-ba32-b724647a9c28 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:14.007173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:14.031186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c05ba8-52df-4b00-ba32-b724647a9c28 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:14.035654 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:14.036309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:14.036590 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:14.036890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] PATH_INFO: `/v3/users/e2ff9f24f59545719211216adf0bffcc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:14.038046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ff9f24f59545719211216adf0bffcc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:14.039986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:14.107754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ff9f24f59545719211216adf0bffcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.108968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ff9f24f59545719211216adf0bffcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.120174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ff9f24f59545719211216adf0bffcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.129319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ff9f24f59545719211216adf0bffcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.130172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ff9f24f59545719211216adf0bffcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.153471 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4230/8437] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:13 2022] DELETE /identity/v3/users/e2ff9f24f59545719211216adf0bffcc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:14.156881 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:14.157186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:14.157583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:14.196575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:14.197870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:14.277290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4208/8438] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:14.290405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:14.292171 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:14.382073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f2f54-363d-497b-b270-a302e5e375a5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:14.385930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:14.386839 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:14.387124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:14.387398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:14.399399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:14.405093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:14.418655 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4231/8439] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:14.804209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:14.805760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:14.818706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9709d-b4aa-4463-9701-29aa112449fc None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:14.824561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:14.825512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:14.826019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:14.826432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] PATH_INFO: `/v3/projects/16cb4161df8d40f1bb5c26dd5dc56128` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:14.828097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cb4161df8d40f1bb5c26dd5dc56128)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:14.830247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:14.883641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cb4161df8d40f1bb5c26dd5dc56128', 'request_id': 'req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.892926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cb4161df8d40f1bb5c26dd5dc56128', 'request_id': 'req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.903534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cb4161df8d40f1bb5c26dd5dc56128', 'request_id': 'req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.926340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] The token cache is being invalidate because project 16cb4161df8d40f1bb5c26dd5dc56128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:14.926636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.927917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:14.931027 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4209/8440] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:14 2022] DELETE /identity/v3/projects/16cb4161df8d40f1bb5c26dd5dc56128 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:14.935926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:14.947294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f940a8-dd8e-4887-959f-6116e342e8dc service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:15.024283 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:15.025087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.025432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.025817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] PATH_INFO: `/v3/users/728d275c18004ba494f879b37df2d905` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.027041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] RBAC: Authorizing `identity:delete_user(user_id=728d275c18004ba494f879b37df2d905)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:15.029174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:15.032442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:15.092690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728d275c18004ba494f879b37df2d905'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.093972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728d275c18004ba494f879b37df2d905'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.105264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728d275c18004ba494f879b37df2d905'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.115136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728d275c18004ba494f879b37df2d905'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.115967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b4f46916-7f0a-410e-bcc6-ec55da5b938d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728d275c18004ba494f879b37df2d905'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.135789 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4232/8441] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:14 2022] DELETE /identity/v3/users/728d275c18004ba494f879b37df2d905 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:15.438320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:15.439931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:15.454152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42503e-780c-4287-85fc-8a5c3aa9e5c7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:15.516525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:15.517226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.517524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.517798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] PATH_INFO: `/v3/projects/f0e343874994409c9699ead6814f23fe` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.518985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e343874994409c9699ead6814f23fe)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:15.521196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:15.540006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:15.578092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e343874994409c9699ead6814f23fe', 'request_id': 'req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.585262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e343874994409c9699ead6814f23fe', 'request_id': 'req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.595147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e343874994409c9699ead6814f23fe', 'request_id': 'req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.614155 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] The token cache is being invalidate because project f0e343874994409c9699ead6814f23fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:15.614495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.615336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-d42faa54-2dbb-45c0-a0c1-1540b1d1625a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:15.616938 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4210/8442] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:15 2022] DELETE /identity/v3/projects/f0e343874994409c9699ead6814f23fe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:15.622145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56e3e4-bc7b-4126-9c0f-4502e8554d13 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.622615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56e3e4-bc7b-4126-9c0f-4502e8554d13 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.622820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56e3e4-bc7b-4126-9c0f-4502e8554d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.643729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ee56e3e4-bc7b-4126-9c0f-4502e8554d13 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:15.645837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ee56e3e4-bc7b-4126-9c0f-4502e8554d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:15.711765 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4233/8443] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:15.720100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac56d41-1979-46d1-a4f1-aac6526c235d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.720380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac56d41-1979-46d1-a4f1-aac6526c235d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.720821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac56d41-1979-46d1-a4f1-aac6526c235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.744369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7ac56d41-1979-46d1-a4f1-aac6526c235d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:15.745634 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7ac56d41-1979-46d1-a4f1-aac6526c235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:15.753603 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4211/8444] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:15.758638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56e3e4-bc7b-4126-9c0f-4502e8554d13 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:15.768508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56e3e4-bc7b-4126-9c0f-4502e8554d13 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:15.773630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:15.774242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.774520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.774829 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.775477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:15.775778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:15.777418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:15.793417 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4234/8445] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:15.794730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac56d41-1979-46d1-a4f1-aac6526c235d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:15.804378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac56d41-1979-46d1-a4f1-aac6526c235d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:15.809544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:15.810159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.810433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.810680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.811309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:15.813293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:15.849300 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4212/8446] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:15 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:15.850788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:15.861215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23531bf-4a34-459f-b2ca-3a6d9e8a7f00 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:15.932358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:15.933155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.933458 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.933859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.934559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:15.936605 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:15.955765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:15.962856 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4235/8447] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:15 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:15.967360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:15.980048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084a234-c790-46ce-a0b7-61daa27187f2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:15.984952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:15.985597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:15.985898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:15.986145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:15.986791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:15.988607 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:16.000821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4213/8448] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:16.004894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:16.016221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678325-5055-4eb1-99aa-cad447ef7ea1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:16.023180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad640d-b1d5-4870-b277-133ed4440cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:16.023797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad640d-b1d5-4870-b277-133ed4440cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:16.024087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad640d-b1d5-4870-b277-133ed4440cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:16.025332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad640d-b1d5-4870-b277-133ed4440cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:16.025332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad640d-b1d5-4870-b277-133ed4440cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:16.026865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad640d-b1d5-4870-b277-133ed4440cf1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:16.043414 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4236/8449] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:16.044978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:16.057113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092e712-cbc8-4fca-94e8-78f44cc3df15 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:16.063239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:16.064281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:16.064619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:16.065004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] PATH_INFO: `/v3/projects/c0b5a82c37db47a386f18c98b77044a0/users/bc86651b19cb4fa498b566c5d76eb4e9/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:16.084734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b5a82c37db47a386f18c98b77044a0, user_id=bc86651b19cb4fa498b566c5d76eb4e9, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:16.087862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:16.113446 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4214/8450] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:16 2022] PUT /identity/v3/projects/c0b5a82c37db47a386f18c98b77044a0/users/bc86651b19cb4fa498b566c5d76eb4e9/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:16.124874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:16.125243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:16.125418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:16.157665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] MFA Rules not processed for user `bc86651b19cb4fa498b566c5d76eb4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:16.158808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:16.236707 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4237/8451] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:16.261216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:16.267130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:16.363889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b17ff-2c23-4055-a53a-71f11d051ecd admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:16.367858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977854b-4c73-44b7-ba18-c8c34892b92a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:16.368496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977854b-4c73-44b7-ba18-c8c34892b92a service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:16.368815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977854b-4c73-44b7-ba18-c8c34892b92a service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:16.369123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977854b-4c73-44b7-ba18-c8c34892b92a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:16.379211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977854b-4c73-44b7-ba18-c8c34892b92a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:16.380689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977854b-4c73-44b7-ba18-c8c34892b92a service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:16.392216 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4215/8452] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:18.426457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:18.427999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:18.499476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ed249-8cd4-47dd-8611-e0f30171b90a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:18.504009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:18.504674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:18.504953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:18.505220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] PATH_INFO: `/v3/users/34a85525c845484280f7c50d88cb5719` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:18.506552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a85525c845484280f7c50d88cb5719)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:18.508610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:18.559113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a85525c845484280f7c50d88cb5719'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:18.559753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a85525c845484280f7c50d88cb5719'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:18.571323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a85525c845484280f7c50d88cb5719'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:18.579064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a85525c845484280f7c50d88cb5719'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:18.579738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a85525c845484280f7c50d88cb5719'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:18.591728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:18.601833 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4238/8453] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:18 2022] DELETE /identity/v3/users/34a85525c845484280f7c50d88cb5719 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:18.606551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:18.607029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:18.607141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:18.627779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:18.629507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:18.671455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:18.695237 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4216/8454] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:18.705320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:18.706921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:18.769902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabc54a-5606-4eda-a0da-223ff7eab2e7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:18.774016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7934ed-9bc2-42d9-ad7d-ef8dbd392711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:18.774616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7934ed-9bc2-42d9-ad7d-ef8dbd392711 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:18.774954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7934ed-9bc2-42d9-ad7d-ef8dbd392711 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:18.775126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7934ed-9bc2-42d9-ad7d-ef8dbd392711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:18.784856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7934ed-9bc2-42d9-ad7d-ef8dbd392711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:18.786515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7934ed-9bc2-42d9-ad7d-ef8dbd392711 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:18.796112 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4239/8455] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:18.982862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:18.986229 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:19.000287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd1906-ea1e-472a-a2e2-f6aecab68e54 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.005922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.006702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.006981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.007273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] PATH_INFO: `/v3/projects/8ff9168912304a0baf394fe13f3dcb14` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.008397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff9168912304a0baf394fe13f3dcb14)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.010406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.059980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff9168912304a0baf394fe13f3dcb14', 'request_id': 'req-9357d5b0-076d-4601-8db1-f43ad9059d1c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:19.066401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff9168912304a0baf394fe13f3dcb14', 'request_id': 'req-9357d5b0-076d-4601-8db1-f43ad9059d1c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:19.076078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff9168912304a0baf394fe13f3dcb14', 'request_id': 'req-9357d5b0-076d-4601-8db1-f43ad9059d1c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:19.086739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] The token cache is being invalidate because project 8ff9168912304a0baf394fe13f3dcb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:19.087028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:19.087652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9357d5b0-076d-4601-8db1-f43ad9059d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:19.089210 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4217/8456] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:18 2022] DELETE /identity/v3/projects/8ff9168912304a0baf394fe13f3dcb14 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:19.094663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c287c4-5935-41ce-ba6c-c504d6c94be3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.094945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c287c4-5935-41ce-ba6c-c504d6c94be3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.095218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c287c4-5935-41ce-ba6c-c504d6c94be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.117338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d4c287c4-5935-41ce-ba6c-c504d6c94be3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:19.118550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d4c287c4-5935-41ce-ba6c-c504d6c94be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:19.178630 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4240/8457] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:19.186749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19e13-8cdd-484c-bb19-4428442f79df None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.187031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19e13-8cdd-484c-bb19-4428442f79df None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.187306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19e13-8cdd-484c-bb19-4428442f79df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.209486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9cc19e13-8cdd-484c-bb19-4428442f79df None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:19.210689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9cc19e13-8cdd-484c-bb19-4428442f79df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:19.219444 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4218/8458] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:19.224591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c287c4-5935-41ce-ba6c-c504d6c94be3 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:19.234825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c287c4-5935-41ce-ba6c-c504d6c94be3 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.240989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.241592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.241868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.242147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.242784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:19.243087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.244705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.258312 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4241/8459] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:19.264420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc19e13-8cdd-484c-bb19-4428442f79df None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:19.274758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc19e13-8cdd-484c-bb19-4428442f79df None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.280994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.281669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.282078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.282520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.283258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.285581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.321451 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4219/8460] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:19 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:19.326480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:19.338803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631a9d3-f413-4c91-8d67-eb67398e0350 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.396108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.396737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.397016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.397320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.397940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.399850 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.425005 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4242/8461] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:19 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:19.430079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:19.441477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4dc96-a773-4962-a43f-431d193e438a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.447668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.448355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.448638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.448950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.449610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.451235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.465748 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4220/8462] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:19.470371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:19.481390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50351a77-4243-477e-ac99-f08590c04e2e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.486269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f557c63-dc63-4af8-b05c-b63fd90e8e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.486948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f557c63-dc63-4af8-b05c-b63fd90e8e7b admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.487304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f557c63-dc63-4af8-b05c-b63fd90e8e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.487628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f557c63-dc63-4af8-b05c-b63fd90e8e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.488476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f557c63-dc63-4af8-b05c-b63fd90e8e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.490125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f557c63-dc63-4af8-b05c-b63fd90e8e7b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.501184 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4243/8463] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:19.505842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:19.523048 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63670e3e-819d-46a7-9add-84f87bfc88c7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.528882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.529680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.529965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.530234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] PATH_INFO: `/v3/projects/1f1b7ce71ad646feaa3a0a954f1fea82/users/5546f6c9030f4db4a9b619a6e21af80b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.546873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1b7ce71ad646feaa3a0a954f1fea82, user_id=5546f6c9030f4db4a9b619a6e21af80b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.549643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.564418 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4221/8464] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:19 2022] PUT /identity/v3/projects/1f1b7ce71ad646feaa3a0a954f1fea82/users/5546f6c9030f4db4a9b619a6e21af80b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:19.577803 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb04d6-f7c0-46bf-8c88-ec02e5dea6ab None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.578029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb04d6-f7c0-46bf-8c88-ec02e5dea6ab None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.578304 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb04d6-f7c0-46bf-8c88-ec02e5dea6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.587536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:19.618931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4fbb04d6-f7c0-46bf-8c88-ec02e5dea6ab None None] MFA Rules not processed for user `5546f6c9030f4db4a9b619a6e21af80b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:19.620201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4fbb04d6-f7c0-46bf-8c88-ec02e5dea6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:19.689613 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4244/8465] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:19.700779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:19.702532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:19.766855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867935f-689a-4bb1-bea7-1b5d555761f7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:19.771187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a6866-36d7-496d-9b98-ce59580bc443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:19.772015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a6866-36d7-496d-9b98-ce59580bc443 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:19.772376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a6866-36d7-496d-9b98-ce59580bc443 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:19.772852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a6866-36d7-496d-9b98-ce59580bc443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:19.782744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a6866-36d7-496d-9b98-ce59580bc443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:19.784358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a6866-36d7-496d-9b98-ce59580bc443 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:19.794568 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4222/8466] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:24.348141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb04d6-f7c0-46bf-8c88-ec02e5dea6ab None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:24.419513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb04d6-f7c0-46bf-8c88-ec02e5dea6ab None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:24.424513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:24.425215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:24.425690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:24.425895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] PATH_INFO: `/v3/users/5546f6c9030f4db4a9b619a6e21af80b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:24.427058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5546f6c9030f4db4a9b619a6e21af80b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:24.429299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:24.480833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5546f6c9030f4db4a9b619a6e21af80b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:24.481492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5546f6c9030f4db4a9b619a6e21af80b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:24.492030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5546f6c9030f4db4a9b619a6e21af80b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:24.499832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5546f6c9030f4db4a9b619a6e21af80b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:24.500563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5546f6c9030f4db4a9b619a6e21af80b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:24.519649 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4245/8467] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:24 2022] DELETE /identity/v3/users/5546f6c9030f4db4a9b619a6e21af80b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:24.524826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:24.525324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:24.525418 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:24.534236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:24.548192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:24.549435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:24.613017 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4223/8468] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:24.623620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:24.625113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:24.642250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:24.690932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fc296-7e52-4ca9-8664-39bae10add16 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:24.695221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ae360-250f-4070-9e16-405efbce1c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:24.695916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae360-250f-4070-9e16-405efbce1c22 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:24.696193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae360-250f-4070-9e16-405efbce1c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:24.696464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae360-250f-4070-9e16-405efbce1c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:24.705896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ae360-250f-4070-9e16-405efbce1c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:24.707592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ae360-250f-4070-9e16-405efbce1c22 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:24.717644 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4246/8469] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:37:24.903771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:24.905678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:24.920854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d77dd7-d346-4c5f-897a-3e1f7b163673 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:24.927033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:24.927729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:24.928058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:24.928420 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] PATH_INFO: `/v3/projects/1f1b7ce71ad646feaa3a0a954f1fea82` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:24.929751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f1b7ce71ad646feaa3a0a954f1fea82)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:24.931885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:24.988443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f1b7ce71ad646feaa3a0a954f1fea82', 'request_id': 'req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:24.994994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f1b7ce71ad646feaa3a0a954f1fea82', 'request_id': 'req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:25.006010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f1b7ce71ad646feaa3a0a954f1fea82', 'request_id': 'req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:25.018033 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] The token cache is being invalidate because project 1f1b7ce71ad646feaa3a0a954f1fea82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:25.018326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:25.018979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-7f95780c-86fa-43ca-bb01-f3fb3f5c7750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:25.020828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4224/8470] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:24 2022] DELETE /identity/v3/projects/1f1b7ce71ad646feaa3a0a954f1fea82 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:25.026031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892729b4-670a-4b7b-b05a-ffa54f9c5a9c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.026319 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892729b4-670a-4b7b-b05a-ffa54f9c5a9c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.026601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892729b4-670a-4b7b-b05a-ffa54f9c5a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.051515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-892729b4-670a-4b7b-b05a-ffa54f9c5a9c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:25.053218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-892729b4-670a-4b7b-b05a-ffa54f9c5a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:25.122394 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4247/8471] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:25.130775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e764b-052f-4100-9343-e40b8ec09fad None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.131056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e764b-052f-4100-9343-e40b8ec09fad None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.131339 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e764b-052f-4100-9343-e40b8ec09fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.151859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-036e764b-052f-4100-9343-e40b8ec09fad None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:25.153142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-036e764b-052f-4100-9343-e40b8ec09fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:25.161856 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4225/8472] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:25.166170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892729b4-670a-4b7b-b05a-ffa54f9c5a9c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:25.176706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892729b4-670a-4b7b-b05a-ffa54f9c5a9c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:25.181453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:25.182039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.182327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.182598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.183250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:25.183553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:25.185202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:25.201617 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4248/8473] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:25.202795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e764b-052f-4100-9343-e40b8ec09fad None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:25.213549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e764b-052f-4100-9343-e40b8ec09fad None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:25.219242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:25.219970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.220451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.220893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.221723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:25.223961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:25.256220 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4226/8474] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:25 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:25.257588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:25.269128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2c243-a1a8-4273-a173-3b30f1aa93bd admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:25.331114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:25.332415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.332725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.333013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.333661 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:25.335898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:25.360709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4249/8475] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:25 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:25.364649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:25.375191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8740-50fa-43fe-8194-e6715c3ecac7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:25.379917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:25.380518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.380828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.381099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.381742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:25.383327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:25.393845 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4227/8476] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:25.398801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:25.410780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a7634-6d46-4c6a-b40a-9bbcc3251fb9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:25.416291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a21483-9c75-48ac-9e99-b1956138b341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:25.417018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a21483-9c75-48ac-9e99-b1956138b341 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.417305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a21483-9c75-48ac-9e99-b1956138b341 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.417561 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a21483-9c75-48ac-9e99-b1956138b341 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.418194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a21483-9c75-48ac-9e99-b1956138b341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:25.419780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a21483-9c75-48ac-9e99-b1956138b341 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:25.430723 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4250/8477] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:25.435483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:25.448514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3962d3-7d87-4c0a-a6ae-38a291e46b7b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:25.456928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:25.458143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.458490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.458835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] PATH_INFO: `/v3/projects/8ea2df7242cf4f8a9ce11f513383be50/users/7f9a97e2b2ae4de298bd09c7dd092ecc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.479404 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea2df7242cf4f8a9ce11f513383be50, user_id=7f9a97e2b2ae4de298bd09c7dd092ecc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:25.482126 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:25.497801 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4228/8478] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:25 2022] PUT /identity/v3/projects/8ea2df7242cf4f8a9ce11f513383be50/users/7f9a97e2b2ae4de298bd09c7dd092ecc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:25.509356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7a01d-b7ac-4f6e-9180-f7a664507191 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.509636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7a01d-b7ac-4f6e-9180-f7a664507191 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.509906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7a01d-b7ac-4f6e-9180-f7a664507191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.539646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2ba7a01d-b7ac-4f6e-9180-f7a664507191 None None] MFA Rules not processed for user `7f9a97e2b2ae4de298bd09c7dd092ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:25.540890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2ba7a01d-b7ac-4f6e-9180-f7a664507191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:25.551151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:25.609304 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4251/8479] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:25.621047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:25.623059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:25.651938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:25.688525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e18a8-9367-4c73-92c0-d008dabf2fa8 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:25.693241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231301cb-5274-418b-bc0a-4d6378fa06eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:25.694186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231301cb-5274-418b-bc0a-4d6378fa06eb service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:25.694441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231301cb-5274-418b-bc0a-4d6378fa06eb service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:25.694707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231301cb-5274-418b-bc0a-4d6378fa06eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:25.704041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231301cb-5274-418b-bc0a-4d6378fa06eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:25.705506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231301cb-5274-418b-bc0a-4d6378fa06eb service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:25.715044 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4229/8480] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:26.077175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7a01d-b7ac-4f6e-9180-f7a664507191 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:26.146091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7a01d-b7ac-4f6e-9180-f7a664507191 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:26.150707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:26.151392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:26.151671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:26.151941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] PATH_INFO: `/v3/users/7f9a97e2b2ae4de298bd09c7dd092ecc` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:26.153086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9a97e2b2ae4de298bd09c7dd092ecc)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:26.155023 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:26.206348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9a97e2b2ae4de298bd09c7dd092ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.207033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9a97e2b2ae4de298bd09c7dd092ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.215462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9a97e2b2ae4de298bd09c7dd092ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.223040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9a97e2b2ae4de298bd09c7dd092ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.223793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9a97e2b2ae4de298bd09c7dd092ecc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.242902 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4252/8481] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:26 2022] DELETE /identity/v3/users/7f9a97e2b2ae4de298bd09c7dd092ecc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:26.247716 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:26.247997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:26.248289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:26.269536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:26.270824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:26.351021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4230/8482] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:26.357392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:26.358890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:26.398089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:26.426940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87595b2a-0b93-49c6-ad35-f0271f4a2760 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:26.430949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450baec1-bd05-4cbe-9b45-ff61398cfb47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:26.431565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450baec1-bd05-4cbe-9b45-ff61398cfb47 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:26.431841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450baec1-bd05-4cbe-9b45-ff61398cfb47 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:26.432108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450baec1-bd05-4cbe-9b45-ff61398cfb47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:26.442264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450baec1-bd05-4cbe-9b45-ff61398cfb47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:26.444238 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450baec1-bd05-4cbe-9b45-ff61398cfb47 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:26.454515 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4253/8483] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:26.743979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:26.745486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:26.760034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5372-9ca1-49d7-93a8-fa9afe494f43 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:26.764933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:26.765747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:26.766026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:26.766297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] PATH_INFO: `/v3/projects/8ea2df7242cf4f8a9ce11f513383be50` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:26.767465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea2df7242cf4f8a9ce11f513383be50)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:26.769748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:26.822406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea2df7242cf4f8a9ce11f513383be50', 'request_id': 'req-e9ed2643-73f6-44f9-b12d-57e76c773d86'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.830867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea2df7242cf4f8a9ce11f513383be50', 'request_id': 'req-e9ed2643-73f6-44f9-b12d-57e76c773d86'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.840540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea2df7242cf4f8a9ce11f513383be50', 'request_id': 'req-e9ed2643-73f6-44f9-b12d-57e76c773d86'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.851753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] The token cache is being invalidate because project 8ea2df7242cf4f8a9ce11f513383be50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:26.852042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.852701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e9ed2643-73f6-44f9-b12d-57e76c773d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:26.854223 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4231/8484] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:26 2022] DELETE /identity/v3/projects/8ea2df7242cf4f8a9ce11f513383be50 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:26.859254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2b652-9fbe-45e7-9907-780f664a7e88 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:26.859536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2b652-9fbe-45e7-9907-780f664a7e88 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:26.859811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2b652-9fbe-45e7-9907-780f664a7e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:26.881101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8bc2b652-9fbe-45e7-9907-780f664a7e88 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:26.882297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8bc2b652-9fbe-45e7-9907-780f664a7e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:26.947123 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4254/8485] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:26.955354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63258a-2f41-4678-8ea4-3b5b148ae14a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:26.955635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63258a-2f41-4678-8ea4-3b5b148ae14a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:26.955909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63258a-2f41-4678-8ea4-3b5b148ae14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:26.976316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4d63258a-2f41-4678-8ea4-3b5b148ae14a None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:26.977544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d63258a-2f41-4678-8ea4-3b5b148ae14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:26.985943 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4232/8486] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:26.990835 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2b652-9fbe-45e7-9907-780f664a7e88 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.001200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2b652-9fbe-45e7-9907-780f664a7e88 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.006336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.006937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.007204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.007503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.008139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:27.008441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.010063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.021619 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4255/8487] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:27.025932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63258a-2f41-4678-8ea4-3b5b148ae14a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.035888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63258a-2f41-4678-8ea4-3b5b148ae14a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.040796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.041401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.041697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.041968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.042596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.044570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.078813 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4233/8488] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:27 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:27.083722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.094963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb923c4-78e6-4fc0-a4a9-695e1c619c03 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.153933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.154530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.154776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.155062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.155705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.157679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.184695 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4256/8489] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:27 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:27.186109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.197640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c57d-1a79-4b20-98df-8b88806db46c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.203503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.204141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.204415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.204710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.205376 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.207496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.218324 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4234/8490] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:27.223475 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.234685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df5c2d-fb09-4ade-a3cf-858ad695acd7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.240320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef7ca1-0b16-48f3-afe2-d3c6551bfc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.240991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef7ca1-0b16-48f3-afe2-d3c6551bfc03 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.241266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef7ca1-0b16-48f3-afe2-d3c6551bfc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.241532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef7ca1-0b16-48f3-afe2-d3c6551bfc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.242161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef7ca1-0b16-48f3-afe2-d3c6551bfc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.243759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef7ca1-0b16-48f3-afe2-d3c6551bfc03 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.255860 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4257/8491] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:27.260208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.271565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b87122-3672-4756-bcc1-a90c86ce50c0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.277534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.278204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.278509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.278786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] PATH_INFO: `/v3/projects/083f8a9f05154994b0cd6423c431cc0d/users/8e454d008f3b43088c2dbc2e6ac0e4e4/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.288865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:27.296811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=083f8a9f05154994b0cd6423c431cc0d, user_id=8e454d008f3b43088c2dbc2e6ac0e4e4, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.299573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.316359 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4235/8492] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:27 2022] PUT /identity/v3/projects/083f8a9f05154994b0cd6423c431cc0d/users/8e454d008f3b43088c2dbc2e6ac0e4e4/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:27.327556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5859e2-0e79-431b-aefd-8c16a418eaf9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.327874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5859e2-0e79-431b-aefd-8c16a418eaf9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.328153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5859e2-0e79-431b-aefd-8c16a418eaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.361469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-8f5859e2-0e79-431b-aefd-8c16a418eaf9 None None] MFA Rules not processed for user `8e454d008f3b43088c2dbc2e6ac0e4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:27.362472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8f5859e2-0e79-431b-aefd-8c16a418eaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:27.394242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:27.436161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4258/8493] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:27.440409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.511902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ed737-2dbe-42b0-90a3-3b3cc8c74593 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.516698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.517362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.517686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.517991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.518691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.520918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.550910 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4236/8494] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:27 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:27.555419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5859e2-0e79-431b-aefd-8c16a418eaf9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.569554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5859e2-0e79-431b-aefd-8c16a418eaf9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.633371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.634118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.634396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.634668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.635321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.637316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.663814 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4259/8495] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:27 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:27.665471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.678045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bb353-4b59-4ea1-af44-89bbd90dff6e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.683364 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.684152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.684463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.684856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.685552 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.687588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.703716 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4237/8496] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:27.705291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.716967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465803-82fa-4bd0-871f-d91a86ad5262 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.722256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.723066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.723421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.723776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] PATH_INFO: `/v3/projects/3d58db3cacfe4faca9a60ef175aa5a38/users/3e93200cc16c48c9ac95f51a3e49dbb9/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.742526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d58db3cacfe4faca9a60ef175aa5a38, user_id=3e93200cc16c48c9ac95f51a3e49dbb9, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.745432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.759197 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4260/8497] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:27 2022] PUT /identity/v3/projects/3d58db3cacfe4faca9a60ef175aa5a38/users/3e93200cc16c48c9ac95f51a3e49dbb9/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:27.763274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.773819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3dfb4-02d0-4b7d-9765-933cc8c7612f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.832189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd893b-4498-4fbb-aac8-843ff842af23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.832851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd893b-4498-4fbb-aac8-843ff842af23 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.833132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd893b-4498-4fbb-aac8-843ff842af23 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.833441 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd893b-4498-4fbb-aac8-843ff842af23 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.834083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd893b-4498-4fbb-aac8-843ff842af23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.835704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd893b-4498-4fbb-aac8-843ff842af23 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.846193 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4238/8498] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:27.850632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:27.861768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80176c-25ea-4ca0-8424-83cd81493f25 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:27.867445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:27.868098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.868373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.868636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] PATH_INFO: `/v3/projects/3d58db3cacfe4faca9a60ef175aa5a38/users/3e93200cc16c48c9ac95f51a3e49dbb9/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:27.870837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d58db3cacfe4faca9a60ef175aa5a38, user_id=3e93200cc16c48c9ac95f51a3e49dbb9, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:27.873557 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:27.884398 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4261/8499] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:27 2022] PUT /identity/v3/projects/3d58db3cacfe4faca9a60ef175aa5a38/users/3e93200cc16c48c9ac95f51a3e49dbb9/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:27.988042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84b37-8445-4f27-a976-b20f253412d8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:27.988298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84b37-8445-4f27-a976-b20f253412d8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:27.988604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84b37-8445-4f27-a976-b20f253412d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:28.020687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0cd84b37-8445-4f27-a976-b20f253412d8 None None] MFA Rules not processed for user `3e93200cc16c48c9ac95f51a3e49dbb9`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:28.021956 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0cd84b37-8445-4f27-a976-b20f253412d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:28.071738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:28.097306 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4239/8500] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:28.109255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:28.111247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:28.190285 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4df5c-4c5f-4a53-81a3-2d0d3c3df363 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:28.195165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:28.195780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:28.196058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:28.196354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:28.207092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:28.208547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:28.218801 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4262/8501] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:28.301093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd84b37-8445-4f27-a976-b20f253412d8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:28.313714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd84b37-8445-4f27-a976-b20f253412d8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:28.317616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6d00b-a30f-43c9-915c-1bdd59d91b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:28.318231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6d00b-a30f-43c9-915c-1bdd59d91b0b service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:28.318866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6d00b-a30f-43c9-915c-1bdd59d91b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:28.319145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6d00b-a30f-43c9-915c-1bdd59d91b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:28.395959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6d00b-a30f-43c9-915c-1bdd59d91b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:28.397724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6d00b-a30f-43c9-915c-1bdd59d91b0b service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:28.408290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4240/8502] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:36.501402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:36.515702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:36.572957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf30c-f5ac-4780-82cc-b3f16151dab5 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:36.577563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:36.578202 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:36.578478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:36.578746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] PATH_INFO: `/v3/users/8e454d008f3b43088c2dbc2e6ac0e4e4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:36.579894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e454d008f3b43088c2dbc2e6ac0e4e4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:36.582006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:36.637719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e454d008f3b43088c2dbc2e6ac0e4e4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:36.638392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e454d008f3b43088c2dbc2e6ac0e4e4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:36.648849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e454d008f3b43088c2dbc2e6ac0e4e4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:36.656020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e454d008f3b43088c2dbc2e6ac0e4e4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:36.656644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e454d008f3b43088c2dbc2e6ac0e4e4'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:36.673909 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4263/8503] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:36 2022] DELETE /identity/v3/users/8e454d008f3b43088c2dbc2e6ac0e4e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:36.678958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:36.679413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:36.679506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:36.699252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:36.700459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:36.767487 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4241/8504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:36.774862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:36.776330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:36.839117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0dfbe-91fa-4166-bbd5-d6e63fa2e520 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:36.843272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:36.843876 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:36.844148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:36.844402 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:36.855235 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:36.856685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:36.866595 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4264/8505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:37:37.142240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:37.143691 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:37.156263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec3f7c-4fe8-45a5-b684-b3959219a398 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:37.161443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:37.162262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.162470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.162742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] PATH_INFO: `/v3/projects/083f8a9f05154994b0cd6423c431cc0d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.163973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] RBAC: Authorizing `identity:delete_project(project_id=083f8a9f05154994b0cd6423c431cc0d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:37.165937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:37.218252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083f8a9f05154994b0cd6423c431cc0d', 'request_id': 'req-13a3211e-54ee-4858-b189-25ded12200df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.225102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083f8a9f05154994b0cd6423c431cc0d', 'request_id': 'req-13a3211e-54ee-4858-b189-25ded12200df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.234013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083f8a9f05154994b0cd6423c431cc0d', 'request_id': 'req-13a3211e-54ee-4858-b189-25ded12200df'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.244994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] The token cache is being invalidate because project 083f8a9f05154994b0cd6423c431cc0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:37.245281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.245908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.247411 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4242/8506] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:37 2022] DELETE /identity/v3/projects/083f8a9f05154994b0cd6423c431cc0d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:37.251611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:37.262149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a61eae-86c5-4c6b-92ea-e133f18d475f service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:37.298413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:37.300489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:37.302289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:37.309882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:37.328570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:37.329195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.329472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.329744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] PATH_INFO: `/v3/users/3e93200cc16c48c9ac95f51a3e49dbb9` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.330882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e93200cc16c48c9ac95f51a3e49dbb9)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:37.332832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:37.377245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3211e-54ee-4858-b189-25ded12200df admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:37.382211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:37.382879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.383162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.383437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] PATH_INFO: `/v3/users/bc86651b19cb4fa498b566c5d76eb4e9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.383959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e93200cc16c48c9ac95f51a3e49dbb9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.384589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e93200cc16c48c9ac95f51a3e49dbb9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.385324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc86651b19cb4fa498b566c5d76eb4e9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:37.387578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:37.399679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e93200cc16c48c9ac95f51a3e49dbb9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.407321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e93200cc16c48c9ac95f51a3e49dbb9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.407971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b7366750-0835-44e2-97ff-b9ce5619a117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e93200cc16c48c9ac95f51a3e49dbb9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.427320 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4265/8507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:37 2022] DELETE /identity/v3/users/3e93200cc16c48c9ac95f51a3e49dbb9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:37.438447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc86651b19cb4fa498b566c5d76eb4e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.439083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc86651b19cb4fa498b566c5d76eb4e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.447205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc86651b19cb4fa498b566c5d76eb4e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.457897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc86651b19cb4fa498b566c5d76eb4e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.474482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc86651b19cb4fa498b566c5d76eb4e9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.481669 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4243/8508] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:37 2022] DELETE /identity/v3/users/bc86651b19cb4fa498b566c5d76eb4e9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:37.487657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.488384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.489174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.514846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:37.516056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:37.576926 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4266/8509] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:37.587210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:37.588648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:37.666559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36044e01-4a0f-4836-b72c-dd99c303afb4 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:37.670337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19253c74-df26-425e-878f-94f957d5b7c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:37.670957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19253c74-df26-425e-878f-94f957d5b7c8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.671238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19253c74-df26-425e-878f-94f957d5b7c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.671514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19253c74-df26-425e-878f-94f957d5b7c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.681524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19253c74-df26-425e-878f-94f957d5b7c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:37.683029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19253c74-df26-425e-878f-94f957d5b7c8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:37.693870 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4244/8510] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:37.754819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:37.757059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:37.770026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b952dd-9f07-4d2a-86e2-25072b928a25 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:37.777072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:37.777767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.778082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.778716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] PATH_INFO: `/v3/projects/3d58db3cacfe4faca9a60ef175aa5a38` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.779937 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d58db3cacfe4faca9a60ef175aa5a38)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:37.781820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:37.838480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d58db3cacfe4faca9a60ef175aa5a38', 'request_id': 'req-59e63a4c-2565-4391-9c31-6e113b3ba61d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.841232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:37.849905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d58db3cacfe4faca9a60ef175aa5a38', 'request_id': 'req-59e63a4c-2565-4391-9c31-6e113b3ba61d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.856968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d58db3cacfe4faca9a60ef175aa5a38', 'request_id': 'req-59e63a4c-2565-4391-9c31-6e113b3ba61d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.869368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] The token cache is being invalidate because project 3d58db3cacfe4faca9a60ef175aa5a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:37.869926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.870487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-59e63a4c-2565-4391-9c31-6e113b3ba61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:37.872004 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4267/8511] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:37 2022] DELETE /identity/v3/projects/3d58db3cacfe4faca9a60ef175aa5a38 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:37.877148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.877522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.877705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.899201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:37.900439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:37.921606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:37.967076 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4245/8512] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:37.975798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589df65-1ce1-457f-be03-2f390134e306 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:37.976079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589df65-1ce1-457f-be03-2f390134e306 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:37.976348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589df65-1ce1-457f-be03-2f390134e306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:37.997812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1589df65-1ce1-457f-be03-2f390134e306 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:37.999144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1589df65-1ce1-457f-be03-2f390134e306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.008683 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4268/8513] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:38.010557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.012124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.012840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589df65-1ce1-457f-be03-2f390134e306 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.022879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589df65-1ce1-457f-be03-2f390134e306 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.025570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38341801-d60a-46ec-ac0f-0738ac86d5ee None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.028626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.029414 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.029700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.029970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.030622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:38.031335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.032353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.032551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.033330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.033890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.034303 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] PATH_INFO: `/v3/projects/c0b5a82c37db47a386f18c98b77044a0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.035797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b5a82c37db47a386f18c98b77044a0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.038296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.050801 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4269/8514] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:37:38.051923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.063585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17445c-1ca5-4a90-92aa-86d9ab8d03c2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.069882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.070500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.070769 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.071063 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.071730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.073791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.093307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b5a82c37db47a386f18c98b77044a0', 'request_id': 'req-407ac03c-3599-4508-bda5-b61283ceb724'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:38.101808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b5a82c37db47a386f18c98b77044a0', 'request_id': 'req-407ac03c-3599-4508-bda5-b61283ceb724'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:38.111357 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4270/8515] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:38.113666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b5a82c37db47a386f18c98b77044a0', 'request_id': 'req-407ac03c-3599-4508-bda5-b61283ceb724'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:38.116009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.126482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] The token cache is being invalidate because project c0b5a82c37db47a386f18c98b77044a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:38.127188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:38.127425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-407ac03c-3599-4508-bda5-b61283ceb724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:38.129188 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4246/8516] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:38 2022] DELETE /identity/v3/projects/c0b5a82c37db47a386f18c98b77044a0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:38.134474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833f1f-6be7-4eed-9cc7-190ef541e1fb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.134763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833f1f-6be7-4eed-9cc7-190ef541e1fb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.135036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833f1f-6be7-4eed-9cc7-190ef541e1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.157065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-38833f1f-6be7-4eed-9cc7-190ef541e1fb None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:38.158331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-38833f1f-6be7-4eed-9cc7-190ef541e1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.189153 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aefe5-50f2-4337-af5e-8c9c13e85a69 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.195388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.196103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.196378 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.196644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.197310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.199262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.224704 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4271/8517] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:38.224908 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4247/8518] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:38.227401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.229593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.231190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872ecc-cafe-4815-86bd-7bb5d5610661 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.231596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872ecc-cafe-4815-86bd-7bb5d5610661 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.231966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872ecc-cafe-4815-86bd-7bb5d5610661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.241978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e0003-bf92-463f-b0cc-1dc1c443bd10 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.246656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.247345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.247618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.247927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.248870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.250330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.256930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3b872ecc-cafe-4815-86bd-7bb5d5610661 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:38.256930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3b872ecc-cafe-4815-86bd-7bb5d5610661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.260901 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4272/8519] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:38.264910 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.268014 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4248/8520] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:38.272204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b872ecc-cafe-4815-86bd-7bb5d5610661 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.275191 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70112e-8c7b-4815-af66-ad46b1748288 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.279862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.280468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.280800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.281065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.281673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b872ecc-cafe-4815-86bd-7bb5d5610661 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.281795 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.283321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.286402 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.287056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.287481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.287756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.288397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:38.288713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.290345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.294313 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4273/8521] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:38.298495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.308984 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4249/8522] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:37:38.309147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c02734-3653-456d-a942-f9c83e0011c3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.310502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.313420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315194-4863-4eb2-ac45-a706623eedfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.314064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315194-4863-4eb2-ac45-a706623eedfb admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.314336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315194-4863-4eb2-ac45-a706623eedfb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.314601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315194-4863-4eb2-ac45-a706623eedfb admin admin] PATH_INFO: `/v3/projects/96e746f920e0493890b54622c6a442d0/users/79a1c44c751146c0a866c3478c969155/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.322173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2e931-4aee-427e-8fe4-9e983f8e681e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.327071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.327666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.327941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.328205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.328936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.329856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315194-4863-4eb2-ac45-a706623eedfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e746f920e0493890b54622c6a442d0, user_id=79a1c44c751146c0a866c3478c969155, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.330865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.332466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315194-4863-4eb2-ac45-a706623eedfb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.346767 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4274/8523] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:38 2022] PUT /identity/v3/projects/96e746f920e0493890b54622c6a442d0/users/79a1c44c751146c0a866c3478c969155/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:38.359495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57743ff-2e0e-4923-99cf-926f7d5c027c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.359772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57743ff-2e0e-4923-99cf-926f7d5c027c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.360039 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57743ff-2e0e-4923-99cf-926f7d5c027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.360437 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4250/8524] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:38.364621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.376062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:38.390192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b57743ff-2e0e-4923-99cf-926f7d5c027c None None] MFA Rules not processed for user `79a1c44c751146c0a866c3478c969155`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:38.391406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b57743ff-2e0e-4923-99cf-926f7d5c027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.426592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=6,keystone:DELETE=3,keystone:SELECT=91 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:38.433280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9362ad1-4e64-4009-900f-c194532c376f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.438164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.438790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.439072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.439346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.440016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.442004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.468710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4251/8525] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:38.470009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.471946 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4275/8526] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:38.476472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57743ff-2e0e-4923-99cf-926f7d5c027c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.480867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdf2fa-f377-4444-8a4e-837906d4aa55 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.485659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.486272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.486551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.486822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.487463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.487929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57743ff-2e0e-4923-99cf-926f7d5c027c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.489078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.493170 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.493808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.494089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.494362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.495001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.496977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.502364 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4252/8527] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:38.503730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.514618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50290b05-073d-4ace-996d-01473f676872 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.519934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.520541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.520914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.521179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.521970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.523615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.527730 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4276/8528] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:38.531940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.539747 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4253/8529] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:38.540775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.550540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede5be3-a552-43ad-a1ed-d9bf8565a92b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.595158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca200c31-2b67-47ff-a3e5-8b95cad694c3 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.601254 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.601800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.602080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.602355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.603029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.605416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.612551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ff2f4-bbee-4363-84eb-c0d0994575d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.613857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff2f4-bbee-4363-84eb-c0d0994575d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.615251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff2f4-bbee-4363-84eb-c0d0994575d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.615682 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff2f4-bbee-4363-84eb-c0d0994575d0 admin admin] PATH_INFO: `/v3/projects/a63c9d35dcc44ddca8ac9e16673649dd/users/eeb17d2ed61f4c1b9eec83b63b0bd67e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.630188 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4277/8530] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:38.633737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ff2f4-bbee-4363-84eb-c0d0994575d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63c9d35dcc44ddca8ac9e16673649dd, user_id=eeb17d2ed61f4c1b9eec83b63b0bd67e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.634442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.636596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ff2f4-bbee-4363-84eb-c0d0994575d0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.645513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb821c-51f4-4e44-b44e-ea71c85020fb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.650880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.651495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.651773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.652045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.652718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.653304 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4254/8531] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:38 2022] PUT /identity/v3/projects/a63c9d35dcc44ddca8ac9e16673649dd/users/eeb17d2ed61f4c1b9eec83b63b0bd67e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:38.654416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.664963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc153fd-a483-4aa8-bc7c-9156c73051e5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.665937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc153fd-a483-4aa8-bc7c-9156c73051e5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.666227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc153fd-a483-4aa8-bc7c-9156c73051e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.673775 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4278/8532] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:38.674712 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.685088 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43649ee-9f6c-446c-8d76-77fa369fe178 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.695911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-cdc153fd-a483-4aa8-bc7c-9156c73051e5 None None] MFA Rules not processed for user `eeb17d2ed61f4c1b9eec83b63b0bd67e`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:38.697240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cdc153fd-a483-4aa8-bc7c-9156c73051e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.747082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.747756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.748031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.748303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] PATH_INFO: `/v3/projects/44076904c7134d42a28d8e2ce6abd798/users/214d970fc4834b20b40308168d5f6e17/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.759217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:38.764970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44076904c7134d42a28d8e2ce6abd798, user_id=214d970fc4834b20b40308168d5f6e17, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.767711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.778995 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4255/8533] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:38.783113 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4279/8534] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:38 2022] PUT /identity/v3/projects/44076904c7134d42a28d8e2ce6abd798/users/214d970fc4834b20b40308168d5f6e17/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:38.787584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc153fd-a483-4aa8-bc7c-9156c73051e5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.790843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.792683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:38.799017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc153fd-a483-4aa8-bc7c-9156c73051e5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.805983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:38.862196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af67f4-451d-483b-be63-2e4ab0ab626d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.864172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.865748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.866028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.866320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.866870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.867094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.867476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.867755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.868029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.868567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.882174 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4256/8535] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:38.883192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:38.893152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2c939-4dab-459b-8421-2531523d075e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:38.898060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0776a4-4482-4ef0-bdd0-6afebd767770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:38.898736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0776a4-4482-4ef0-bdd0-6afebd767770 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.899010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0776a4-4482-4ef0-bdd0-6afebd767770 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.899278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0776a4-4482-4ef0-bdd0-6afebd767770 admin admin] PATH_INFO: `/v3/projects/44076904c7134d42a28d8e2ce6abd798/users/214d970fc4834b20b40308168d5f6e17/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.901355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0776a4-4482-4ef0-bdd0-6afebd767770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44076904c7134d42a28d8e2ce6abd798, user_id=214d970fc4834b20b40308168d5f6e17, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.903904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0776a4-4482-4ef0-bdd0-6afebd767770 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.915607 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4257/8536] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:38 2022] PUT /identity/v3/projects/44076904c7134d42a28d8e2ce6abd798/users/214d970fc4834b20b40308168d5f6e17/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:38.925773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29d76d-c91d-43ba-8f22-da27f353aae7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:38.926054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29d76d-c91d-43ba-8f22-da27f353aae7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:38.926327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29d76d-c91d-43ba-8f22-da27f353aae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:38.926466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:38.927918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:38.937445 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4280/8537] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:37:38.956196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ed29d76d-c91d-43ba-8f22-da27f353aae7 None None] MFA Rules not processed for user `214d970fc4834b20b40308168d5f6e17`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:38.957526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ed29d76d-c91d-43ba-8f22-da27f353aae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:39.037919 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4258/8538] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:39.051401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:39.053250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:39.069893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57027f0-a90b-4cc5-b9d9-018601e9d893 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:39.125326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:39.125853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:39.126129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:39.126403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:39.136477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:39.137956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:39.150591 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4281/8539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:39.504927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29d76d-c91d-43ba-8f22-da27f353aae7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:39.514751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29d76d-c91d-43ba-8f22-da27f353aae7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:39.519989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '214d970fc4834b20b40308168d5f6e17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44076904c7134d42a28d8e2ce6abd798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:39.520608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:39.520972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:39.521249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:39.521936 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:39.522245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:39.523870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:39.536194 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4259/8540] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:39.541054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:39.550714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df682087-9101-4878-8dbe-02525d74e71d service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:39.555775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '214d970fc4834b20b40308168d5f6e17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44076904c7134d42a28d8e2ce6abd798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:39.556374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:39.556678 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:39.556959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:39.557587 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:39.559502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:39.592048 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4282/8541] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:39 2022] POST /identity/v3/projects => generated 353 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:39.596695 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:39.608540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf9df1-df89-490e-9614-c7a175a2c1ae tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:39.659014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:39.676530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '214d970fc4834b20b40308168d5f6e17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44076904c7134d42a28d8e2ce6abd798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:39.677383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:39.677507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:39.677760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:39.678443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:39.678747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:39.680380 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:39.692993 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4260/8542] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:39.731838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:39.742882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b99bc-0da2-4c72-a9cc-979742ccdf66 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:39.747665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '214d970fc4834b20b40308168d5f6e17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44076904c7134d42a28d8e2ce6abd798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:39.748294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:39.748571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:39.748874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] PATH_INFO: `/v3/projects/cfb4285dcc864d9283a58100e1051608` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:39.750034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cfb4285dcc864d9283a58100e1051608)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:39.751920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:39.794539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:39.803272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb4285dcc864d9283a58100e1051608', 'request_id': 'req-b44c3d12-87bd-4d31-a521-ee5b53a0af69'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:39.812259 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb4285dcc864d9283a58100e1051608', 'request_id': 'req-b44c3d12-87bd-4d31-a521-ee5b53a0af69'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:39.818531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb4285dcc864d9283a58100e1051608', 'request_id': 'req-b44c3d12-87bd-4d31-a521-ee5b53a0af69'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:39.831125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] The token cache is being invalidate because project cfb4285dcc864d9283a58100e1051608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:39.831421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:39.832382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b44c3d12-87bd-4d31-a521-ee5b53a0af69 tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:39.834337 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4283/8543] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:39 2022] DELETE /identity/v3/projects/cfb4285dcc864d9283a58100e1051608 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:40.041249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:40.042971 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:40.110510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b117cd2-be48-4c0c-a192-84ca1ea350cb tempest-VolumeQuotaClassesTest-1725809915 tempest-VolumeQuotaClassesTest-1725809915-project-admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:40.171709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:40.172372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:40.172649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:40.172955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] PATH_INFO: `/v3/users/79a1c44c751146c0a866c3478c969155` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:40.174150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a1c44c751146c0a866c3478c969155)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:40.176079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:40.239589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a1c44c751146c0a866c3478c969155'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.240279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a1c44c751146c0a866c3478c969155'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.247809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a1c44c751146c0a866c3478c969155'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.256428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a1c44c751146c0a866c3478c969155'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.268909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a1c44c751146c0a866c3478c969155'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.276132 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4261/8544] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:40 2022] DELETE /identity/v3/users/79a1c44c751146c0a866c3478c969155 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:40.281350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:40.281711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:40.281958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:40.302205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:40.303410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:40.363920 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4284/8545] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:40.374857 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:40.376469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:40.384142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:40.445940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7253a8-5967-474f-a6e1-ee5202f32d65 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:40.450211 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:40.450836 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:40.451123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:40.451526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:40.462077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:40.463517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:40.474022 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4262/8546] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:40.757762 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:40.759613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:40.772309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fd351-32bc-4f88-ad95-9faddf71a40f None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:40.778005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:40.778690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:40.778990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:40.779257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] PATH_INFO: `/v3/projects/96e746f920e0493890b54622c6a442d0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:40.780488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e746f920e0493890b54622c6a442d0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:40.782463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:40.834631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e746f920e0493890b54622c6a442d0', 'request_id': 'req-1f1e265f-4185-49b3-9934-0826f2a588eb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.846942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e746f920e0493890b54622c6a442d0', 'request_id': 'req-1f1e265f-4185-49b3-9934-0826f2a588eb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.856326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e746f920e0493890b54622c6a442d0', 'request_id': 'req-1f1e265f-4185-49b3-9934-0826f2a588eb'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.869960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] The token cache is being invalidate because project 96e746f920e0493890b54622c6a442d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:40.870247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.870908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:40.872426 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4285/8547] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:40 2022] DELETE /identity/v3/projects/96e746f920e0493890b54622c6a442d0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:40.877188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:40.890004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2d63-8fa2-48e3-bee5-5180887835e3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:40.954019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:40.954653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:40.954926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:40.955197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] PATH_INFO: `/v3/users/214d970fc4834b20b40308168d5f6e17` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:40.956414 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=214d970fc4834b20b40308168d5f6e17)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:40.958468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:41.019190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '214d970fc4834b20b40308168d5f6e17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.020093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '214d970fc4834b20b40308168d5f6e17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.028047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '214d970fc4834b20b40308168d5f6e17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.046008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '214d970fc4834b20b40308168d5f6e17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.062558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-687e11a3-0d5c-40d4-b06e-9202927acc7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '214d970fc4834b20b40308168d5f6e17'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.069828 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4263/8548] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:40 2022] DELETE /identity/v3/users/214d970fc4834b20b40308168d5f6e17 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:41.362174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:41.363970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:41.383714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e265f-4185-49b3-9934-0826f2a588eb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:41.436077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:41.450103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:41.451263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.451572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.451939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] PATH_INFO: `/v3/projects/44076904c7134d42a28d8e2ce6abd798` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.453767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44076904c7134d42a28d8e2ce6abd798)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:41.456924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:41.511073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44076904c7134d42a28d8e2ce6abd798', 'request_id': 'req-df76c478-e09e-4e25-b1b3-152b9efec938'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.521156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44076904c7134d42a28d8e2ce6abd798', 'request_id': 'req-df76c478-e09e-4e25-b1b3-152b9efec938'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.527495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44076904c7134d42a28d8e2ce6abd798', 'request_id': 'req-df76c478-e09e-4e25-b1b3-152b9efec938'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.540757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] The token cache is being invalidate because project 44076904c7134d42a28d8e2ce6abd798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:41.541060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.541822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df76c478-e09e-4e25-b1b3-152b9efec938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:41.543360 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4286/8549] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:41 2022] DELETE /identity/v3/projects/44076904c7134d42a28d8e2ce6abd798 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:41.553570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a866d5-6de8-44b2-9a8f-237b29bd0fa8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.553862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a866d5-6de8-44b2-9a8f-237b29bd0fa8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.554147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a866d5-6de8-44b2-9a8f-237b29bd0fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.575516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d1a866d5-6de8-44b2-9a8f-237b29bd0fa8 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:41.576791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d1a866d5-6de8-44b2-9a8f-237b29bd0fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:41.598620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:41.649876 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4264/8550] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:41.658161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d8b81-51fe-4de1-82c3-270869689b84 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.658439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d8b81-51fe-4de1-82c3-270869689b84 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.658710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d8b81-51fe-4de1-82c3-270869689b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.680181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-545d8b81-51fe-4de1-82c3-270869689b84 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:41.681425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-545d8b81-51fe-4de1-82c3-270869689b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:41.692879 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4287/8551] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:41.694330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a866d5-6de8-44b2-9a8f-237b29bd0fa8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:41.704886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a866d5-6de8-44b2-9a8f-237b29bd0fa8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:41.711316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:41.712110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.712309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.712547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.713220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:41.713572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:41.715409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:41.731392 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4265/8552] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:41.736138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d8b81-51fe-4de1-82c3-270869689b84 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:41.746394 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d8b81-51fe-4de1-82c3-270869689b84 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:41.755713 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:41.756446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.756758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.757615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.757744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:41.759571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:41.789812 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4288/8553] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:41 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:41.794075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:41.805399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07585fcd-d3ef-45d4-9705-4858051fd43a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:41.862163 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:41.862778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.863083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.863331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.863990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:41.865985 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:41.890208 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4266/8554] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:41 2022] POST /identity/v3/users => generated 305 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:41.894339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:41.905514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb2806-6e21-4255-9478-216d29eb7c03 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:41.910704 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:41.912706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.913172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.913781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.914934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:41.916598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:41.929009 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4289/8555] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:41.933113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:41.943977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28325a02-1e7a-45a5-9a15-0946069eb3e6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:41.948930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7befa-8a39-4531-9c1a-c3f790ac8a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:41.949608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7befa-8a39-4531-9c1a-c3f790ac8a80 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.949938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7befa-8a39-4531-9c1a-c3f790ac8a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.950261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7befa-8a39-4531-9c1a-c3f790ac8a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:41.950992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7befa-8a39-4531-9c1a-c3f790ac8a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:41.952686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7befa-8a39-4531-9c1a-c3f790ac8a80 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:41.967375 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4267/8556] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:41.971630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:41.983293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1df2f-818d-4ea6-83b9-4c3bcb652e06 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:41.988756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:41.990209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:41.990478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:41.990733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] PATH_INFO: `/v3/projects/7a6eaa5db43e4b1fb9a22a0f856d955d/users/ff151cf6ba214570a63235c66c5c6625/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.006491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a6eaa5db43e4b1fb9a22a0f856d955d, user_id=ff151cf6ba214570a63235c66c5c6625, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.009182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.024770 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4290/8557] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:41 2022] PUT /identity/v3/projects/7a6eaa5db43e4b1fb9a22a0f856d955d/users/ff151cf6ba214570a63235c66c5c6625/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:42.037332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a5c2-e9ad-4b9d-9045-ef8579403a5a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.037639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a5c2-e9ad-4b9d-9045-ef8579403a5a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.037913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4a5c2-e9ad-4b9d-9045-ef8579403a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.067162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-84e4a5c2-e9ad-4b9d-9045-ef8579403a5a None None] MFA Rules not processed for user `ff151cf6ba214570a63235c66c5c6625`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:42.068706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-84e4a5c2-e9ad-4b9d-9045-ef8579403a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:42.135640 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4268/8558] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:42.140381 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.205555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2974874-f400-40a3-b669-0794f1f013d4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.211057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.211705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.211961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.212216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.212894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.214808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.243722 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4291/8559] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:42 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:42.247964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a5c2-e9ad-4b9d-9045-ef8579403a5a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.259309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4a5c2-e9ad-4b9d-9045-ef8579403a5a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.278173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:42.319160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.319602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.319940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.320223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.320927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.322879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.346705 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4269/8560] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:42 2022] POST /identity/v3/users => generated 311 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:42.350964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.357174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:42.365024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66457073-eca6-4d0b-98d3-6ef0cfd97097 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.371755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.372380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.372682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.372963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.373609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.375228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.386447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4292/8561] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:42.390893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.402550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e002966-af72-477a-9405-8d79d790f62a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.407663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.408451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.408772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.409076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] PATH_INFO: `/v3/projects/9e835b8485fa496e9b28f15c1d12b948/users/2943ef2f987f4022b4cc9282ad6d983d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.424332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e835b8485fa496e9b28f15c1d12b948, user_id=2943ef2f987f4022b4cc9282ad6d983d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.427245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.441140 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4270/8562] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:42 2022] PUT /identity/v3/projects/9e835b8485fa496e9b28f15c1d12b948/users/2943ef2f987f4022b4cc9282ad6d983d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:42.445638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.462386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2879c5d-8f6b-43bb-b558-e83c43c89902 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.522065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e1b2d-f724-4f95-be69-071dbfe29e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.522666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e1b2d-f724-4f95-be69-071dbfe29e13 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.522939 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e1b2d-f724-4f95-be69-071dbfe29e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.523205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e1b2d-f724-4f95-be69-071dbfe29e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.523862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e1b2d-f724-4f95-be69-071dbfe29e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.525497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e1b2d-f724-4f95-be69-071dbfe29e13 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.535150 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4293/8563] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:42.539774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.550890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6330b-63f6-4eaf-94aa-624008459fe3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.557004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.557690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.557968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.558244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] PATH_INFO: `/v3/projects/9e835b8485fa496e9b28f15c1d12b948/users/2943ef2f987f4022b4cc9282ad6d983d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.560509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e835b8485fa496e9b28f15c1d12b948, user_id=2943ef2f987f4022b4cc9282ad6d983d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.563224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.574869 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4271/8564] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:42 2022] PUT /identity/v3/projects/9e835b8485fa496e9b28f15c1d12b948/users/2943ef2f987f4022b4cc9282ad6d983d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:42.585361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.585643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.585920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.616013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] MFA Rules not processed for user `2943ef2f987f4022b4cc9282ad6d983d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:42.617268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:42.687476 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4294/8565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:42.699292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.701239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:42.770535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa226440-34bc-4cc6-9211-dc2844a16680 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.774635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.775270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.775547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.775811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:42.785841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:42.787756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:42.798130 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4272/8566] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:42 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:42.966788 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:42.968771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:42.984999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e1765-b159-47de-b1c8-f5d6019d89b6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:42.988921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442650ba-74ae-4934-8be6-36b9a7f4249b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:42.990227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442650ba-74ae-4934-8be6-36b9a7f4249b service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:42.990537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442650ba-74ae-4934-8be6-36b9a7f4249b service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:42.990807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442650ba-74ae-4934-8be6-36b9a7f4249b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:43.049995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442650ba-74ae-4934-8be6-36b9a7f4249b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:43.051509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442650ba-74ae-4934-8be6-36b9a7f4249b service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:43.062518 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4295/8567] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:42 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:48.473137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:48.537156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149044d-919b-4029-bcdf-79a7763892e7 service cinder] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:48.542453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:48.543110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:48.543383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:48.543651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] PATH_INFO: `/v3/users/ff151cf6ba214570a63235c66c5c6625` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:48.544811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff151cf6ba214570a63235c66c5c6625)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:48.546721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:48.569093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:48.597820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff151cf6ba214570a63235c66c5c6625'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:48.598477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff151cf6ba214570a63235c66c5c6625'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:48.606057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff151cf6ba214570a63235c66c5c6625'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:48.616882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff151cf6ba214570a63235c66c5c6625'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:48.629738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff151cf6ba214570a63235c66c5c6625'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:48.635917 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4273/8568] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:48 2022] DELETE /identity/v3/users/ff151cf6ba214570a63235c66c5c6625 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:48.640446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:48.640956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:48.641062 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:48.660787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:48.662049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:48.667432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:48.726990 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4296/8569] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:48.737269 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:48.738834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:48.804693 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e2625-0471-471f-8090-1efcf8446e86 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:48.809741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:48.810360 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:48.810676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:48.810957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:48.822249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:48.823701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:48.833946 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4274/8570] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:49.141106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:49.142820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.146842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:49.148285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.157676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c159bc-8f29-4780-af24-3c58e207f740 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:49.163160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0ac6-201d-4b66-ae67-66b8f79973f4 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:49.164962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:49.165576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.165853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.166121 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] PATH_INFO: `/v3/users/eeb17d2ed61f4c1b9eec83b63b0bd67e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.167351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb17d2ed61f4c1b9eec83b63b0bd67e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:49.168276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:49.168973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.169253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.169387 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:49.169685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] PATH_INFO: `/v3/projects/7a6eaa5db43e4b1fb9a22a0f856d955d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.170676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a6eaa5db43e4b1fb9a22a0f856d955d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:49.172639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:49.223070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb17d2ed61f4c1b9eec83b63b0bd67e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.223748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb17d2ed61f4c1b9eec83b63b0bd67e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.225432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a6eaa5db43e4b1fb9a22a0f856d955d', 'request_id': 'req-0a245c15-61b9-433f-a288-5bdc7792ef8b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.232568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a6eaa5db43e4b1fb9a22a0f856d955d', 'request_id': 'req-0a245c15-61b9-433f-a288-5bdc7792ef8b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.237569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb17d2ed61f4c1b9eec83b63b0bd67e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.241485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a6eaa5db43e4b1fb9a22a0f856d955d', 'request_id': 'req-0a245c15-61b9-433f-a288-5bdc7792ef8b'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.246177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb17d2ed61f4c1b9eec83b63b0bd67e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.247114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-bd8f1e3c-3749-4c0d-87cc-451e035fe084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb17d2ed61f4c1b9eec83b63b0bd67e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.252926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] The token cache is being invalidate because project 7a6eaa5db43e4b1fb9a22a0f856d955d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:49.253220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.253876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.255429 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4275/8571] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:49 2022] DELETE /identity/v3/projects/7a6eaa5db43e4b1fb9a22a0f856d955d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:49.261233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:49.262743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.269110 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4297/8572] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:49 2022] DELETE /identity/v3/users/eeb17d2ed61f4c1b9eec83b63b0bd67e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:49.273745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.274029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.274301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.288827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:49.297106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:49.298454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.328953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245c15-61b9-433f-a288-5bdc7792ef8b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:49.360481 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4298/8573] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:49.370522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:49.371861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.387057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:49.387916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.388190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.388461 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] PATH_INFO: `/v3/users/2943ef2f987f4022b4cc9282ad6d983d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.389640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2943ef2f987f4022b4cc9282ad6d983d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:49.391629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:49.399324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:49.437979 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67617e5-4a72-4af6-bfa4-b6bbcffb1022 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:49.441744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:49.442689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.442817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2943ef2f987f4022b4cc9282ad6d983d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.442959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.443231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.443558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2943ef2f987f4022b4cc9282ad6d983d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.445298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.451507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2943ef2f987f4022b4cc9282ad6d983d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.459325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:49.460124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2943ef2f987f4022b4cc9282ad6d983d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.460927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:49.472293 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4299/8574] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:37:49.472467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2943ef2f987f4022b4cc9282ad6d983d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.478813 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4276/8575] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:49 2022] DELETE /identity/v3/users/2943ef2f987f4022b4cc9282ad6d983d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:49.756694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:49.758421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.771775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41098-a700-47e2-8cbb-6d9f0aea0806 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:49.771948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:49.773647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.834542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:49.839634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:49.840307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.840626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.840959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] PATH_INFO: `/v3/projects/a63c9d35dcc44ddca8ac9e16673649dd` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.842199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63c9d35dcc44ddca8ac9e16673649dd)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:49.844195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:49.854779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4460e0-143a-4da4-a61e-5e126cb3cfe3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:49.859657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:49.860368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.860756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.861094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] PATH_INFO: `/v3/projects/9e835b8485fa496e9b28f15c1d12b948` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.862337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e835b8485fa496e9b28f15c1d12b948)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:49.864392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:49.894687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63c9d35dcc44ddca8ac9e16673649dd', 'request_id': 'req-54eb59db-39cf-4de8-98ef-90395bcece97'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.902690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63c9d35dcc44ddca8ac9e16673649dd', 'request_id': 'req-54eb59db-39cf-4de8-98ef-90395bcece97'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.908626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63c9d35dcc44ddca8ac9e16673649dd', 'request_id': 'req-54eb59db-39cf-4de8-98ef-90395bcece97'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.916746 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e835b8485fa496e9b28f15c1d12b948', 'request_id': 'req-06ece9ea-2e44-4d81-9fa5-646482afc37c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.920160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] The token cache is being invalidate because project a63c9d35dcc44ddca8ac9e16673649dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:49.920451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.921154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-54eb59db-39cf-4de8-98ef-90395bcece97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.922664 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4300/8576] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:49 2022] DELETE /identity/v3/projects/a63c9d35dcc44ddca8ac9e16673649dd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:49.923290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e835b8485fa496e9b28f15c1d12b948', 'request_id': 'req-06ece9ea-2e44-4d81-9fa5-646482afc37c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.927440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd9d3c-ea90-4ed2-85db-5d3791438aef None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.927781 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd9d3c-ea90-4ed2-85db-5d3791438aef None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.927978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd9d3c-ea90-4ed2-85db-5d3791438aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.932840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e835b8485fa496e9b28f15c1d12b948', 'request_id': 'req-06ece9ea-2e44-4d81-9fa5-646482afc37c'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.943991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] The token cache is being invalidate because project 9e835b8485fa496e9b28f15c1d12b948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:49.944280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.945074 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-06ece9ea-2e44-4d81-9fa5-646482afc37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:49.950131 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4277/8577] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:49 2022] DELETE /identity/v3/projects/9e835b8485fa496e9b28f15c1d12b948 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:49.950391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5ccd9d3c-ea90-4ed2-85db-5d3791438aef None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:49.951608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117589c-db97-457f-9937-e3c9dc5ac4aa None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:49.951738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5ccd9d3c-ea90-4ed2-85db-5d3791438aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:49.952008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117589c-db97-457f-9937-e3c9dc5ac4aa None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:49.952325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117589c-db97-457f-9937-e3c9dc5ac4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:49.973502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-f117589c-db97-457f-9937-e3c9dc5ac4aa None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:49.974747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f117589c-db97-457f-9937-e3c9dc5ac4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:50.006058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:50.021057 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4301/8578] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:50.025445 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a75d7-34b1-48a9-b091-ee61c2c2c57e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.025719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a75d7-34b1-48a9-b091-ee61c2c2c57e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.025992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a75d7-34b1-48a9-b091-ee61c2c2c57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.042662 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4278/8579] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:50.046246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5f8a75d7-34b1-48a9-b091-ee61c2c2c57e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:50.047816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5f8a75d7-34b1-48a9-b091-ee61c2c2c57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:50.051770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4630bd-03e6-4646-9a85-211391d3e37d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.052053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4630bd-03e6-4646-9a85-211391d3e37d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.052328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4630bd-03e6-4646-9a85-211391d3e37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.055826 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4302/8580] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:50.059841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a75d7-34b1-48a9-b091-ee61c2c2c57e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.069464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a75d7-34b1-48a9-b091-ee61c2c2c57e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.073487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ec4630bd-03e6-4646-9a85-211391d3e37d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:50.074375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.074692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ec4630bd-03e6-4646-9a85-211391d3e37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:50.074990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.075260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.075691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.076469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:50.076816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.078516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.083063 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4279/8581] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:50.087240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4630bd-03e6-4646-9a85-211391d3e37d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.090781 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4303/8582] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:37:50.095205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.096650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4630bd-03e6-4646-9a85-211391d3e37d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.101715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.102317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.102596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.102867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.103511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:50.103813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.105120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d9208-f638-4cba-8e95-d5fd284462ee admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.105778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.109968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.110558 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.110857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.111127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.111754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.113714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.118800 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4280/8583] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:37:50.123097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.133101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50dd15-cd8d-4cd3-a6a5-fbd244291d13 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.138690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.139292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.139572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.139845 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.140509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.142635 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.151959 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4304/8584] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:50.153024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.182383 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4281/8585] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:50.184554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.216578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb866f2-907d-4eef-aebb-817db9ede970 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.225572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:50.249437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7120f6-af0d-4e5f-8384-c07021b86b32 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.254647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.255342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.255684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.255958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.256665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.258863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.273594 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.274215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.274520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.274798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.275448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.277406 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.285340 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4282/8586] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:50.286351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.296615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388524f-782d-461d-97bf-2cbce57075d0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.300520 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4305/8587] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:50.301381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.301981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.302249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.302516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.303151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.304651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.304849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.315152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28868b6e-8715-415d-9c11-4177340b0b0f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.318969 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4283/8588] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:50.319806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.320593 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.321230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.321567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.321859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.322506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.324157 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.329978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e1b1b-5aaf-49a6-87b7-b26f27feb22b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.334636 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4306/8589] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:50.334773 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.335331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.335699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.336072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.336843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.338515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.338628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.348513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f57b40-b61e-4a94-90b7-624bd2011533 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.353000 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4284/8590] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:37:50.353760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.353882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.354361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.354673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.354948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.355583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.357229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.363474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304e9f4-6f8d-4a26-a2b6-bafebb285427 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.371433 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4307/8591] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:50.371571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db302d5-1203-4320-abf1-6c4d98a0e4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.371571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db302d5-1203-4320-abf1-6c4d98a0e4d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.371571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db302d5-1203-4320-abf1-6c4d98a0e4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.371877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db302d5-1203-4320-abf1-6c4d98a0e4d6 admin admin] PATH_INFO: `/v3/projects/271c7ee9fcde4daaab81c659025b4113/users/fd9947f6cec746078c6efa0f7c760d92/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.372391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.382666 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73b51b-60d7-41f9-beac-8f0e0f247b5f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.386905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db302d5-1203-4320-abf1-6c4d98a0e4d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271c7ee9fcde4daaab81c659025b4113, user_id=fd9947f6cec746078c6efa0f7c760d92, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.387572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80422c87-28a9-46de-a026-66f74e6fd7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.388312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80422c87-28a9-46de-a026-66f74e6fd7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.388585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80422c87-28a9-46de-a026-66f74e6fd7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.388880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80422c87-28a9-46de-a026-66f74e6fd7f8 admin admin] PATH_INFO: `/v3/projects/dcb31ee64b334635bd6fcc7b8632faca/users/c3524dd067b54dc69691738921a83bfc/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.389628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db302d5-1203-4320-abf1-6c4d98a0e4d6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.403882 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4285/8592] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:50 2022] PUT /identity/v3/projects/271c7ee9fcde4daaab81c659025b4113/users/fd9947f6cec746078c6efa0f7c760d92/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:50.406841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80422c87-28a9-46de-a026-66f74e6fd7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb31ee64b334635bd6fcc7b8632faca, user_id=c3524dd067b54dc69691738921a83bfc, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.409680 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80422c87-28a9-46de-a026-66f74e6fd7f8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.414343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c9896-7274-434a-8c20-2fbdd675356f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.414742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c9896-7274-434a-8c20-2fbdd675356f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.414934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c9896-7274-434a-8c20-2fbdd675356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.425756 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4308/8593] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:50 2022] PUT /identity/v3/projects/dcb31ee64b334635bd6fcc7b8632faca/users/c3524dd067b54dc69691738921a83bfc/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:50.437493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.437759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.438051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.445856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-686c9896-7274-434a-8c20-2fbdd675356f None None] MFA Rules not processed for user `fd9947f6cec746078c6efa0f7c760d92`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:50.447252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-686c9896-7274-434a-8c20-2fbdd675356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:50.452027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:50.468818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] MFA Rules not processed for user `c3524dd067b54dc69691738921a83bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:50.470096 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:50.514103 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4286/8594] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:50.518683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c9896-7274-434a-8c20-2fbdd675356f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.535447 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4309/8595] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:50.548056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.549488 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:50.582623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c9896-7274-434a-8c20-2fbdd675356f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.587396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.588017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.588298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.588573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.589236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.591149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.607658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:50.617282 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd7ccc-294b-483c-b9df-8f8ef3e7c3c9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.621446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.622083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.622356 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.622612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.625666 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4287/8596] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:50.626543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.633320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.634881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.644565 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4310/8597] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:37:50.692783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19b30-c467-4243-b1f4-d3f0ae550a59 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.698206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.698818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.699092 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.699358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.699982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.701918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.726889 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4288/8598] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:50.730536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.742321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d215f-2026-4738-9d8d-e978b30dfc8f service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.747284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.747898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.748203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.748478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.749194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.750878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.761824 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4311/8599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:50.767579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.778452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa5252-4147-4683-8cc2-93cf770fd148 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.783681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.784573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.784973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.785336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] PATH_INFO: `/v3/projects/7971fd3999034a31a6465af6900131f2/users/1cce337ee8344b62bbea575f0325c253/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.801309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7971fd3999034a31a6465af6900131f2, user_id=1cce337ee8344b62bbea575f0325c253, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.803978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.819051 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4289/8600] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:50 2022] PUT /identity/v3/projects/7971fd3999034a31a6465af6900131f2/users/1cce337ee8344b62bbea575f0325c253/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:50.823218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.833399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf550ca8-0c47-4d96-91ce-4fb833c4b5df admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.892911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6866b-1b6f-4501-880e-2d93daf07e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.893592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6866b-1b6f-4501-880e-2d93daf07e63 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.893874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6866b-1b6f-4501-880e-2d93daf07e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.894145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6866b-1b6f-4501-880e-2d93daf07e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.894782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6866b-1b6f-4501-880e-2d93daf07e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.897133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6866b-1b6f-4501-880e-2d93daf07e63 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.907695 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4312/8601] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:50.912366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:50.924098 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac34f8-73ac-4865-bd73-8bde070e78bb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:50.929021 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:50.930259 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.930539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.930810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] PATH_INFO: `/v3/projects/7971fd3999034a31a6465af6900131f2/users/1cce337ee8344b62bbea575f0325c253/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.932915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7971fd3999034a31a6465af6900131f2, user_id=1cce337ee8344b62bbea575f0325c253, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:50.935966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:50.948330 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4290/8602] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:50 2022] PUT /identity/v3/projects/7971fd3999034a31a6465af6900131f2/users/1cce337ee8344b62bbea575f0325c253/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:50.958750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac7064-3b33-4e6d-8983-e9f3bd7892b9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:50.959048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac7064-3b33-4e6d-8983-e9f3bd7892b9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:50.959322 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac7064-3b33-4e6d-8983-e9f3bd7892b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:50.988323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-42ac7064-3b33-4e6d-8983-e9f3bd7892b9 None None] MFA Rules not processed for user `1cce337ee8344b62bbea575f0325c253`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:50.989688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-42ac7064-3b33-4e6d-8983-e9f3bd7892b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:51.002982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:51.091889 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4313/8603] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:51.112253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:51.114139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:51.185554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceba6d7-dc8d-46c8-98d5-ff2bfe230ed0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:51.189231 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88210e4-d657-45a9-84a9-0f69934ebae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:51.189850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88210e4-d657-45a9-84a9-0f69934ebae1 service cinder] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:51.190133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88210e4-d657-45a9-84a9-0f69934ebae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:51.190440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88210e4-d657-45a9-84a9-0f69934ebae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:51.200493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88210e4-d657-45a9-84a9-0f69934ebae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:51.201990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88210e4-d657-45a9-84a9-0f69934ebae1 service cinder] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:51.211860 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4291/8604] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:51.669995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac7064-3b33-4e6d-8983-e9f3bd7892b9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:51.746205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac7064-3b33-4e6d-8983-e9f3bd7892b9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:51.759085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:51.760040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:51.760457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:51.760919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] PATH_INFO: `/v3/users/fd9947f6cec746078c6efa0f7c760d92` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:51.762896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9947f6cec746078c6efa0f7c760d92)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:51.766347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:51.837014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9947f6cec746078c6efa0f7c760d92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:51.837833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9947f6cec746078c6efa0f7c760d92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:51.852011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9947f6cec746078c6efa0f7c760d92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:51.862377 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9947f6cec746078c6efa0f7c760d92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:51.866222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9947f6cec746078c6efa0f7c760d92'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:51.886883 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4314/8605] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:51 2022] DELETE /identity/v3/users/fd9947f6cec746078c6efa0f7c760d92 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:51.891710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:51.892188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:51.892276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:51.913305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:51.914528 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:51.972455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:51.990071 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4292/8606] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:52.002126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:52.003915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:52.047972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:52.088056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3531-318b-45cb-8138-a3ad5f97fb7a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:52.092291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:52.092959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:52.093237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:52.093508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:52.105771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:52.107256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:52.118284 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4315/8607] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:52.452389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:52.454595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:52.470617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ca7f8-97bd-4f3d-bf5b-fefeeb7f7e08 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:52.482130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:52.483065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:52.483389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:52.483737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] PATH_INFO: `/v3/projects/271c7ee9fcde4daaab81c659025b4113` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:52.485377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=271c7ee9fcde4daaab81c659025b4113)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:52.487884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:52.536562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271c7ee9fcde4daaab81c659025b4113', 'request_id': 'req-68b8ad1d-2697-4206-9ef1-63b49fd91702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.542912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271c7ee9fcde4daaab81c659025b4113', 'request_id': 'req-68b8ad1d-2697-4206-9ef1-63b49fd91702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.551732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271c7ee9fcde4daaab81c659025b4113', 'request_id': 'req-68b8ad1d-2697-4206-9ef1-63b49fd91702'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.564509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] The token cache is being invalidate because project 271c7ee9fcde4daaab81c659025b4113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:52.564825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.565608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.567451 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4293/8608] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:52 2022] DELETE /identity/v3/projects/271c7ee9fcde4daaab81c659025b4113 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:52.573248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:52.583325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c93f-702a-44d3-ba94-48d81a8e687c service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:52.642123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:52.642757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:52.643033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:52.643298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] PATH_INFO: `/v3/users/1cce337ee8344b62bbea575f0325c253` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:52.644409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cce337ee8344b62bbea575f0325c253)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:52.646350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:52.702403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cce337ee8344b62bbea575f0325c253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.703048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cce337ee8344b62bbea575f0325c253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.712826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cce337ee8344b62bbea575f0325c253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.721163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cce337ee8344b62bbea575f0325c253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.721861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-79ae0549-f6ce-4168-8914-bc21228d4672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cce337ee8344b62bbea575f0325c253'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:52.740550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4316/8609] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:52 2022] DELETE /identity/v3/users/1cce337ee8344b62bbea575f0325c253 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:53.021712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.023192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:53.036513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8ad1d-2697-4206-9ef1-63b49fd91702 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.094612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.095190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.095468 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.095714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] PATH_INFO: `/v3/projects/7971fd3999034a31a6465af6900131f2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.096907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7971fd3999034a31a6465af6900131f2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.098863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.149065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7971fd3999034a31a6465af6900131f2', 'request_id': 'req-046acba5-1ae9-4391-b1f8-f42cab78afb1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:53.155557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7971fd3999034a31a6465af6900131f2', 'request_id': 'req-046acba5-1ae9-4391-b1f8-f42cab78afb1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:53.164279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7971fd3999034a31a6465af6900131f2', 'request_id': 'req-046acba5-1ae9-4391-b1f8-f42cab78afb1'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:53.175175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] The token cache is being invalidate because project 7971fd3999034a31a6465af6900131f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:53.175462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:53.176118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-046acba5-1ae9-4391-b1f8-f42cab78afb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:53.177672 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4294/8610] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:53 2022] DELETE /identity/v3/projects/7971fd3999034a31a6465af6900131f2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:53.182648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dda503-79b6-46d3-b61a-e54c1d5ce142 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.183120 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dda503-79b6-46d3-b61a-e54c1d5ce142 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.183250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dda503-79b6-46d3-b61a-e54c1d5ce142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.206240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e2dda503-79b6-46d3-b61a-e54c1d5ce142 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:53.208018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e2dda503-79b6-46d3-b61a-e54c1d5ce142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:53.262814 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:53.283037 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4317/8611] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:53.294078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc7c1-c2ad-4cd0-ad9e-6c7e7c33e0f3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.294373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc7c1-c2ad-4cd0-ad9e-6c7e7c33e0f3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.294650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc7c1-c2ad-4cd0-ad9e-6c7e7c33e0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.303875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:53.316579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1c1fc7c1-c2ad-4cd0-ad9e-6c7e7c33e0f3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:53.318953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1c1fc7c1-c2ad-4cd0-ad9e-6c7e7c33e0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:53.328065 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4295/8612] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:53.332637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dda503-79b6-46d3-b61a-e54c1d5ce142 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.343018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dda503-79b6-46d3-b61a-e54c1d5ce142 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.347904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.348429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.348679 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.348991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.349637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:37:53.349931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.351524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.363477 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4318/8613] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:37:53.369487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fc7c1-c2ad-4cd0-ad9e-6c7e7c33e0f3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.380484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fc7c1-c2ad-4cd0-ad9e-6c7e7c33e0f3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.387182 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.387959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.388109 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.388335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.389014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.390921 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.422035 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4296/8614] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:53 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:37:53.426542 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.437675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76f2d-95fc-407c-8a9c-09105773dd81 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.504805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.505494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.505812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.506124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.506916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.509116 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.535828 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4319/8615] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:53 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:53.540549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.552218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fec563-651c-421a-ba19-a065629fd4fc admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.559281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.559943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.560261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.560605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.561338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.562974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.575674 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4297/8616] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:53.579798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.590072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300dec2-3a57-410d-9567-319684ea474a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.594733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35617a8-2330-4d92-929f-edc3dee6c1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.595323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35617a8-2330-4d92-929f-edc3dee6c1bc admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.595595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35617a8-2330-4d92-929f-edc3dee6c1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.595862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35617a8-2330-4d92-929f-edc3dee6c1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.596492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35617a8-2330-4d92-929f-edc3dee6c1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.598172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35617a8-2330-4d92-929f-edc3dee6c1bc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.610058 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4320/8617] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:53.614356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.625793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdff027-65e8-4bd1-b056-5d2f2286c9e5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.631465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.632149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.632826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.633020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] PATH_INFO: `/v3/projects/60f3e5639c294a53899e29af88c9c38b/users/88dbdf0286c34c179364a772529dbfe8/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.651185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f3e5639c294a53899e29af88c9c38b, user_id=88dbdf0286c34c179364a772529dbfe8, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.653925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.670567 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4298/8618] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:53 2022] PUT /identity/v3/projects/60f3e5639c294a53899e29af88c9c38b/users/88dbdf0286c34c179364a772529dbfe8/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:53.676131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0e76-11b6-4801-9e05-fd1f0990a671 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.676416 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0e76-11b6-4801-9e05-fd1f0990a671 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.676683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0e76-11b6-4801-9e05-fd1f0990a671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.696991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1cca0e76-11b6-4801-9e05-fd1f0990a671 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:53.698201 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1cca0e76-11b6-4801-9e05-fd1f0990a671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:53.768190 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4321/8619] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:53.778286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:53.780072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:53.841442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d08f6f-15f7-4c1f-b6d0-de1e8076917c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:53.845389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:53.846014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:53.846290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:53.846563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:53.857369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:53.858751 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:53.869265 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4299/8620] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:54.541772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d95ad3-2546-443a-8a08-880c4a8f7b41 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:54.542328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d95ad3-2546-443a-8a08-880c4a8f7b41 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:54.542328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d95ad3-2546-443a-8a08-880c4a8f7b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:54.591969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f3d95ad3-2546-443a-8a08-880c4a8f7b41 None None] MFA Rules not processed for user `88dbdf0286c34c179364a772529dbfe8`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:54.593244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f3d95ad3-2546-443a-8a08-880c4a8f7b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:54.652810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:54.687576 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4322/8621] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:54.692518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:54.702758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d18788-0ecb-4f69-b132-316fac383de0 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:54.707791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:54.708517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:54.708892 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:54.709238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:54.709987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:54.712144 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:54.746789 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4300/8622] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:37:54 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:54.751208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d95ad3-2546-443a-8a08-880c4a8f7b41 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:54.764494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d95ad3-2546-443a-8a08-880c4a8f7b41 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:54.822118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:54.822786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:54.823011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:54.823270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:54.823891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:54.825824 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:54.852570 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4323/8623] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:37:54 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:37:54.858336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:54.869204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c83462-3fae-4464-bb2c-d593a5cf81eb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:54.874331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:54.874942 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:54.875216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:54.875487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:54.876147 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:54.877803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:54.889506 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4301/8624] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:54.893511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:54.905305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3356-1757-4bdf-8d40-96acb17f3966 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:54.911649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:54.912308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:54.912617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:54.912923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] PATH_INFO: `/v3/projects/98efcf47bd264e9787ceec9aa2eb0bd8/users/31c8ca539bcd4e5aa205953129f07af0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:54.930263 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98efcf47bd264e9787ceec9aa2eb0bd8, user_id=31c8ca539bcd4e5aa205953129f07af0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:54.932944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:54.949777 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4324/8625] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:54 2022] PUT /identity/v3/projects/98efcf47bd264e9787ceec9aa2eb0bd8/users/31c8ca539bcd4e5aa205953129f07af0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:37:54.954227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:54.965246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a81d1-4bbf-4f9f-8ac7-0a5616986cf2 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:55.031545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccfb30-b0c4-436e-988d-6caab5b06432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:55.032307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccfb30-b0c4-436e-988d-6caab5b06432 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:55.032644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccfb30-b0c4-436e-988d-6caab5b06432 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:55.033042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccfb30-b0c4-436e-988d-6caab5b06432 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:55.033680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccfb30-b0c4-436e-988d-6caab5b06432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:55.035865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccfb30-b0c4-436e-988d-6caab5b06432 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:55.051102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4302/8626] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:37:55.056577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:55.069413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94e337-86a6-4cb8-a1ca-a7feb3266c02 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:55.078038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e4222-715d-4d57-8238-6f81af0ebd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:55.079058 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e4222-715d-4d57-8238-6f81af0ebd39 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:55.079412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e4222-715d-4d57-8238-6f81af0ebd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:55.079773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e4222-715d-4d57-8238-6f81af0ebd39 admin admin] PATH_INFO: `/v3/projects/98efcf47bd264e9787ceec9aa2eb0bd8/users/31c8ca539bcd4e5aa205953129f07af0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:55.082526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e4222-715d-4d57-8238-6f81af0ebd39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98efcf47bd264e9787ceec9aa2eb0bd8, user_id=31c8ca539bcd4e5aa205953129f07af0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:55.086656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e4222-715d-4d57-8238-6f81af0ebd39 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:55.098443 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4325/8627] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:37:55 2022] PUT /identity/v3/projects/98efcf47bd264e9787ceec9aa2eb0bd8/users/31c8ca539bcd4e5aa205953129f07af0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:55.776128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:55.776593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:55.776723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:55.796463 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:55.812868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] MFA Rules not processed for user `31c8ca539bcd4e5aa205953129f07af0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:55.817661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:55.910501 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4303/8628] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:55.922301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:55.922867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:55.923066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:55.945806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] MFA Rules not processed for user `88dbdf0286c34c179364a772529dbfe8`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:55.947311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:56.008056 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4326/8629] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:37:56.022699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:56.024788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:56.109667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502f48-cb5e-4f48-9aea-0512694f307e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:56.116173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:56.116835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:56.117172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:56.117452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:56.126851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:56.128566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:56.138389 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4304/8630] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:56.418005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:56.419522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:56.470355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:56.485418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36634d2d-6ff4-4046-8ddd-529610315d7d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:56.489208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77037ee-ca92-4315-a92d-83c8b8452418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:56.489820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77037ee-ca92-4315-a92d-83c8b8452418 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:56.490093 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77037ee-ca92-4315-a92d-83c8b8452418 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:56.490358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77037ee-ca92-4315-a92d-83c8b8452418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:56.492318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f77037ee-ca92-4315-a92d-83c8b8452418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:56.506848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77037ee-ca92-4315-a92d-83c8b8452418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:56.508851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77037ee-ca92-4315-a92d-83c8b8452418 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:56.520644 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4327/8631] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:56.781987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:56.783527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:56.857379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd768e-307e-4b44-b1cf-061b88dba6a2 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:56.862522 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:56.863149 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:56.863427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:56.863703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] PATH_INFO: `/v3/users/c3524dd067b54dc69691738921a83bfc` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:56.864864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3524dd067b54dc69691738921a83bfc)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:56.866778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:56.890551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:56.918306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3524dd067b54dc69691738921a83bfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:56.918958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3524dd067b54dc69691738921a83bfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:56.926122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3524dd067b54dc69691738921a83bfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:56.934719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3524dd067b54dc69691738921a83bfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:56.947346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3524dd067b54dc69691738921a83bfc'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:56.953497 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4305/8632] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:37:56 2022] DELETE /identity/v3/users/c3524dd067b54dc69691738921a83bfc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:56.958364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:56.958869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:56.958966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:56.978077 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:37:56.979305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:57.040091 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4328/8633] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:37:57.050089 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:57.051689 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:57.115917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d128-ebd7-43f5-8456-c9b533272b51 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:57.120347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:57.121002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:57.121283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:57.121554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:57.131571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:57.133176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:57.143640 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4306/8634] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:37:57.445143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:57.446344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:57.459585 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d03ac-4557-4bc1-b60b-3fa238548826 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:57.465476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:57.466136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:57.466411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:57.466682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] PATH_INFO: `/v3/projects/dcb31ee64b334635bd6fcc7b8632faca` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:57.467805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcb31ee64b334635bd6fcc7b8632faca)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:57.469785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:57.543626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcb31ee64b334635bd6fcc7b8632faca', 'request_id': 'req-604927b2-35e1-4ce4-b7d7-801facf03fc9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:57.553358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcb31ee64b334635bd6fcc7b8632faca', 'request_id': 'req-604927b2-35e1-4ce4-b7d7-801facf03fc9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:57.560524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcb31ee64b334635bd6fcc7b8632faca', 'request_id': 'req-604927b2-35e1-4ce4-b7d7-801facf03fc9'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:57.576969 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] The token cache is being invalidate because project dcb31ee64b334635bd6fcc7b8632faca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:37:57.577327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:57.578040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:37:57.579793 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4329/8635] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:37:57 2022] DELETE /identity/v3/projects/dcb31ee64b334635bd6fcc7b8632faca => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:37:57.880788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:37:57.882438 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:57.968950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b667-518f-4edb-8aaa-deeda90e32a8 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:37:58.020541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:58.044408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79646114-4e83-482a-a300-20829040d63c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:37:58.045171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79646114-4e83-482a-a300-20829040d63c service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:37:58.045447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79646114-4e83-482a-a300-20829040d63c service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:37:58.045725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79646114-4e83-482a-a300-20829040d63c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:37:58.047548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79646114-4e83-482a-a300-20829040d63c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:37:58.063543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79646114-4e83-482a-a300-20829040d63c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:37:58.065283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79646114-4e83-482a-a300-20829040d63c service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:37:58.080169 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4307/8636] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:37:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:00.847206 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:00.848810 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:00.912955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604927b2-35e1-4ce4-b7d7-801facf03fc9 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:00.916958 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:00.917597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:00.917875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:00.918149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:00.919984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:00.932507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:00.933974 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:00.944894 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4330/8637] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:01.695123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79646114-4e83-482a-a300-20829040d63c service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:01.697285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79646114-4e83-482a-a300-20829040d63c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:01.782859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79646114-4e83-482a-a300-20829040d63c service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:01.788324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:01.789152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:01.789476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:01.789738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:01.791444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:01.807204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:01.808834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:01.822987 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4308/8638] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:06.699237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:06.700862 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:06.723739 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb5780-ea3e-4324-a5b9-91bc7d8ae4df service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:06.730189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:06.730984 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:06.731349 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:06.731627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:06.733931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:06.750134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:06.751760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:06.760266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:06.767650 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4331/8639] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:09.105677 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:09.118769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef647-d915-413e-af3d-bd90e47379a9 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:09.124546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:09.125246 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:09.125516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:09.125806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:09.127860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:09.141737 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:09.143294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:09.155572 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4309/8640] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:11.669748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:11.671300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:11.745616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd2a1d-bd89-4ead-bce6-cf31381911d9 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:11.749940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:11.750547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:11.750796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:11.751087 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:11.752927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:11.822915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:11.824385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:11.834064 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4332/8641] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:19.148835 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=53 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:21.829737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=50 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:30.849349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:30.850923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:30.934745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601ef49-35e7-4f85-b735-38e2ee36d361 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:30.940284 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:30.940965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:30.941241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:30.941527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] PATH_INFO: `/v3/users/88dbdf0286c34c179364a772529dbfe8` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:30.942663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] RBAC: Authorizing `identity:delete_user(user_id=88dbdf0286c34c179364a772529dbfe8)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:30.944630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:30.998542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88dbdf0286c34c179364a772529dbfe8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:30.999283 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88dbdf0286c34c179364a772529dbfe8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.007218 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88dbdf0286c34c179364a772529dbfe8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.017356 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88dbdf0286c34c179364a772529dbfe8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.031362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88dbdf0286c34c179364a772529dbfe8'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.037936 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4310/8642] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:38:30 2022] DELETE /identity/v3/users/88dbdf0286c34c179364a772529dbfe8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:31.223113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:31.237266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4063c-dffc-4f84-818d-a8f5046ebd51 service cinder] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:31.302347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:31.302988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:31.303302 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:31.303523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] PATH_INFO: `/v3/projects/60f3e5639c294a53899e29af88c9c38b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:31.304676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f3e5639c294a53899e29af88c9c38b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:31.306604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:31.358352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f3e5639c294a53899e29af88c9c38b', 'request_id': 'req-f3aab474-8f58-417e-aacb-7fb706fb59a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.366691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f3e5639c294a53899e29af88c9c38b', 'request_id': 'req-f3aab474-8f58-417e-aacb-7fb706fb59a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.372498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f3e5639c294a53899e29af88c9c38b', 'request_id': 'req-f3aab474-8f58-417e-aacb-7fb706fb59a6'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.384237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] The token cache is being invalidate because project 60f3e5639c294a53899e29af88c9c38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:38:31.384508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.385318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.386956 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4333/8643] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:38:31 2022] DELETE /identity/v3/projects/60f3e5639c294a53899e29af88c9c38b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:31.391464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:31.393125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:31.408185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f65641-eb43-49b7-82f9-bfcfb8af518f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:31.471207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:31.471850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:31.472128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:31.472411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] PATH_INFO: `/v3/users/31c8ca539bcd4e5aa205953129f07af0` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:31.473587 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c8ca539bcd4e5aa205953129f07af0)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:31.475534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:31.497596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:31.525992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c8ca539bcd4e5aa205953129f07af0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.526626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c8ca539bcd4e5aa205953129f07af0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.533622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c8ca539bcd4e5aa205953129f07af0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.542523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c8ca539bcd4e5aa205953129f07af0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.555162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-c4e358f5-5bc3-4be3-8557-8e16111998d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c8ca539bcd4e5aa205953129f07af0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.561244 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4311/8644] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:38:31 2022] DELETE /identity/v3/users/31c8ca539bcd4e5aa205953129f07af0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:31.749658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:31.751164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:31.765013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aab474-8f58-417e-aacb-7fb706fb59a6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:31.824427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:31.825091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:31.825369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:31.825616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] PATH_INFO: `/v3/projects/98efcf47bd264e9787ceec9aa2eb0bd8` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:31.826802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98efcf47bd264e9787ceec9aa2eb0bd8)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:31.829135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:31.866070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:31.880168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98efcf47bd264e9787ceec9aa2eb0bd8', 'request_id': 'req-369a8ae5-e582-4d29-bf2c-c3dc444fa059'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.889233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98efcf47bd264e9787ceec9aa2eb0bd8', 'request_id': 'req-369a8ae5-e582-4d29-bf2c-c3dc444fa059'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.895040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98efcf47bd264e9787ceec9aa2eb0bd8', 'request_id': 'req-369a8ae5-e582-4d29-bf2c-c3dc444fa059'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.907247 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] The token cache is being invalidate because project 98efcf47bd264e9787ceec9aa2eb0bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:38:31.907531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.908249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-369a8ae5-e582-4d29-bf2c-c3dc444fa059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:31.909934 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4334/8645] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:38:31 2022] DELETE /identity/v3/projects/98efcf47bd264e9787ceec9aa2eb0bd8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:31.915306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c94ef4-d644-48eb-9197-dd283097e0a4 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:31.915792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c94ef4-d644-48eb-9197-dd283097e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:31.915899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c94ef4-d644-48eb-9197-dd283097e0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:31.938151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-65c94ef4-d644-48eb-9197-dd283097e0a4 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:31.939637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-65c94ef4-d644-48eb-9197-dd283097e0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:32.002110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4312/8646] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:32.011875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b02a0-9cbe-4df0-8759-ff1e42a5446d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.011875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b02a0-9cbe-4df0-8759-ff1e42a5446d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.012143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b02a0-9cbe-4df0-8759-ff1e42a5446d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.032705 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e39b02a0-9cbe-4df0-8759-ff1e42a5446d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:32.033920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e39b02a0-9cbe-4df0-8759-ff1e42a5446d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:32.041883 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4335/8647] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:32.046610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c94ef4-d644-48eb-9197-dd283097e0a4 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:32.057029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c94ef4-d644-48eb-9197-dd283097e0a4 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:32.062390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:32.063000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.063293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.063576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.064545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:38:32.064909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:32.066513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:32.079027 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4313/8648] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:38:32.083466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b02a0-9cbe-4df0-8759-ff1e42a5446d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:32.093430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b02a0-9cbe-4df0-8759-ff1e42a5446d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:32.099300 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:32.099907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.100184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.100454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.101127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:32.103031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:32.133640 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4336/8649] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:38:32 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:38:32.138001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:32.148676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9caf-3bba-4240-8dca-0be98f1250d7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:32.211154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:32.211688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.211969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.212241 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.212905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:32.214814 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:32.239386 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4314/8650] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:38:32 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:38:32.243455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:32.254719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79af16-0abd-43e0-ac41-a9835e170d35 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:32.260241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:32.260907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.261185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.261480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.262124 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:32.263745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:32.275046 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4337/8651] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:38:32.278428 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:32.289469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc4cf5-91b3-4262-a2c9-ba5b693e047e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:32.295712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c6994-953d-4b5a-a9d7-f2dc8a016679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:32.296300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6994-953d-4b5a-a9d7-f2dc8a016679 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.296600 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6994-953d-4b5a-a9d7-f2dc8a016679 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.297171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6994-953d-4b5a-a9d7-f2dc8a016679 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.297832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6994-953d-4b5a-a9d7-f2dc8a016679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:32.299495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6994-953d-4b5a-a9d7-f2dc8a016679 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:32.311179 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4315/8652] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:38:32.316173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:32.326671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cd352-5736-49f6-80ea-8e9152081ba8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:32.332017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:32.332691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.332970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.333239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] PATH_INFO: `/v3/projects/3de8116c00564abd82f93555f3f81993/users/64c73ef93e3b4d8da546f4ac18520560/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.349352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de8116c00564abd82f93555f3f81993, user_id=64c73ef93e3b4d8da546f4ac18520560, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:32.351987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:32.368043 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4338/8653] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:38:32 2022] PUT /identity/v3/projects/3de8116c00564abd82f93555f3f81993/users/64c73ef93e3b4d8da546f4ac18520560/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:32.378582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab20492-1a8b-4620-8cab-e49c005c9c3a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.378860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab20492-1a8b-4620-8cab-e49c005c9c3a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.379128 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab20492-1a8b-4620-8cab-e49c005c9c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.387760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:32.410130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2ab20492-1a8b-4620-8cab-e49c005c9c3a None None] MFA Rules not processed for user `64c73ef93e3b4d8da546f4ac18520560`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:32.411382 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2ab20492-1a8b-4620-8cab-e49c005c9c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:32.476345 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4316/8654] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:32.488392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:32.490493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:32.554174 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446f3d-d0cf-4e4e-abd2-7859a7a29837 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:32.558863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25750d-7726-404f-a21a-27cbd3d3f4c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b324af05ec3e4b35a9bfc188b3b3abec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:32.559469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25750d-7726-404f-a21a-27cbd3d3f4c5 service cinder] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:32.559745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25750d-7726-404f-a21a-27cbd3d3f4c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:32.560011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25750d-7726-404f-a21a-27cbd3d3f4c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:32.569598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25750d-7726-404f-a21a-27cbd3d3f4c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:32.571026 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25750d-7726-404f-a21a-27cbd3d3f4c5 service cinder] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:32.580571 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4339/8655] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:40.674817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab20492-1a8b-4620-8cab-e49c005c9c3a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:40.750593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab20492-1a8b-4620-8cab-e49c005c9c3a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:40.755243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:40.755874 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:40.756150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:40.756419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] PATH_INFO: `/v3/users/64c73ef93e3b4d8da546f4ac18520560` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:40.757599 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c73ef93e3b4d8da546f4ac18520560)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:40.759513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:40.819509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c73ef93e3b4d8da546f4ac18520560'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:40.821016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c73ef93e3b4d8da546f4ac18520560'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:40.830932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c73ef93e3b4d8da546f4ac18520560'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:40.840790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c73ef93e3b4d8da546f4ac18520560'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:40.853467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c73ef93e3b4d8da546f4ac18520560'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:40.859639 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4317/8656] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:38:40 2022] DELETE /identity/v3/users/64c73ef93e3b4d8da546f4ac18520560 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:40.864409 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:40.864941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:40.865033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:40.885756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:40.886977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:40.941346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:40.951650 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4340/8657] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:40.961605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:40.963180 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:40.981661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:41.045194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd97ae-e427-4339-80a3-859cd26e904a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.050061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3c5d4-2d16-4280-90b8-8afbfdd1cb18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.050706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3c5d4-2d16-4280-90b8-8afbfdd1cb18 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.050998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3c5d4-2d16-4280-90b8-8afbfdd1cb18 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.051263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3c5d4-2d16-4280-90b8-8afbfdd1cb18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.061655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3c5d4-2d16-4280-90b8-8afbfdd1cb18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.063212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3c5d4-2d16-4280-90b8-8afbfdd1cb18 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.073608 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4318/8658] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:41.347709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:41.349210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:41.361991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be17a3-4333-450b-829a-a5771924d017 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.367427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.368074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.368346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.368610 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] PATH_INFO: `/v3/projects/3de8116c00564abd82f93555f3f81993` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.369857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de8116c00564abd82f93555f3f81993)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.371767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.421432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de8116c00564abd82f93555f3f81993', 'request_id': 'req-83176116-152d-4b6e-8f9b-caf36ada5fcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:41.430738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de8116c00564abd82f93555f3f81993', 'request_id': 'req-83176116-152d-4b6e-8f9b-caf36ada5fcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:41.436902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de8116c00564abd82f93555f3f81993', 'request_id': 'req-83176116-152d-4b6e-8f9b-caf36ada5fcc'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:41.448604 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] The token cache is being invalidate because project 3de8116c00564abd82f93555f3f81993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:38:41.448930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:41.449694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-83176116-152d-4b6e-8f9b-caf36ada5fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:38:41.451198 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4341/8659] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:38:41 2022] DELETE /identity/v3/projects/3de8116c00564abd82f93555f3f81993 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:41.456310 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13cee5-6b8f-42e2-9869-b4433ae01563 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.456589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13cee5-6b8f-42e2-9869-b4433ae01563 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.456883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13cee5-6b8f-42e2-9869-b4433ae01563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.477094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-bc13cee5-6b8f-42e2-9869-b4433ae01563 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:41.478295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-bc13cee5-6b8f-42e2-9869-b4433ae01563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:41.542835 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4319/8660] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:41.551499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a68224-f088-4e8f-a3d1-f05a84d197be None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.551782 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a68224-f088-4e8f-a3d1-f05a84d197be None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.552079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a68224-f088-4e8f-a3d1-f05a84d197be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.574368 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b2a68224-f088-4e8f-a3d1-f05a84d197be None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:41.575595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b2a68224-f088-4e8f-a3d1-f05a84d197be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:41.585164 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4342/8661] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:41.589667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13cee5-6b8f-42e2-9869-b4433ae01563 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:41.600261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13cee5-6b8f-42e2-9869-b4433ae01563 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.606099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.606770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.606998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.607294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.608349 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:38:41.608455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.610194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.623799 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4320/8662] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:38:41.628466 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a68224-f088-4e8f-a3d1-f05a84d197be None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:41.638211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a68224-f088-4e8f-a3d1-f05a84d197be None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.643476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.644080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.644345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.644609 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.645257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.647156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.677042 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4343/8663] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:38:41 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:38:41.681490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:41.692551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace39d2-d37f-4416-aa58-25f1de90999d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.749672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.750393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.750715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.750981 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.751604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.753557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.777931 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4321/8664] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:38:41 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:38:41.782522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:41.793183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba7737-5fde-4d4b-8506-36f43160cbac admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.798624 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.799231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.799506 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.799799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.800438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.802112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.812559 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4344/8665] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:38:41.817400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:41.828091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19794a-2761-4694-9f49-b8eee884fe00 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.834803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76b46c-22d4-4257-b9ea-8a5d8ad1a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.835430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76b46c-22d4-4257-b9ea-8a5d8ad1a131 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.835690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76b46c-22d4-4257-b9ea-8a5d8ad1a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.835961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76b46c-22d4-4257-b9ea-8a5d8ad1a131 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.836744 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76b46c-22d4-4257-b9ea-8a5d8ad1a131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.838353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76b46c-22d4-4257-b9ea-8a5d8ad1a131 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.849558 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4322/8666] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:38:41.854080 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:41.864700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1854fd-ddee-4816-a948-be4c89aed782 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:41.870980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:41.871635 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.871908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.872173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] PATH_INFO: `/v3/projects/91f4b185afc0495eaba8f2ece9562eb2/users/ee4c3d891eb9466c8cc00cf5d28b3924/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.891847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f4b185afc0495eaba8f2ece9562eb2, user_id=ee4c3d891eb9466c8cc00cf5d28b3924, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:41.894916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:41.912829 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4345/8667] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:38:41 2022] PUT /identity/v3/projects/91f4b185afc0495eaba8f2ece9562eb2/users/ee4c3d891eb9466c8cc00cf5d28b3924/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:41.918335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc459f1-dcbb-43b3-b5e9-a6ba421b2558 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:41.918626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc459f1-dcbb-43b3-b5e9-a6ba421b2558 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:41.918901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc459f1-dcbb-43b3-b5e9-a6ba421b2558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:41.940488 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7bc459f1-dcbb-43b3-b5e9-a6ba421b2558 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:41.941957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7bc459f1-dcbb-43b3-b5e9-a6ba421b2558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:41.961758 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:42.011952 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4323/8668] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:42.022106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:42.024049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:42.085694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a53f-1844-4204-bf19-9d15a774aecf admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:42.091460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:42.092278 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:42.092611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:42.092987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:42.108920 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:42.110902 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:42.120904 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4346/8669] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:38:42.804467 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3fdd8-fdb5-45b4-a95b-e661957ec5d3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:42.805088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3fdd8-fdb5-45b4-a95b-e661957ec5d3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:42.805088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3fdd8-fdb5-45b4-a95b-e661957ec5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:42.838288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-74c3fdd8-fdb5-45b4-a95b-e661957ec5d3 None None] MFA Rules not processed for user `ee4c3d891eb9466c8cc00cf5d28b3924`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:42.839630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-74c3fdd8-fdb5-45b4-a95b-e661957ec5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:42.908279 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4324/8670] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:42.913219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:42.923481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bb821-8dc8-4a5f-98b5-d95d4127ebda service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:42.928128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:42.928765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:42.929047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:42.929320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:42.929978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:42.931896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:42.956243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:42.964747 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4347/8671] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:38:42 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:38:42.969378 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3fdd8-fdb5-45b4-a95b-e661957ec5d3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:42.981039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3fdd8-fdb5-45b4-a95b-e661957ec5d3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:43.053948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:43.054561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:43.054837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:43.055105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:43.055729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:43.057668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:43.084899 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4325/8672] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:38:42 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:38:43.086484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:43.097353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611273-76c8-44c1-aa25-622ae93c9fec admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:43.102420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:43.103203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:43.103490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:43.104128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:43.104813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:43.106468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:43.117641 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4348/8673] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:38:43.121965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:43.132575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96088ed2-6f7b-4f04-9eef-014a845bcad0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:43.137232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:43.137888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:43.138167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:43.138442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] PATH_INFO: `/v3/projects/58bf0312ea86427dbd5e654b15539934/users/ec786d7477464c6e96506555f8062a1c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:43.153994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] RBAC: Authorizing `identity:create_grant(project_id=58bf0312ea86427dbd5e654b15539934, user_id=ec786d7477464c6e96506555f8062a1c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:43.157487 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:43.175126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4326/8674] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:38:43 2022] PUT /identity/v3/projects/58bf0312ea86427dbd5e654b15539934/users/ec786d7477464c6e96506555f8062a1c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:43.179425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:43.190738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8843d-b440-489e-b4a3-061957913d1d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:43.251962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48813c5-5783-499d-a460-2ab2756f2d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:43.252581 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48813c5-5783-499d-a460-2ab2756f2d44 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:43.252884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48813c5-5783-499d-a460-2ab2756f2d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:43.253156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48813c5-5783-499d-a460-2ab2756f2d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:43.253792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48813c5-5783-499d-a460-2ab2756f2d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:43.255450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48813c5-5783-499d-a460-2ab2756f2d44 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:43.265433 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4349/8675] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:38:43.270425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:43.281745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfcc99-bb0c-49fb-8b43-4c08cdc8248b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:43.287223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f17b77-35d1-4002-ad08-825da4d406c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:43.287877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f17b77-35d1-4002-ad08-825da4d406c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:43.288177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f17b77-35d1-4002-ad08-825da4d406c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:43.288430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f17b77-35d1-4002-ad08-825da4d406c6 admin admin] PATH_INFO: `/v3/projects/58bf0312ea86427dbd5e654b15539934/users/ec786d7477464c6e96506555f8062a1c/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:43.290462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f17b77-35d1-4002-ad08-825da4d406c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58bf0312ea86427dbd5e654b15539934, user_id=ec786d7477464c6e96506555f8062a1c, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:43.293129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f17b77-35d1-4002-ad08-825da4d406c6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:43.304272 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4327/8676] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:38:43 2022] PUT /identity/v3/projects/58bf0312ea86427dbd5e654b15539934/users/ec786d7477464c6e96506555f8062a1c/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:38:43.961253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:43.961758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:43.961758 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:43.992160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] MFA Rules not processed for user `ec786d7477464c6e96506555f8062a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:43.993562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:44.069100 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4350/8677] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:44.082471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88b08-9b5d-4613-944b-854ca4446325 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:44.083130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88b08-9b5d-4613-944b-854ca4446325 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:44.083130 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88b08-9b5d-4613-944b-854ca4446325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:44.099973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:44.112650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5da88b08-9b5d-4613-944b-854ca4446325 None None] MFA Rules not processed for user `ee4c3d891eb9466c8cc00cf5d28b3924`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:38:44.113907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5da88b08-9b5d-4613-944b-854ca4446325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:44.170075 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4328/8678] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:38:44.188960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:44.190709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:44.267592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800d73e-7e49-4b73-8250-e0d4e5c742ea None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:44.277036 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:44.277647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:44.277917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:44.278186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:44.286481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=100 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:44.289856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:44.291973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:44.301893 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4351/8679] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:48.867963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88b08-9b5d-4613-944b-854ca4446325 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:38:48.885902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88b08-9b5d-4613-944b-854ca4446325 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:38:48.892631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:38:48.893289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:38:48.893569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:38:48.893842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:38:48.895827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:38:48.982618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:38:48.984320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:38:48.995239 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4329/8680] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:38:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:38:54.297546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:58.989544 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=60 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:08.714656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:08.716367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:08.796509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f86d1-ab67-49ce-87c1-5f6e1a9e1ed1 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:08.801787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:08.802399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:08.802677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:08.802950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:08.804892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:08.819867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:08.821534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:08.835395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4352/8681] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:39:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:39:09.393431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:09.411309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae31c2c-bc77-4702-aa59-7a0ece289b42 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:09.418555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:09.419186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:09.419466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:09.419798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:09.421639 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:09.439251 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:09.440831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:09.455835 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4330/8682] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:39:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:39:09.510845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:09.585511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064f65c-14e4-4e2f-9a5f-f1b20d9dc323 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:09.590518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:09.591253 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:09.591532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:09.591802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] PATH_INFO: `/v3/users/ee4c3d891eb9466c8cc00cf5d28b3924` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:09.592938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee4c3d891eb9466c8cc00cf5d28b3924)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:09.594938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:09.658269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee4c3d891eb9466c8cc00cf5d28b3924'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:09.659738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee4c3d891eb9466c8cc00cf5d28b3924'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:09.671719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee4c3d891eb9466c8cc00cf5d28b3924'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:09.681295 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee4c3d891eb9466c8cc00cf5d28b3924'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:09.682128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee4c3d891eb9466c8cc00cf5d28b3924'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:09.704160 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4353/8683] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:39:09 2022] DELETE /identity/v3/users/ee4c3d891eb9466c8cc00cf5d28b3924 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:09.878293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:09.890112 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f7af-acac-4f1b-9085-419d6184a838 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:09.950170 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:09.950798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:09.951072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:09.951345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] PATH_INFO: `/v3/projects/91f4b185afc0495eaba8f2ece9562eb2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:09.952470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f4b185afc0495eaba8f2ece9562eb2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:09.954434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:10.004574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f4b185afc0495eaba8f2ece9562eb2', 'request_id': 'req-35e1529d-b577-4991-9a15-52439d247efa'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.011058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f4b185afc0495eaba8f2ece9562eb2', 'request_id': 'req-35e1529d-b577-4991-9a15-52439d247efa'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.020505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f4b185afc0495eaba8f2ece9562eb2', 'request_id': 'req-35e1529d-b577-4991-9a15-52439d247efa'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.031580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] The token cache is being invalidate because project 91f4b185afc0495eaba8f2ece9562eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:39:10.031868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.032489 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.034034 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4331/8684] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:39:09 2022] DELETE /identity/v3/projects/91f4b185afc0495eaba8f2ece9562eb2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:10.038591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:10.039863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:10.053371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0c41-a384-42a7-9d87-375d6c48b7a1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:10.058384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:10.113648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:10.114284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:10.114595 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:10.114867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] PATH_INFO: `/v3/users/ec786d7477464c6e96506555f8062a1c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:10.116013 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec786d7477464c6e96506555f8062a1c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:10.118016 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:10.168884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec786d7477464c6e96506555f8062a1c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.169541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec786d7477464c6e96506555f8062a1c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.179284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec786d7477464c6e96506555f8062a1c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.186218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec786d7477464c6e96506555f8062a1c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.187197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec786d7477464c6e96506555f8062a1c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.205263 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4354/8685] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:39:10 2022] DELETE /identity/v3/users/ec786d7477464c6e96506555f8062a1c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:10.396686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:10.398351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:10.412398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1529d-b577-4991-9a15-52439d247efa admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:10.478448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:10.479435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:10.479718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:10.479976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] PATH_INFO: `/v3/projects/58bf0312ea86427dbd5e654b15539934` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:10.481189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58bf0312ea86427dbd5e654b15539934)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:10.483114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:10.506051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:10.536305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58bf0312ea86427dbd5e654b15539934', 'request_id': 'req-9ff17ae5-827a-460f-8793-6440bc10b918'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.543016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58bf0312ea86427dbd5e654b15539934', 'request_id': 'req-9ff17ae5-827a-460f-8793-6440bc10b918'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.552893 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58bf0312ea86427dbd5e654b15539934', 'request_id': 'req-9ff17ae5-827a-460f-8793-6440bc10b918'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.564442 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] The token cache is being invalidate because project 58bf0312ea86427dbd5e654b15539934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:39:10.564670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.565332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9ff17ae5-827a-460f-8793-6440bc10b918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:10.566865 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4332/8686] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:39:10 2022] DELETE /identity/v3/projects/58bf0312ea86427dbd5e654b15539934 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:20.201383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:20.560724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:28.871828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:28.873328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:28.890420 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113095b-1d2f-4627-85f9-ecc0cad40752 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:28.952941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:28.953577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:28.953854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:28.954130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] PATH_INFO: `/v3/users/0bd8a19d0ab44011b5430f26757a0efe` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:28.955342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd8a19d0ab44011b5430f26757a0efe)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:28.957324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:29.006038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd8a19d0ab44011b5430f26757a0efe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.006685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd8a19d0ab44011b5430f26757a0efe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.015890 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd8a19d0ab44011b5430f26757a0efe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.023117 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd8a19d0ab44011b5430f26757a0efe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.023742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd8a19d0ab44011b5430f26757a0efe'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.040356 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4355/8687] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:39:28 2022] DELETE /identity/v3/users/0bd8a19d0ab44011b5430f26757a0efe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:29.045608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:29.046124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:29.046124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:29.072288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:39:29.074069 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:29.146246 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4333/8688] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:39:29.156909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:29.158545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:29.225139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1270755-c05c-40e7-a72c-5b30ec0c773e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:29.229589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:29.230195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:29.230536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:29.230807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:29.244130 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:29.246292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:29.260705 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4356/8689] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:39:29.447826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:29.449630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:29.463931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d5149-dda3-46db-88d4-2855909d2b4e None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:29.469557 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:29.470215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:29.470491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:29.470764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] PATH_INFO: `/v3/projects/86e71212d96f46a78680afaacd8709ca` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:29.472533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e71212d96f46a78680afaacd8709ca)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:29.473982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:29.524588 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e71212d96f46a78680afaacd8709ca', 'request_id': 'req-594ac1ce-539e-457e-9793-32af342c5dcb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.531289 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e71212d96f46a78680afaacd8709ca', 'request_id': 'req-594ac1ce-539e-457e-9793-32af342c5dcb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.539852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e71212d96f46a78680afaacd8709ca', 'request_id': 'req-594ac1ce-539e-457e-9793-32af342c5dcb'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.550783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] The token cache is being invalidate because project 86e71212d96f46a78680afaacd8709ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:39:29.551040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.551727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.553296 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4334/8690] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:39:29 2022] DELETE /identity/v3/projects/86e71212d96f46a78680afaacd8709ca => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:29.557602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:29.570030 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f3ff1-6ff3-4846-bf91-286b83fd5c1d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:29.592115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:29.638753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:29.639390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:29.639695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:29.639948 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] PATH_INFO: `/v3/users/5436a87ddec64a47bde28e63c0a3a368` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:29.656049 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5436a87ddec64a47bde28e63c0a3a368)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:29.658017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:29.714018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5436a87ddec64a47bde28e63c0a3a368'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.714732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5436a87ddec64a47bde28e63c0a3a368'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.724465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5436a87ddec64a47bde28e63c0a3a368'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.732139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5436a87ddec64a47bde28e63c0a3a368'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.732859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6165cd0e-00ac-4315-b888-dcb355177e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5436a87ddec64a47bde28e63c0a3a368'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:29.751361 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4357/8691] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:39:29 2022] DELETE /identity/v3/users/5436a87ddec64a47bde28e63c0a3a368 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:29.935100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:29.936976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:29.951925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ac1ce-539e-457e-9793-32af342c5dcb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.014177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.014815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.015091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.015361 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] PATH_INFO: `/v3/projects/d4cd350726034f9985727755c395c181` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.026344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cd350726034f9985727755c395c181)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.028309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.047891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:30.085789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cd350726034f9985727755c395c181', 'request_id': 'req-0882c9d7-f505-4119-977c-8a6623dbb9f9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:30.093531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cd350726034f9985727755c395c181', 'request_id': 'req-0882c9d7-f505-4119-977c-8a6623dbb9f9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:30.102931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cd350726034f9985727755c395c181', 'request_id': 'req-0882c9d7-f505-4119-977c-8a6623dbb9f9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:30.114373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] The token cache is being invalidate because project d4cd350726034f9985727755c395c181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:39:30.114702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:30.115322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0882c9d7-f505-4119-977c-8a6623dbb9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:39:30.116891 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4335/8692] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:39:29 2022] DELETE /identity/v3/projects/d4cd350726034f9985727755c395c181 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:30.122428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce0a1f-7ff0-4dfb-8902-c84772bca6f6 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.122906 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce0a1f-7ff0-4dfb-8902-c84772bca6f6 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.123006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce0a1f-7ff0-4dfb-8902-c84772bca6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.144210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f0ce0a1f-7ff0-4dfb-8902-c84772bca6f6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:39:30.145454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f0ce0a1f-7ff0-4dfb-8902-c84772bca6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:30.214692 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4358/8693] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:39:30.223153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0320c2-d8ac-42b7-9876-a41bc8c807cf None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.223526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0320c2-d8ac-42b7-9876-a41bc8c807cf None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.223804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0320c2-d8ac-42b7-9876-a41bc8c807cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.246589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-df0320c2-d8ac-42b7-9876-a41bc8c807cf None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:39:30.248010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-df0320c2-d8ac-42b7-9876-a41bc8c807cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:30.256590 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4336/8694] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:39:30.261135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce0a1f-7ff0-4dfb-8902-c84772bca6f6 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:30.271503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce0a1f-7ff0-4dfb-8902-c84772bca6f6 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.277177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.277842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.278118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.278384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.279033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:39:30.279363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.281024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.293007 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4359/8695] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:39:30.299038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0320c2-d8ac-42b7-9876-a41bc8c807cf None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:30.310770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0320c2-d8ac-42b7-9876-a41bc8c807cf None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.317253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.318156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.318543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.318905 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.319798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.322771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.358310 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4337/8696] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:39:30 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:39:30.363242 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:30.377090 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421029d-3ba8-46c3-a185-d5cf6c237b76 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.391372 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:30.441314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.441929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.442207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.442483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.443112 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.445502 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.470725 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4360/8697] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:39:30 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:39:30.474980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:30.486732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fe8a5-28e3-4bb1-bd58-9ff6dc96b783 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.492008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.492618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.492940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.493266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.493916 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.495520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.506163 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4338/8698] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:39:30.510534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:30.521388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c4760-0703-4c7d-8d9d-4cb8847d1769 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.526279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c0dd0-c06d-42b8-ad9b-abca5f401016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.526877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c0dd0-c06d-42b8-ad9b-abca5f401016 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.527169 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c0dd0-c06d-42b8-ad9b-abca5f401016 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.527898 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c0dd0-c06d-42b8-ad9b-abca5f401016 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.528326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c0dd0-c06d-42b8-ad9b-abca5f401016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.529944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c0dd0-c06d-42b8-ad9b-abca5f401016 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.541309 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4361/8699] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:39:30.546271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:30.562029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577914a-dc5d-47ee-91eb-bc974c068b5f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.568177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.568851 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.569150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.569539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] PATH_INFO: `/v3/projects/874d3b71bb7f44379f7efb4f68c7a14f/users/43ffa23d4c3843539e4be19528794e3c/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.586653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] RBAC: Authorizing `identity:create_grant(project_id=874d3b71bb7f44379f7efb4f68c7a14f, user_id=43ffa23d4c3843539e4be19528794e3c, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.589358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.607491 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4339/8700] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:39:30 2022] PUT /identity/v3/projects/874d3b71bb7f44379f7efb4f68c7a14f/users/43ffa23d4c3843539e4be19528794e3c/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:39:30.612923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389f172-5162-4ba4-b2a0-37adadda563f None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.613209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389f172-5162-4ba4-b2a0-37adadda563f None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.613483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389f172-5162-4ba4-b2a0-37adadda563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.634308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5389f172-5162-4ba4-b2a0-37adadda563f None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:39:30.635534 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5389f172-5162-4ba4-b2a0-37adadda563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:30.697995 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4362/8701] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:39:30.707900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:30.709779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:30.774445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcc7c3-5391-42ba-a993-9f382962830b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:30.780547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:30.781451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:30.781806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:30.782102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:30.796263 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:30.797788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:30.808290 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4340/8702] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:39:35.016351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da78ab8-f5e7-4ef5-a70e-a73b78f08cf5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:35.016916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da78ab8-f5e7-4ef5-a70e-a73b78f08cf5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:35.016916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da78ab8-f5e7-4ef5-a70e-a73b78f08cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:35.047431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4da78ab8-f5e7-4ef5-a70e-a73b78f08cf5 None None] MFA Rules not processed for user `43ffa23d4c3843539e4be19528794e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:39:35.048641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4da78ab8-f5e7-4ef5-a70e-a73b78f08cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:35.117150 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4363/8703] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:39:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:39:35.121849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:35.132651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55481ba9-a3c9-4e43-82cf-2e744f48183c service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:35.137702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:35.138327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:35.138638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:35.138877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:35.139509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:35.141899 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:35.175851 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4341/8704] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:39:35 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:39:35.177147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da78ab8-f5e7-4ef5-a70e-a73b78f08cf5 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:35.189301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da78ab8-f5e7-4ef5-a70e-a73b78f08cf5 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:35.215257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:35.253512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:35.254280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:35.254615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:35.254957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:35.255665 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:35.257715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:35.282542 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4364/8705] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:39:35 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:39:35.286190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:35.297886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54f8dd-fd90-4bdb-8cf2-aa5c334cb621 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:35.303050 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:35.303706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:35.303978 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:35.304353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:35.305121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:35.306817 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:35.319360 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4342/8706] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:39:35.323362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:35.334064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e16ab-85c5-42cc-b6c0-006db9fb1d1f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:35.338868 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:35.339547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:35.339820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:35.340133 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] PATH_INFO: `/v3/projects/bde5ae10f99049d89827918d3143c49d/users/97da417f37344608bc67c794e9b5f606/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:35.357916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde5ae10f99049d89827918d3143c49d, user_id=97da417f37344608bc67c794e9b5f606, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:35.360987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:35.376880 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4365/8707] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:39:35 2022] PUT /identity/v3/projects/bde5ae10f99049d89827918d3143c49d/users/97da417f37344608bc67c794e9b5f606/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:35.380685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:35.391386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87134f9-1b97-4a41-ab1f-c537b5681f92 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:35.400101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:35.455642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100014a-4f25-4f9b-b952-9156ab94cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:35.456445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100014a-4f25-4f9b-b952-9156ab94cfde admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:35.456856 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100014a-4f25-4f9b-b952-9156ab94cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:35.457027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100014a-4f25-4f9b-b952-9156ab94cfde admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:35.457708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100014a-4f25-4f9b-b952-9156ab94cfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:35.459351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100014a-4f25-4f9b-b952-9156ab94cfde admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:35.470271 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4343/8708] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:39:35.474449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:35.485783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3ce69-7f0a-404b-b0bc-64183ebdb551 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:35.490471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:35.491147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:35.491425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:35.491698 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] PATH_INFO: `/v3/projects/bde5ae10f99049d89827918d3143c49d/users/97da417f37344608bc67c794e9b5f606/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:35.493653 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde5ae10f99049d89827918d3143c49d, user_id=97da417f37344608bc67c794e9b5f606, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:35.496363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:35.507610 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4366/8709] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:39:35 2022] PUT /identity/v3/projects/bde5ae10f99049d89827918d3143c49d/users/97da417f37344608bc67c794e9b5f606/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:39:39.986673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:39.987176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:39.987277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:40.033952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] MFA Rules not processed for user `97da417f37344608bc67c794e9b5f606`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:39:40.035656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:40.160704 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4344/8710] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:39:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:39:40.172451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:40.175159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:40.287165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5938aea-67cc-4f6d-bbdc-251a725779b5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:40.291229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1523b9-c229-435a-93b5-16fb5762deaa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:40.292064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1523b9-c229-435a-93b5-16fb5762deaa service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:40.292640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1523b9-c229-435a-93b5-16fb5762deaa service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:40.293114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1523b9-c229-435a-93b5-16fb5762deaa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:40.386417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1523b9-c229-435a-93b5-16fb5762deaa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:40.388185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1523b9-c229-435a-93b5-16fb5762deaa service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:40.402253 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4367/8711] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:39:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:39:41.876221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:39:41.877779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:41.955466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aac489-ab4d-40fa-a2ad-933fd11a1ad8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:39:41.961240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:39:41.961865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:41.962145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:41.962416 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:41.964281 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:41.979192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:39:41.980954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:39:41.994966 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4345/8712] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:39:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:39:50.393881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:51.987056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:57.724336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:39:57.724915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:39:57.724915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:39:57.745611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] MFA Rules not processed for user `43ffa23d4c3843539e4be19528794e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:39:57.746911 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:39:57.754943 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4368/8713] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:39:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:40:07.739701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:12.107581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:40:12.171209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9709c0-1c8a-4eda-b891-2d04c8bf31f2 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:40:12.175384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:40:12.175994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:40:12.176271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:40:12.176701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:40:12.186736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:40:12.188157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:40:12.198093 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4346/8714] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:40:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:40:13.888312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:40:13.889970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:40:13.974742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03834d87-4c9a-46fd-a702-e4047242bbbb None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:40:13.980492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:40:13.981229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:40:13.981583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:40:13.982243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:40:13.983714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:40:13.999111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:40:14.000776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:40:14.015041 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4369/8715] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:40:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:40:22.193608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=26 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:24.009628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=34 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:47.344010 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:40:47.360598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5d1bd-bb5a-435d-8af6-29f36fa4ee36 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:40:47.366445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:40:47.367064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:40:47.367649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:40:47.367649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:40:47.369932 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:40:47.387940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:40:47.389809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:40:47.408680 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4347/8716] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:40:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:40:57.396873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:10.713380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:10.800006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf24fc5-a12e-498d-be1b-516df52e159b service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:10.804738 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:10.805728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:10.806001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:10.806269 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] PATH_INFO: `/v3/users/43ffa23d4c3843539e4be19528794e3c` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:10.807388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ffa23d4c3843539e4be19528794e3c)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:10.809374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:10.859668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ffa23d4c3843539e4be19528794e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:10.860321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ffa23d4c3843539e4be19528794e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:10.873364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ffa23d4c3843539e4be19528794e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:10.882267 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ffa23d4c3843539e4be19528794e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:10.882952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ffa23d4c3843539e4be19528794e3c'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:10.900882 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4370/8717] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:41:10 2022] DELETE /identity/v3/users/43ffa23d4c3843539e4be19528794e3c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:11.100267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:11.133335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9228e1-8f18-4421-9d62-f3311972e9c1 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:11.193996 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:11.194665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:11.194946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:11.195226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] PATH_INFO: `/v3/projects/874d3b71bb7f44379f7efb4f68c7a14f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:11.196381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=874d3b71bb7f44379f7efb4f68c7a14f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:11.198350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:11.248003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874d3b71bb7f44379f7efb4f68c7a14f', 'request_id': 'req-64d57013-08ad-4dd1-a049-844ed4f936d3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.254842 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874d3b71bb7f44379f7efb4f68c7a14f', 'request_id': 'req-64d57013-08ad-4dd1-a049-844ed4f936d3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.264406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874d3b71bb7f44379f7efb4f68c7a14f', 'request_id': 'req-64d57013-08ad-4dd1-a049-844ed4f936d3'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.275829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] The token cache is being invalidate because project 874d3b71bb7f44379f7efb4f68c7a14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:41:11.276118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.276784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.278318 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4348/8718] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:41:11 2022] DELETE /identity/v3/projects/874d3b71bb7f44379f7efb4f68c7a14f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:11.282648 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:11.284314 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:11.310608 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aa4c7-6d62-4445-97ae-cebfe64e5526 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:11.359257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:11.369817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:11.370457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:11.370998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:11.371418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] PATH_INFO: `/v3/users/97da417f37344608bc67c794e9b5f606` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:11.372756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=97da417f37344608bc67c794e9b5f606)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:11.374673 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:11.424908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97da417f37344608bc67c794e9b5f606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.425599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97da417f37344608bc67c794e9b5f606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.437382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97da417f37344608bc67c794e9b5f606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.444763 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97da417f37344608bc67c794e9b5f606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.445413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-875e27a4-dbea-4e6d-9ad9-4092dd6708ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97da417f37344608bc67c794e9b5f606'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.464575 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4371/8719] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:41:11 2022] DELETE /identity/v3/users/97da417f37344608bc67c794e9b5f606 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:11.642156 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:11.643685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:11.657256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d57013-08ad-4dd1-a049-844ed4f936d3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:11.716333 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:11.717003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:11.717273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:11.717542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] PATH_INFO: `/v3/projects/bde5ae10f99049d89827918d3143c49d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:11.718678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde5ae10f99049d89827918d3143c49d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:11.720651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:11.738659 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:11.773863 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde5ae10f99049d89827918d3143c49d', 'request_id': 'req-b7e425f4-081b-4478-8d92-9923e1027ff7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.780138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde5ae10f99049d89827918d3143c49d', 'request_id': 'req-b7e425f4-081b-4478-8d92-9923e1027ff7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.790153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde5ae10f99049d89827918d3143c49d', 'request_id': 'req-b7e425f4-081b-4478-8d92-9923e1027ff7'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.800779 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] The token cache is being invalidate because project bde5ae10f99049d89827918d3143c49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:41:11.801067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.801684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b7e425f4-081b-4478-8d92-9923e1027ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:11.803179 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4349/8720] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:41:11 2022] DELETE /identity/v3/projects/bde5ae10f99049d89827918d3143c49d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:21.460752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:21.797770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:22.301802 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7e3a8-b9c7-4386-a92f-7e04225ad197 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.302072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7e3a8-b9c7-4386-a92f-7e04225ad197 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.302437 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7e3a8-b9c7-4386-a92f-7e04225ad197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.325656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c7e7e3a8-b9c7-4386-a92f-7e04225ad197 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:22.326897 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c7e7e3a8-b9c7-4386-a92f-7e04225ad197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:22.390895 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4372/8721] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:41:22.402792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b7222-e77f-4a2d-bafb-a16541d62df9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.403020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b7222-e77f-4a2d-bafb-a16541d62df9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.403272 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b7222-e77f-4a2d-bafb-a16541d62df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.428043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2b0b7222-e77f-4a2d-bafb-a16541d62df9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:22.430101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2b0b7222-e77f-4a2d-bafb-a16541d62df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:22.439300 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4350/8722] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:22.444222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7e3a8-b9c7-4386-a92f-7e04225ad197 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:22.454993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7e3a8-b9c7-4386-a92f-7e04225ad197 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:22.461139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:22.461741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.462008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.462271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.462921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:41:22.463199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:22.464867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:22.478290 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4373/8723] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:41:22.494265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b7222-e77f-4a2d-bafb-a16541d62df9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:22.504223 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b7222-e77f-4a2d-bafb-a16541d62df9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:22.510861 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:22.511482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.511764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.512043 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.512732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:22.514680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:22.547279 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4351/8724] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:41:22 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:41:22.551954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:22.562880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5f1d7-0b84-4454-a6ef-707d31f25ea8 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:22.626694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:22.627311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.627582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.627845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.628462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:22.630447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:22.657765 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4374/8725] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:41:22 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:41:22.662710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:22.673754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc824a-8394-4b08-a558-9bbabf77d884 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:22.680066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:22.680688 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.680974 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.681242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.681888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:22.683493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:22.694134 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4352/8726] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:41:22.698367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:22.708872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec2ce0-3f91-4243-9304-be30d146d86c admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:22.714953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5813d1a-1e0a-45fd-a989-d509b0e1a6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:22.715559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813d1a-1e0a-45fd-a989-d509b0e1a6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.715856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813d1a-1e0a-45fd-a989-d509b0e1a6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.716126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813d1a-1e0a-45fd-a989-d509b0e1a6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.716817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5813d1a-1e0a-45fd-a989-d509b0e1a6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:22.718426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5813d1a-1e0a-45fd-a989-d509b0e1a6e1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:22.730138 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4375/8727] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:41:22.734539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:22.745789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90801a5-d655-45f4-bd9f-549c3f20b7f5 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:22.751625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:22.752287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.752568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.752884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] PATH_INFO: `/v3/projects/dc6b14a8306e4e23a8f88b44ec0b65e9/users/1c0ef8d535824421863e905c2b3d6784/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.768539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6b14a8306e4e23a8f88b44ec0b65e9, user_id=1c0ef8d535824421863e905c2b3d6784, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:22.771275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:22.787149 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4353/8728] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:41:22 2022] PUT /identity/v3/projects/dc6b14a8306e4e23a8f88b44ec0b65e9/users/1c0ef8d535824421863e905c2b3d6784/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:22.800877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44492e-b72a-4dc5-b29a-e57e9f967d11 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.801161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44492e-b72a-4dc5-b29a-e57e9f967d11 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.801435 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44492e-b72a-4dc5-b29a-e57e9f967d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.833547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-aa44492e-b72a-4dc5-b29a-e57e9f967d11 None None] MFA Rules not processed for user `1c0ef8d535824421863e905c2b3d6784`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:22.834761 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa44492e-b72a-4dc5-b29a-e57e9f967d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:22.892352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:22.902154 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4376/8729] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:22.906811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:22.989939 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9095-f0b2-4477-954a-cf2b37c847ce admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:22.996694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:22.997288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:22.997571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:22.997897 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:22.998525 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:23.000439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:23.036366 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4354/8730] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:41:22 2022] POST /identity/v3/projects => generated 391 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:41:23.040498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44492e-b72a-4dc5-b29a-e57e9f967d11 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:23.052037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44492e-b72a-4dc5-b29a-e57e9f967d11 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:23.111189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:23.111727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.112024 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.112309 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.113068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:23.115008 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:23.140990 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4377/8731] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:41:23 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:41:23.145175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:23.155995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe8a9b-9291-492e-9380-45d7d6256d61 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:23.160894 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:23.161509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.161802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.162068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.162859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:23.165090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:23.177212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4355/8732] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:41:23.182095 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:23.192917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb3403-3bef-4fc8-9564-7fca498561d1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:23.199245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:23.199901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.200172 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.200421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] PATH_INFO: `/v3/projects/1bc3c71db8db43ac92d01f72920edb9a/users/4e8b680265904df18425b959c0bc85ce/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.216168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc3c71db8db43ac92d01f72920edb9a, user_id=4e8b680265904df18425b959c0bc85ce, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:23.218849 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:23.234220 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4378/8733] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:41:23 2022] PUT /identity/v3/projects/1bc3c71db8db43ac92d01f72920edb9a/users/4e8b680265904df18425b959c0bc85ce/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:23.238337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:23.248797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27218-7f65-4dfb-99ba-f8839fb1b988 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:23.308566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d08cd-a1b7-402c-af29-fb7cfabfb9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:23.309219 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d08cd-a1b7-402c-af29-fb7cfabfb9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.309494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d08cd-a1b7-402c-af29-fb7cfabfb9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.309761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d08cd-a1b7-402c-af29-fb7cfabfb9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.310395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d08cd-a1b7-402c-af29-fb7cfabfb9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:23.311988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d08cd-a1b7-402c-af29-fb7cfabfb9d8 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:23.325120 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4356/8734] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:41:23.330408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:23.347424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae4ce9-c085-4d86-a13c-95cf8efa052a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:23.354861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d102e-cbe9-48d1-a990-b7e9cba22b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:23.355521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d102e-cbe9-48d1-a990-b7e9cba22b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.355798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d102e-cbe9-48d1-a990-b7e9cba22b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.356111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d102e-cbe9-48d1-a990-b7e9cba22b84 admin admin] PATH_INFO: `/v3/projects/1bc3c71db8db43ac92d01f72920edb9a/users/4e8b680265904df18425b959c0bc85ce/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.358317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d102e-cbe9-48d1-a990-b7e9cba22b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc3c71db8db43ac92d01f72920edb9a, user_id=4e8b680265904df18425b959c0bc85ce, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:23.361032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d102e-cbe9-48d1-a990-b7e9cba22b84 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:23.372473 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4379/8735] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:41:23 2022] PUT /identity/v3/projects/1bc3c71db8db43ac92d01f72920edb9a/users/4e8b680265904df18425b959c0bc85ce/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:41:23.383351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.383658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.383933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.398207 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:23.412422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] MFA Rules not processed for user `4e8b680265904df18425b959c0bc85ce`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:23.414288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:23.492385 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4357/8736] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:23.505836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89c14d-93dc-4407-8441-b972dee10050 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.506366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89c14d-93dc-4407-8441-b972dee10050 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.506493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89c14d-93dc-4407-8441-b972dee10050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.531529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ea89c14d-93dc-4407-8441-b972dee10050 None None] MFA Rules not processed for user `1c0ef8d535824421863e905c2b3d6784`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:23.532771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ea89c14d-93dc-4407-8441-b972dee10050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:23.601067 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4380/8737] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:41:23.614717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:23.616649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:23.703957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3d86b-7159-4a46-a630-00110099a8c9 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:23.711911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dd571-d03a-4b07-b15a-3afe8e9d021c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:23.712521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dd571-d03a-4b07-b15a-3afe8e9d021c service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:23.712827 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dd571-d03a-4b07-b15a-3afe8e9d021c service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:23.713095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dd571-d03a-4b07-b15a-3afe8e9d021c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:23.722192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dd571-d03a-4b07-b15a-3afe8e9d021c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:23.723777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dd571-d03a-4b07-b15a-3afe8e9d021c service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:23.733299 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4358/8738] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:33.592817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:33.728968 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=65 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:35.966701 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89c14d-93dc-4407-8441-b972dee10050 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:35.983799 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89c14d-93dc-4407-8441-b972dee10050 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:35.991143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:35.991774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:35.992035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:35.992296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:36.002100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:36.003691 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:36.014059 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4381/8739] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:35 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:36.029234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd3ae1-c7a3-4382-9abc-b12c1265e29d None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:36.029748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd3ae1-c7a3-4382-9abc-b12c1265e29d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:36.029748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd3ae1-c7a3-4382-9abc-b12c1265e29d None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:36.031126 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4359/8740] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:41:36.036211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:41:36.058731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:36.067579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f219e-3e84-490a-9350-471592cff066 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:36.073860 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8b680265904df18425b959c0bc85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1bc3c71db8db43ac92d01f72920edb9a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:36.075012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:36.075288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:36.075592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] PATH_INFO: `/v3/projects/1bc3c71db8db43ac92d01f72920edb9a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:36.076909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] RBAC: Authorizing `identity:get_project(project_id=1bc3c71db8db43ac92d01f72920edb9a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:36.078748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:36.080426 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4382/8741] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:41:36 2022] GET /identity/v3/projects/1bc3c71db8db43ac92d01f72920edb9a => generated 391 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:41:36.408748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e72d-c8f5-4ac2-9bd8-a7b4ea5c394c None None] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:36.409105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e72d-c8f5-4ac2-9bd8-a7b4ea5c394c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:36.409447 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e72d-c8f5-4ac2-9bd8-a7b4ea5c394c None None] PATH_INFO: `/` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:36.410954 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4360/8742] 10.222.0.26 () {54 vars in 912 bytes} [Tue Sep 27 10:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 10:41:36.416396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Authenticating service token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 27 10:41:36.418657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:36.438189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:36.447006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7f62-276a-4c25-a39b-8a3b33c92520 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:36.452520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8b680265904df18425b959c0bc85ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1bc3c71db8db43ac92d01f72920edb9a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:36.453333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:36.453577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:36.453864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] PATH_INFO: `/v3/projects/1bc3c71db8db43ac92d01f72920edb9a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:36.455606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] RBAC: Authorizing `identity:get_project(project_id=1bc3c71db8db43ac92d01f72920edb9a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:36.457796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:36.460032 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4383/8743] 10.222.0.26 () {58 vars in 1443 bytes} [Tue Sep 27 10:41:36 2022] GET /identity/v3/projects/1bc3c71db8db43ac92d01f72920edb9a => generated 391 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:41:36.628931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e72d-c8f5-4ac2-9bd8-a7b4ea5c394c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:36.630903 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f1c1e72d-c8f5-4ac2-9bd8-a7b4ea5c394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:36.696343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e72d-c8f5-4ac2-9bd8-a7b4ea5c394c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:36.700615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54844e33-d15b-4729-8640-6bd2df8c6bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:36.701266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54844e33-d15b-4729-8640-6bd2df8c6bea service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:36.701538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54844e33-d15b-4729-8640-6bd2df8c6bea service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:36.701805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54844e33-d15b-4729-8640-6bd2df8c6bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:36.703798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-54844e33-d15b-4729-8640-6bd2df8c6bea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:36.716725 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54844e33-d15b-4729-8640-6bd2df8c6bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:36.718174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54844e33-d15b-4729-8640-6bd2df8c6bea service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:36.729184 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4361/8744] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:39.526035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:39.595325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a32601-ed01-442f-b13c-f84567078a6b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:39.600531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:39.601198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:39.601471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:39.601742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] PATH_INFO: `/v3/users/1c0ef8d535824421863e905c2b3d6784` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:39.602907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0ef8d535824421863e905c2b3d6784)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:39.604861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:39.657569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0ef8d535824421863e905c2b3d6784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:39.658217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0ef8d535824421863e905c2b3d6784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:39.667301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0ef8d535824421863e905c2b3d6784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:39.676210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0ef8d535824421863e905c2b3d6784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:39.677081 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0ef8d535824421863e905c2b3d6784'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:39.697617 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4384/8745] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:41:39 2022] DELETE /identity/v3/users/1c0ef8d535824421863e905c2b3d6784 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:39.702537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:39.703054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:39.703159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:39.722990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:39.724177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:39.793425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4362/8746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:39.804476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:39.806484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:39.859686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:39.893383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895375f-c124-47d6-9855-f19bd6ccb455 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:39.897961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:39.898620 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:39.898887 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:39.899156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:39.909275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:39.910726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:39.920770 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4385/8747] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:40.099660 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:40.101280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:40.117482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a8222-ab18-47ef-ac85-41fb61eea1e3 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:40.122249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:40.123168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:40.123459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:40.123767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] PATH_INFO: `/v3/projects/dc6b14a8306e4e23a8f88b44ec0b65e9` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:40.125386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc6b14a8306e4e23a8f88b44ec0b65e9)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:40.127366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:40.178172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'request_id': 'req-b8085b67-b58a-4215-838e-b39242793bac'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.184891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'request_id': 'req-b8085b67-b58a-4215-838e-b39242793bac'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.196172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'request_id': 'req-b8085b67-b58a-4215-838e-b39242793bac'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.213238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] The token cache is being invalidate because project dc6b14a8306e4e23a8f88b44ec0b65e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:41:40.213549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.214351 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.216648 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4363/8748] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:41:40 2022] DELETE /identity/v3/projects/dc6b14a8306e4e23a8f88b44ec0b65e9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:40.221674 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:40.232125 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ee22-c394-4806-8fbd-30ccfa33bff0 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:40.290982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:40.291618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:40.291903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:40.292175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] PATH_INFO: `/v3/users/4e8b680265904df18425b959c0bc85ce` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:40.293354 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e8b680265904df18425b959c0bc85ce)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:40.295293 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:40.345914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e8b680265904df18425b959c0bc85ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.346523 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e8b680265904df18425b959c0bc85ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.358640 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e8b680265904df18425b959c0bc85ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.365854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e8b680265904df18425b959c0bc85ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.366522 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f1d969c-2c5f-44f4-86be-e3523622c249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e8b680265904df18425b959c0bc85ce'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.384886 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4386/8749] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:41:40 2022] DELETE /identity/v3/users/4e8b680265904df18425b959c0bc85ce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:40.675070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:40.676879 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:40.692580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8085b67-b58a-4215-838e-b39242793bac admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:40.720962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:40.756061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:40.756789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:40.757110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:40.757471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] PATH_INFO: `/v3/projects/1bc3c71db8db43ac92d01f72920edb9a` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:40.758703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc3c71db8db43ac92d01f72920edb9a)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:40.760727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:40.810909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc3c71db8db43ac92d01f72920edb9a', 'request_id': 'req-72bf1875-cd28-4a9b-8987-59d6b9af1778'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.817334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc3c71db8db43ac92d01f72920edb9a', 'request_id': 'req-72bf1875-cd28-4a9b-8987-59d6b9af1778'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.827285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc3c71db8db43ac92d01f72920edb9a', 'request_id': 'req-72bf1875-cd28-4a9b-8987-59d6b9af1778'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.838519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] The token cache is being invalidate because project 1bc3c71db8db43ac92d01f72920edb9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:41:40.838854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.839490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-72bf1875-cd28-4a9b-8987-59d6b9af1778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:41:40.841249 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4364/8750] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:41:40 2022] DELETE /identity/v3/projects/1bc3c71db8db43ac92d01f72920edb9a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:40.847516 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fdd01-1dff-4401-883d-f6077fe6a0e1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:40.848059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fdd01-1dff-4401-883d-f6077fe6a0e1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:40.848059 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fdd01-1dff-4401-883d-f6077fe6a0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:40.869539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-336fdd01-1dff-4401-883d-f6077fe6a0e1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:40.870659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-336fdd01-1dff-4401-883d-f6077fe6a0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:40.917714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:40.934599 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4387/8751] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:40.942993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc5e1-aa6d-43f7-b8c8-2257d8e7e418 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:40.943275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc5e1-aa6d-43f7-b8c8-2257d8e7e418 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:40.943545 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc5e1-aa6d-43f7-b8c8-2257d8e7e418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:40.964083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d61dc5e1-aa6d-43f7-b8c8-2257d8e7e418 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:40.965314 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d61dc5e1-aa6d-43f7-b8c8-2257d8e7e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:40.973798 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4365/8752] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:40.978194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fdd01-1dff-4401-883d-f6077fe6a0e1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:40.988182 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fdd01-1dff-4401-883d-f6077fe6a0e1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:40.993155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:40.993741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:40.994011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:40.994276 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:40.994909 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:41:40.995204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:40.996840 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:41.009002 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4388/8753] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:41:41.013308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dc5e1-aa6d-43f7-b8c8-2257d8e7e418 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:41.022840 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dc5e1-aa6d-43f7-b8c8-2257d8e7e418 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:41.027742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:41.028337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:41.028619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:41.028904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:41.029531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:41.031479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:41.063312 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4366/8754] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:41:41 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:41:41.068776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:41.079212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf267fd-0edc-4527-8f3b-ced86588aeba admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:41.298884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:41.299493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:41.299746 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:41.300037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:41.300681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:41.302634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:41.328602 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4389/8755] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:41:41 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:41:41.333060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:41.344576 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff8af5-9b5b-4bbe-bc1f-7c1acb2c1a22 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:41.350430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:41.351035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:41.351373 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:41.351647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:41.352293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:41.353934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:41.364531 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4367/8756] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 10:41:41.368686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:41.379543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eac59f-bfc5-4e9c-a8a0-e6f159c32857 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:41.384412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef6598-8ef4-4b55-884d-6c5f996cf3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:41.385060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6598-8ef4-4b55-884d-6c5f996cf3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:41.385331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6598-8ef4-4b55-884d-6c5f996cf3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:41.385598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6598-8ef4-4b55-884d-6c5f996cf3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:41.386250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef6598-8ef4-4b55-884d-6c5f996cf3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:41.387817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef6598-8ef4-4b55-884d-6c5f996cf3a3 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:41.397940 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4390/8757] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:41:41.402343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:41.412784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad5db-5ffe-4949-bc0d-c85229689cf9 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:41.418072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:41.418719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:41.418989 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:41.419258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] PATH_INFO: `/v3/projects/8ef200fc20ce4b0da3c92b5179a40e63/users/8e0f9f1c851743c7b466be4e42d4a11b/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:41.435476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef200fc20ce4b0da3c92b5179a40e63, user_id=8e0f9f1c851743c7b466be4e42d4a11b, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:41.438164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:41.454759 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4368/8758] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:41:41 2022] PUT /identity/v3/projects/8ef200fc20ce4b0da3c92b5179a40e63/users/8e0f9f1c851743c7b466be4e42d4a11b/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:41:41.459980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc606daa-31b8-459f-818c-6c987f44aff3 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:41.460260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc606daa-31b8-459f-818c-6c987f44aff3 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:41.460529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc606daa-31b8-459f-818c-6c987f44aff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:41.481057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fc606daa-31b8-459f-818c-6c987f44aff3 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:41.482241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fc606daa-31b8-459f-818c-6c987f44aff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:41.542230 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4391/8759] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:41.552808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:41.555234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:41.626805 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72ae96-7f2b-4d27-befa-d3e30493a5c3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:41.630821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fa91e-b57c-4bb3-9649-bdf14bacd1fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:41.631464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fa91e-b57c-4bb3-9649-bdf14bacd1fa service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:41.631741 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fa91e-b57c-4bb3-9649-bdf14bacd1fa service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:41.632013 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fa91e-b57c-4bb3-9649-bdf14bacd1fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:41.639516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:41.642287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fa91e-b57c-4bb3-9649-bdf14bacd1fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:41.644274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fa91e-b57c-4bb3-9649-bdf14bacd1fa service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:41.656171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4369/8760] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:45.744426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:45.745001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:45.745001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:45.774744 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] MFA Rules not processed for user `8e0f9f1c851743c7b466be4e42d4a11b`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:45.775981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:45.821225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:45.843722 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4392/8761] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:41:45.855233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:45.855735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:45.855853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:45.879531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] MFA Rules not processed for user `8e0f9f1c851743c7b466be4e42d4a11b`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:41:45.880794 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:45.887913 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4370/8762] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:41:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:41:45.901040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:45.902578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:45.986636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3ce5f-f144-41d5-bb33-4eab20d1df33 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:45.993321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:45.993960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:45.994250 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:45.994497 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:46.005118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:46.006768 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:46.016609 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4393/8763] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:46.060178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:46.061926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:46.127945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7e603-bda4-47a1-b744-be9a4073f5d5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:46.131657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:46.132277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:46.132555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:46.132860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:46.134596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:46.152097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:46.153596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:46.170782 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4371/8764] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:47.826950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:41:47.843313 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318074d-e0fc-49aa-8c90-1407a39f0707 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:41:47.847577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:41:47.848301 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:41:47.848724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:41:47.849091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:41:47.851084 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:41:47.869596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:41:47.871310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:41:47.888416 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4394/8765] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:41:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:41:56.162743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=31 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:57.879268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=47 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:00.075997 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:00.077478 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:00.092847 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc51c18-7d9c-44dd-bb51-308db19fcda5 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:00.097134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:00.097755 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:00.098035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:00.098309 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:00.100392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:00.114005 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:00.116087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:00.127591 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4372/8766] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:42:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:42:00.215113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:00.216684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:00.291892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d58a72-7453-4f15-92bc-3b972647a889 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:00.298582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:00.299190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:00.299469 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:00.299726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:00.301783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:00.317343 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:00.319000 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:00.331709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4395/8767] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:42:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:42:00.513873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:00.535410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a16f9-254e-4f5a-ab97-75aa9cb3db2f service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:00.540439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:00.541353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:00.541685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:00.542016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:00.544276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:00.563585 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:00.564790 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:00.582715 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4373/8768] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:42:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:42:10.324872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=32 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:10.571148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:40.134956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:40.148113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7112-f4d3-4965-8b73-989206ab74f3 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:40.153043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:40.153667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.153960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.154222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] PATH_INFO: `/v3/users/8e0f9f1c851743c7b466be4e42d4a11b` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.155344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0f9f1c851743c7b466be4e42d4a11b)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:40.157328 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:40.214489 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0f9f1c851743c7b466be4e42d4a11b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.215098 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0f9f1c851743c7b466be4e42d4a11b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.225134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0f9f1c851743c7b466be4e42d4a11b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.232772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0f9f1c851743c7b466be4e42d4a11b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.233462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c83197ee-74dc-4e48-afe2-0ef1776d290f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0f9f1c851743c7b466be4e42d4a11b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.251161 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4396/8769] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:42:40 2022] DELETE /identity/v3/users/8e0f9f1c851743c7b466be4e42d4a11b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:42:40.433171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:40.444332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982da361-fbaf-45b8-8868-40888512d03f service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:40.517623 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:40.518248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.518524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.518769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] PATH_INFO: `/v3/projects/8ef200fc20ce4b0da3c92b5179a40e63` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.519885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef200fc20ce4b0da3c92b5179a40e63)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:40.521935 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:40.573406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef200fc20ce4b0da3c92b5179a40e63', 'request_id': 'req-038d9e48-9429-4d86-80fd-86a2aa7f0925'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.580002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef200fc20ce4b0da3c92b5179a40e63', 'request_id': 'req-038d9e48-9429-4d86-80fd-86a2aa7f0925'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.588962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef200fc20ce4b0da3c92b5179a40e63', 'request_id': 'req-038d9e48-9429-4d86-80fd-86a2aa7f0925'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.601379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] The token cache is being invalidate because project 8ef200fc20ce4b0da3c92b5179a40e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:42:40.602002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.602558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-038d9e48-9429-4d86-80fd-86a2aa7f0925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:42:40.604651 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4374/8770] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:42:40 2022] DELETE /identity/v3/projects/8ef200fc20ce4b0da3c92b5179a40e63 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:42:40.610421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad4852-e37f-45c6-bd38-5ba236227bd9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.610563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad4852-e37f-45c6-bd38-5ba236227bd9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.610857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad4852-e37f-45c6-bd38-5ba236227bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.631220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-86ad4852-e37f-45c6-bd38-5ba236227bd9 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:42:40.632434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-86ad4852-e37f-45c6-bd38-5ba236227bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:40.692267 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4397/8771] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:42:40.701767 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d6c99-2853-41ac-8120-c569bcdafbc0 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.702055 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d6c99-2853-41ac-8120-c569bcdafbc0 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.702334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d6c99-2853-41ac-8120-c569bcdafbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.723719 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b25d6c99-2853-41ac-8120-c569bcdafbc0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:42:40.725129 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b25d6c99-2853-41ac-8120-c569bcdafbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:40.733437 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4375/8772] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:42:40.737775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad4852-e37f-45c6-bd38-5ba236227bd9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:40.748487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad4852-e37f-45c6-bd38-5ba236227bd9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:40.753896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:40.754499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.754777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.755034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.755682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:42:40.755983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:40.757623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:40.770274 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4398/8773] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:42:40.774901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d6c99-2853-41ac-8120-c569bcdafbc0 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:40.785631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d6c99-2853-41ac-8120-c569bcdafbc0 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:40.792014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:40.792611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.792915 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.793249 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.793928 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:40.795825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:40.830789 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4376/8774] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:42:40 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:42:40.835162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:40.846529 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b661c-fde3-4c4e-b5cb-4d3a69f50348 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:40.908619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:40.909252 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.909526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.909798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.910417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:40.912388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:40.920879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:40.938487 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4399/8775] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:42:40 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:42:40.942859 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:40.954408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea7bc0-9a1f-4573-b146-e5a8b46c7c98 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:40.959371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:40.959973 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.960248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.960514 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.961221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:40.962883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:40.973468 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4377/8776] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:42:40.977715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:40.988457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10934e-56d4-40e5-8d2d-31252ac2ea59 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:40.994200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689de252-abf8-4354-b544-147d2c562107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:40.994853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de252-abf8-4354-b544-147d2c562107 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:40.995127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de252-abf8-4354-b544-147d2c562107 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:40.995388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de252-abf8-4354-b544-147d2c562107 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:40.996015 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689de252-abf8-4354-b544-147d2c562107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:40.997649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689de252-abf8-4354-b544-147d2c562107 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:41.008270 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4400/8777] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:42:41.012732 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:41.023407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc5446-53d4-4dec-adc1-cb5afcabc771 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:41.030122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:41.030777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:41.031060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:41.031337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] PATH_INFO: `/v3/projects/063971e11eb044b4a844a563c6ccbc5d/users/74e30240d7c840edba43dddf3a862c4f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:41.046914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063971e11eb044b4a844a563c6ccbc5d, user_id=74e30240d7c840edba43dddf3a862c4f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:41.049614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:41.066248 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4378/8778] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:42:41 2022] PUT /identity/v3/projects/063971e11eb044b4a844a563c6ccbc5d/users/74e30240d7c840edba43dddf3a862c4f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:42:41.071395 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712361c5-51af-4bc4-9ebc-0cc2bbba0d4d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:41.071677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712361c5-51af-4bc4-9ebc-0cc2bbba0d4d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:41.071945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712361c5-51af-4bc4-9ebc-0cc2bbba0d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:41.091891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-712361c5-51af-4bc4-9ebc-0cc2bbba0d4d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:42:41.093113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-712361c5-51af-4bc4-9ebc-0cc2bbba0d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:41.168225 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4401/8779] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:42:41.178798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:41.180826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:41.213287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:41.256714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8a623-cd04-4fb3-b3d1-1d481f0f6548 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:41.261214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510cca7-2e07-4ae5-9742-d4cb74367cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:41.261854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510cca7-2e07-4ae5-9742-d4cb74367cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:41.262127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510cca7-2e07-4ae5-9742-d4cb74367cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:41.262395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510cca7-2e07-4ae5-9742-d4cb74367cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:41.271706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510cca7-2e07-4ae5-9742-d4cb74367cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:41.273203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510cca7-2e07-4ae5-9742-d4cb74367cc9 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:41.282797 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4379/8780] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:42:45.606447 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:45.607027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:45.607027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:45.639376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] MFA Rules not processed for user `74e30240d7c840edba43dddf3a862c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:42:45.640606 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:45.709080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4402/8781] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:42:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:42:45.719919 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:45.720494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:45.720494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:45.743186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] MFA Rules not processed for user `74e30240d7c840edba43dddf3a862c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:42:45.744400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:45.753257 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4380/8782] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:42:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:42:45.767004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:42:45.768573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:42:45.849932 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18771709-3b25-4714-96ad-3d3f1cd5dcd9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:42:45.856199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:42:45.856872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:42:45.857128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:42:45.857393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:42:45.862825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:45.867776 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:42:45.869689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:42:45.881420 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4403/8783] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:42:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:42:55.736253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=22 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:55.878512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:30.921009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:43:30.922603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:30.938306 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58226-d450-4250-978f-8ce6e504d1da None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:43:30.942162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:43:30.942976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:30.943221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:30.943491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:30.945276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:30.957775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:43:30.959315 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:43:30.970369 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4381/8784] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:43:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:43:31.199825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:43:31.212545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db723-8c4b-4493-8c27-3c20a7d46d56 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:43:31.216845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:43:31.217550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:31.217823 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:31.218092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:31.219949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:31.232858 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:43:31.234311 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:43:31.245469 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4404/8785] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:43:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:43:31.667811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:43:31.685188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a9533-463f-47de-946b-8ef36f1fef8a service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:43:31.692031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed4ed4-d9d4-40d0-8e1c-a46ebe2f5dcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:43:31.692787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed4ed4-d9d4-40d0-8e1c-a46ebe2f5dcb service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:31.693076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed4ed4-d9d4-40d0-8e1c-a46ebe2f5dcb service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:31.693344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed4ed4-d9d4-40d0-8e1c-a46ebe2f5dcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:31.696369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-abed4ed4-d9d4-40d0-8e1c-a46ebe2f5dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:31.712957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed4ed4-d9d4-40d0-8e1c-a46ebe2f5dcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:43:31.714589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed4ed4-d9d4-40d0-8e1c-a46ebe2f5dcb service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:43:31.729010 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4382/8786] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:43:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:43:38.320961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:43:38.322582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:38.342642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b819be1-9374-469a-80bf-9b4773a45cca service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:43:38.349046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256119d-60a6-40ee-bfea-1103386702b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:43:38.350055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256119d-60a6-40ee-bfea-1103386702b0 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:38.350562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256119d-60a6-40ee-bfea-1103386702b0 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:38.351345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256119d-60a6-40ee-bfea-1103386702b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:38.353651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a256119d-60a6-40ee-bfea-1103386702b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:38.375503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256119d-60a6-40ee-bfea-1103386702b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:43:38.378217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256119d-60a6-40ee-bfea-1103386702b0 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:43:38.399462 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4405/8787] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:43:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:43:39.133837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544941b7-c4c3-4b36-9410-05a2739b36e2 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:39.134392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544941b7-c4c3-4b36-9410-05a2739b36e2 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:39.134521 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544941b7-c4c3-4b36-9410-05a2739b36e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:39.154907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-544941b7-c4c3-4b36-9410-05a2739b36e2 None None] MFA Rules not processed for user `74e30240d7c840edba43dddf3a862c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:43:39.156194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-544941b7-c4c3-4b36-9410-05a2739b36e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:39.164821 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4383/8788] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:43:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:43:39.430432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:39.430956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:39.430956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:39.495869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:43:39.497903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:39.505250 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4406/8789] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Sep 27 10:43:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:43:39.511739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544941b7-c4c3-4b36-9410-05a2739b36e2 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:43:39.522359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544941b7-c4c3-4b36-9410-05a2739b36e2 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:43:39.526982 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:43:39.528123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:39.528359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:39.528740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:39.530850 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:39.544160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:43:39.545667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:43:39.556725 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4384/8790] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:43:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:43:39.902076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:43:39.903720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:39.979350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b4e63-b720-4a73-ba6a-3c69df2b8f0b None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:43:39.984773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:43:39.985380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:43:39.985659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:43:39.985931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:43:39.987748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:43:40.004179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:43:40.006749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:43:40.021519 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4407/8791] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:43:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:43:49.550820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=21 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:50.015266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=61 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:34.303986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:34.305445 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:34.323664 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba598db-a4c5-4c80-92b5-66aaeef7b3f8 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:34.328990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:34.329538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:34.329810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:34.330079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:34.331809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:34.346967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:34.348554 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:34.362425 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4385/8792] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:44:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:44:34.371467 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:34.409376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd397b24-cb89-4c31-8d81-ef279e868e6f service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:34.414361 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:34.415007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:34.415297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:34.415550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] PATH_INFO: `/v3/users/74e30240d7c840edba43dddf3a862c4f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:34.416754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e30240d7c840edba43dddf3a862c4f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:34.418664 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:34.474672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e30240d7c840edba43dddf3a862c4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.476147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e30240d7c840edba43dddf3a862c4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.486588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e30240d7c840edba43dddf3a862c4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.494596 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e30240d7c840edba43dddf3a862c4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.495423 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-c374e563-3f46-4048-a945-ec3d2ea23ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e30240d7c840edba43dddf3a862c4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.515055 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4408/8793] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:44:34 2022] DELETE /identity/v3/users/74e30240d7c840edba43dddf3a862c4f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:44:34.706239 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:34.718697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6994b9-a57a-48ee-ba16-9977935d603c service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:34.779266 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:34.779955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:34.780238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:34.780513 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] PATH_INFO: `/v3/projects/063971e11eb044b4a844a563c6ccbc5d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:34.781709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=063971e11eb044b4a844a563c6ccbc5d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:34.783619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:34.833024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063971e11eb044b4a844a563c6ccbc5d', 'request_id': 'req-001f3135-f081-4df6-8beb-c0fe806754cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.839503 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063971e11eb044b4a844a563c6ccbc5d', 'request_id': 'req-001f3135-f081-4df6-8beb-c0fe806754cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.848649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063971e11eb044b4a844a563c6ccbc5d', 'request_id': 'req-001f3135-f081-4df6-8beb-c0fe806754cf'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.874410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] The token cache is being invalidate because project 063971e11eb044b4a844a563c6ccbc5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:44:34.874702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.875429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-001f3135-f081-4df6-8beb-c0fe806754cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:44:34.877092 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4386/8794] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:44:34 2022] DELETE /identity/v3/projects/063971e11eb044b4a844a563c6ccbc5d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:44:34.882545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204102be-16b9-4275-b57b-688740bf8f96 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:34.882830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204102be-16b9-4275-b57b-688740bf8f96 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:34.883101 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204102be-16b9-4275-b57b-688740bf8f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:34.934438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-204102be-16b9-4275-b57b-688740bf8f96 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:44:34.935643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-204102be-16b9-4275-b57b-688740bf8f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:34.995872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4409/8795] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:44:35.005666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449ba95-690e-4d0e-bb0d-4875119f5b1b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.006046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449ba95-690e-4d0e-bb0d-4875119f5b1b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.006327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449ba95-690e-4d0e-bb0d-4875119f5b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.027324 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3449ba95-690e-4d0e-bb0d-4875119f5b1b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:44:35.028608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3449ba95-690e-4d0e-bb0d-4875119f5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:35.037754 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4387/8796] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:44:35.042754 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204102be-16b9-4275-b57b-688740bf8f96 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:35.058233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204102be-16b9-4275-b57b-688740bf8f96 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:35.065386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:35.066140 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.066413 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.066676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.067631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:44:35.067924 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:35.069547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:35.081319 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4410/8797] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 10:44:35.085823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449ba95-690e-4d0e-bb0d-4875119f5b1b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:35.095676 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449ba95-690e-4d0e-bb0d-4875119f5b1b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:35.101430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:35.102025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.102302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.102577 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.103204 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:35.105151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:35.151682 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4388/8798] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:44:35 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 10:44:35.156057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:35.167462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885157b-1281-42eb-997c-74f8680d0d5f admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:35.193203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:35.239656 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:35.239773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.240105 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.240490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.241185 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:35.243203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:35.267514 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4411/8799] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:44:35 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:44:35.272031 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:35.282486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5c9e5-dd8a-44ca-9aef-fd010cf07e2a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:35.287319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:35.287909 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.288181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.288448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.289116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:35.290711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:35.301862 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4389/8800] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:44:35.306307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:35.318831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e504c9b-d479-4a9f-91f9-1d564baa0fd4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:35.324126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614a82-c4ed-4375-a5c3-cbb00e4fd20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:35.324775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614a82-c4ed-4375-a5c3-cbb00e4fd20c admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.325054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614a82-c4ed-4375-a5c3-cbb00e4fd20c admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.325324 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614a82-c4ed-4375-a5c3-cbb00e4fd20c admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.325967 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41614a82-c4ed-4375-a5c3-cbb00e4fd20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:35.327570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41614a82-c4ed-4375-a5c3-cbb00e4fd20c admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:35.340294 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4412/8801] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:44:35.344838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:35.355325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3f041-a59b-46d7-a00a-f45088651feb admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:35.360140 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:35.360820 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.361095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.361369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] PATH_INFO: `/v3/projects/32845fb8bd064f55bb99d01e5aeff6f3/users/396fe9ce0f7542c382b01d1f5779c13f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.377702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32845fb8bd064f55bb99d01e5aeff6f3, user_id=396fe9ce0f7542c382b01d1f5779c13f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:35.380452 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:35.382706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:35.396996 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4390/8802] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:44:35 2022] PUT /identity/v3/projects/32845fb8bd064f55bb99d01e5aeff6f3/users/396fe9ce0f7542c382b01d1f5779c13f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:44:35.402244 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0331-e082-43f9-bca8-3c2bc567e255 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.402797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0331-e082-43f9-bca8-3c2bc567e255 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.402797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0331-e082-43f9-bca8-3c2bc567e255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.425498 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-940c0331-e082-43f9-bca8-3c2bc567e255 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:44:35.427811 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-940c0331-e082-43f9-bca8-3c2bc567e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:35.492782 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4413/8803] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:44:35.503371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:35.506197 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:35.587734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b1e4-f878-45d5-9ed2-30f88bc2eb92 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:35.591986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:35.592608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:35.592922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:35.593194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:35.606958 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:35.608103 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:35.620203 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4391/8804] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:44:39.419142 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d827-2798-49c9-af56-f902208d9e14 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:39.419915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d827-2798-49c9-af56-f902208d9e14 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:39.420194 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8d827-2798-49c9-af56-f902208d9e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:39.449853 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4aa8d827-2798-49c9-af56-f902208d9e14 None None] MFA Rules not processed for user `396fe9ce0f7542c382b01d1f5779c13f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:44:39.450947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4aa8d827-2798-49c9-af56-f902208d9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:39.521872 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4414/8805] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:44:39.538482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:39.540179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:39.635550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a562-4010-4d75-abad-b0d402b2d260 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:39.642058 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d403e-20e3-42d2-9720-c4413f3e8640 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:39.642711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d403e-20e3-42d2-9720-c4413f3e8640 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:39.643014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d403e-20e3-42d2-9720-c4413f3e8640 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:39.643287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d403e-20e3-42d2-9720-c4413f3e8640 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:39.652797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d403e-20e3-42d2-9720-c4413f3e8640 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:39.654502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d403e-20e3-42d2-9720-c4413f3e8640 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:39.665017 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4392/8806] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:44:41.101478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:41.101991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:41.102107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:41.121660 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] MFA Rules not processed for user `396fe9ce0f7542c382b01d1f5779c13f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:44:41.122870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:41.130522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4415/8807] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:44:41.857214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10b56d-503e-41a5-861e-e4775bfb035f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:41.857678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10b56d-503e-41a5-861e-e4775bfb035f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:41.857775 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10b56d-503e-41a5-861e-e4775bfb035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:41.914754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3c10b56d-503e-41a5-861e-e4775bfb035f None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:44:41.915977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3c10b56d-503e-41a5-861e-e4775bfb035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:41.952858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:41.993603 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4393/8808] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:44:42.005963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:42.007355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:42.025109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231b3cc-4ea3-4efc-9cac-d4a618e0ceb2 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:42.030195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:42.030815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:42.031167 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:42.031382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:42.042611 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:42.044539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:42.056768 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4416/8809] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:44:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:44:44.902122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10b56d-503e-41a5-861e-e4775bfb035f None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:44:44.917252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10b56d-503e-41a5-861e-e4775bfb035f None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:44:44.923093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb0042-05ce-47a6-a083-a64972edad82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:44:44.923546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb0042-05ce-47a6-a083-a64972edad82 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:44:44.923821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb0042-05ce-47a6-a083-a64972edad82 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:44:44.924095 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb0042-05ce-47a6-a083-a64972edad82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:44:44.925901 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-02eb0042-05ce-47a6-a083-a64972edad82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:44:44.941535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb0042-05ce-47a6-a083-a64972edad82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:44:44.943878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb0042-05ce-47a6-a083-a64972edad82 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:44:44.957233 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4394/8810] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:44:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:44:52.051787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:54.950145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=21 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:54.254791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:45:54.256500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:45:54.276817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab9444-a0b7-4d63-97d1-a25d944ab557 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:45:54.283688 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:45:54.284271 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:45:54.284544 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:45:54.284852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:45:54.286756 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:45:54.301899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:45:54.303481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:45:54.316472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4417/8811] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:45:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:45:59.671590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c408981-2119-42ee-aca6-49442c136a75 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:45:59.672205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c408981-2119-42ee-aca6-49442c136a75 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:45:59.672205 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c408981-2119-42ee-aca6-49442c136a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:45:59.694745 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6c408981-2119-42ee-aca6-49442c136a75 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:45:59.695999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6c408981-2119-42ee-aca6-49442c136a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:45:59.705963 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4395/8812] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:45:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:45:59.720193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:45:59.734251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe89c8-1415-4f32-acab-91fec9199c72 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:45:59.739752 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:45:59.740507 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:45:59.740813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:45:59.741083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:45:59.749872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:45:59.751541 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:45:59.762522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4418/8813] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:45:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:46:00.005385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf02208-8e09-40e8-b778-7f904d59846b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:46:00.005702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf02208-8e09-40e8-b778-7f904d59846b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:46:00.006001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf02208-8e09-40e8-b778-7f904d59846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:46:00.028476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-6cf02208-8e09-40e8-b778-7f904d59846b None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:46:00.029735 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6cf02208-8e09-40e8-b778-7f904d59846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:46:00.040102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4396/8814] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:46:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:46:00.053877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:46:00.067827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264ffa7-852b-4566-bc68-12586f07b167 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:46:00.073921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:46:00.074617 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:46:00.074978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:46:00.075245 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:46:00.084308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:46:00.085929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:46:00.095930 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4419/8815] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:46:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:46:10.023540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:10.091632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=18 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:24.574290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258adcb-a733-4217-aa40-45e966acdc1f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:46:24.574823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258adcb-a733-4217-aa40-45e966acdc1f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:46:24.574823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258adcb-a733-4217-aa40-45e966acdc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:46:24.595510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e258adcb-a733-4217-aa40-45e966acdc1f None None] MFA Rules not processed for user `396fe9ce0f7542c382b01d1f5779c13f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:46:24.596771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e258adcb-a733-4217-aa40-45e966acdc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:46:24.604149 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4397/8816] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:46:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:46:34.590473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:49.265865 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:47:49.276439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cafd0f-1262-49a7-877b-76f217f089fd service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:47:49.280186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:47:49.280834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:47:49.281115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:47:49.281385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:47:49.283440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:47:49.321566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:47:49.323017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:47:49.337883 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4420/8817] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:47:50.542890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3479a-73f9-4237-8a46-dead01c069cd None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:47:50.543524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3479a-73f9-4237-8a46-dead01c069cd None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:47:50.543524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3479a-73f9-4237-8a46-dead01c069cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:47:50.564687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b4b3479a-73f9-4237-8a46-dead01c069cd None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:47:50.565900 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b4b3479a-73f9-4237-8a46-dead01c069cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:47:50.573594 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4398/8818] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Sep 27 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:47:50.581628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:47:50.590631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95868d-4e6f-430e-b572-2fc82ad91171 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:47:50.595034 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:47:50.595663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:47:50.596094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:47:50.596367 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:47:50.598103 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:47:50.615428 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:47:50.616921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:47:50.630480 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4421/8819] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:48:00.529371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3479a-73f9-4237-8a46-dead01c069cd None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:48:00.538039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3479a-73f9-4237-8a46-dead01c069cd None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:48:00.542088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cb0e0-24d0-4fa3-9023-88e2ca5d4c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:48:00.542681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cb0e0-24d0-4fa3-9023-88e2ca5d4c0e service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:48:00.542952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cb0e0-24d0-4fa3-9023-88e2ca5d4c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:48:00.543220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cb0e0-24d0-4fa3-9023-88e2ca5d4c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:48:00.545030 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-013cb0e0-24d0-4fa3-9023-88e2ca5d4c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:48:00.558224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cb0e0-24d0-4fa3-9023-88e2ca5d4c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:48:00.559669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cb0e0-24d0-4fa3-9023-88e2ca5d4c0e service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:48:00.571706 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4399/8820] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:48:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:48:00.623002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=20 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:00.874675 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:48:00.884903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536c27-cb26-4081-9916-03af41f16b69 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:48:00.889567 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:48:00.890189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:48:00.890465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:48:00.890709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:48:00.892650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:48:00.909419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:48:00.910953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:48:00.925398 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4422/8821] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:48:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:48:10.564471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=9 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:10.917525 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:17.173877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:48:17.174397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:48:17.174493 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:48:17.197860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] MFA Rules not processed for user `396fe9ce0f7542c382b01d1f5779c13f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:48:17.199316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:48:17.207728 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4400/8822] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:48:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:48:27.189296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:16.901926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:49:16.914340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaf95e-eb35-45df-8fb1-65d13d860ebe service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:49:16.919326 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:49:16.919933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:49:16.920211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:49:16.920477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:49:16.922634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:49:16.937493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:49:16.938928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:49:16.951712 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4423/8823] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:49:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:49:26.944453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:40.797371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:49:40.798839 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:49:40.821365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f2ca8-d2de-4c9c-b97a-c48672eab345 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:49:40.828819 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:49:40.829434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:49:40.829708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:49:40.829976 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:49:40.839047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:49:40.840690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:49:40.850469 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4401/8824] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:49:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:49:49.872997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:49:49.888805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6d2f2-cee9-4e34-aa12-d215b3bf7811 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:49:49.894613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:49:49.895225 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:49:49.895505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:49:49.895765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:49:49.907856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:49:49.909499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:49:49.923008 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4424/8825] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:49:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:49:50.846094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:55.206437 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:49:55.216571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28173a-d1cf-407a-b31a-e82534501258 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:49:55.220752 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:49:55.221374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:49:55.221648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:49:55.222220 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:49:55.224035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:49:55.236813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:49:55.238236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:49:55.249260 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4402/8826] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:49:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:49:55.661951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:49:55.678401 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4c71a-bd4d-4f3f-81b0-2ea162da39a0 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:49:55.684929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38f393-3ac8-4d80-b6a5-b7fa7b5f8b65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:49:55.685532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38f393-3ac8-4d80-b6a5-b7fa7b5f8b65 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:49:55.685812 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38f393-3ac8-4d80-b6a5-b7fa7b5f8b65 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:49:55.686082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38f393-3ac8-4d80-b6a5-b7fa7b5f8b65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:49:55.688727 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0f38f393-3ac8-4d80-b6a5-b7fa7b5f8b65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:49:55.705421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38f393-3ac8-4d80-b6a5-b7fa7b5f8b65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:49:55.707069 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38f393-3ac8-4d80-b6a5-b7fa7b5f8b65 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:49:55.721835 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4425/8827] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:49:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:50:02.236889 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:50:02.260017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b086aea-d56b-47b5-8daa-d0ead3a9bcfe service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:50:02.269307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:50:02.270173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:50:02.270818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:50:02.271256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:50:02.273224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:50:02.296740 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:50:02.298490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:50:02.324885 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4403/8828] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:50:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:50:05.713511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=12 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:12.305386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:12.621257 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd870d3a-5e36-4cb7-bc13-cbb330d4603c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:50:12.621745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd870d3a-5e36-4cb7-bc13-cbb330d4603c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:50:12.622128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd870d3a-5e36-4cb7-bc13-cbb330d4603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:50:12.642355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-bd870d3a-5e36-4cb7-bc13-cbb330d4603c None None] MFA Rules not processed for user `396fe9ce0f7542c382b01d1f5779c13f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:50:12.643750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-bd870d3a-5e36-4cb7-bc13-cbb330d4603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:50:12.651201 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4426/8829] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:50:12.975948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:50:12.985597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152b8d-897b-406f-91a0-2ff5434484ab service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:50:12.989631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:50:12.990227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:50:12.990510 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:50:12.990783 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:50:12.992574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:50:13.006705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:50:13.008158 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:50:13.018964 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4404/8830] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:50:13.372634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd870d3a-5e36-4cb7-bc13-cbb330d4603c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:50:13.387936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd870d3a-5e36-4cb7-bc13-cbb330d4603c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:50:13.393041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094c106-18f8-433a-8617-a06158875545 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:50:13.393734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c106-18f8-433a-8617-a06158875545 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:50:13.393983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c106-18f8-433a-8617-a06158875545 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:50:13.394255 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c106-18f8-433a-8617-a06158875545 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:50:13.405623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094c106-18f8-433a-8617-a06158875545 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:50:13.407204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094c106-18f8-433a-8617-a06158875545 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:50:13.417950 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4427/8831] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:50:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:50:23.012934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:23.414107 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:25.737294 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:50:25.755142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03644f9-0043-485f-90e7-fa9e4ea66c85 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:50:25.762006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:50:25.762648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:50:25.762912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:50:25.763186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:50:25.765076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:50:25.781291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:50:25.782878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:50:25.797810 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4405/8832] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:50:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:50:35.791466 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:58.265148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094c106-18f8-433a-8617-a06158875545 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:50:58.279205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094c106-18f8-433a-8617-a06158875545 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:50:58.285750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:50:58.286508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:50:58.286783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:50:58.287050 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:50:58.298995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:50:58.300825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:50:58.313895 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4428/8833] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:50:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:51:06.529080 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:06.545772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567316-6000-41ea-b865-ff0eaecc70d5 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:06.551529 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:06.552279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:06.552558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:06.552914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:06.562234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:06.563837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:06.575061 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4406/8834] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:51:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:51:06.841001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:06.855224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b6b08-6aa8-45a8-9707-95606dd7c10b service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:06.860854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:06.861438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:06.861729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:06.862003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:06.870708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:06.872274 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:06.882848 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4429/8835] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:51:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:51:16.570596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:16.879141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=12 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:44.717056 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:44.738451 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bc5c2-dd25-473b-bcbc-a54bb4fcce87 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:44.743234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:44.743954 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:44.744240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:44.744507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:44.779669 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:44.781145 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:44.791401 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4407/8836] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:51:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:51:47.755708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:47.824337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402903-3fb8-4a9f-8f19-a34b77cc0365 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:47.830434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:47.830987 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:47.832106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:47.832386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:47.832726 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:47.834573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:47.842484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f4168-c506-406d-a615-78779c9932b9 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:47.847875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:47.848702 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:47.848841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:47.849115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] PATH_INFO: `/v3/users/396fe9ce0f7542c382b01d1f5779c13f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:47.850243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=396fe9ce0f7542c382b01d1f5779c13f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:47.853051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:47.874837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:47.876613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:47.891814 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4430/8837] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:51:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:51:47.934060 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396fe9ce0f7542c382b01d1f5779c13f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:47.934918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396fe9ce0f7542c382b01d1f5779c13f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:47.942804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396fe9ce0f7542c382b01d1f5779c13f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:47.952179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396fe9ce0f7542c382b01d1f5779c13f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:47.965570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-541d4aa9-d99d-4886-9eeb-799ad9679b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396fe9ce0f7542c382b01d1f5779c13f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:47.971907 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4408/8838] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:51:47 2022] DELETE /identity/v3/users/396fe9ce0f7542c382b01d1f5779c13f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:51:48.154091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.171292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84eb2b-94ee-4626-94c1-f49d3cc5f0fa service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.232168 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.232818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.233085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.233353 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] PATH_INFO: `/v3/projects/32845fb8bd064f55bb99d01e5aeff6f3` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.234468 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32845fb8bd064f55bb99d01e5aeff6f3)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.236418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.286749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32845fb8bd064f55bb99d01e5aeff6f3', 'request_id': 'req-ffe1a915-f48b-4c12-9e92-894de1056a71'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:48.297714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32845fb8bd064f55bb99d01e5aeff6f3', 'request_id': 'req-ffe1a915-f48b-4c12-9e92-894de1056a71'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:48.305303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32845fb8bd064f55bb99d01e5aeff6f3', 'request_id': 'req-ffe1a915-f48b-4c12-9e92-894de1056a71'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:48.317294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] The token cache is being invalidate because project 32845fb8bd064f55bb99d01e5aeff6f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:51:48.317575 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:48.318281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-ffe1a915-f48b-4c12-9e92-894de1056a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:51:48.319752 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4431/8839] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:51:48 2022] DELETE /identity/v3/projects/32845fb8bd064f55bb99d01e5aeff6f3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:51:48.325102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa1a85-9a19-43a0-9742-43f48f8b7f86 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.325390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa1a85-9a19-43a0-9742-43f48f8b7f86 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.325671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa1a85-9a19-43a0-9742-43f48f8b7f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.346495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-76fa1a85-9a19-43a0-9742-43f48f8b7f86 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:51:48.347710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-76fa1a85-9a19-43a0-9742-43f48f8b7f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:48.414449 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4409/8840] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:51:48.423709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de13dc-f754-4ae7-adae-5d0bc6cf4aed None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.424009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de13dc-f754-4ae7-adae-5d0bc6cf4aed None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.424258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de13dc-f754-4ae7-adae-5d0bc6cf4aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.445357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b4de13dc-f754-4ae7-adae-5d0bc6cf4aed None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:51:48.446562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b4de13dc-f754-4ae7-adae-5d0bc6cf4aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:48.455923 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4432/8841] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:51:48.460512 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa1a85-9a19-43a0-9742-43f48f8b7f86 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.471127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa1a85-9a19-43a0-9742-43f48f8b7f86 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.475959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.476559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.476867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.477134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.477777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:51:48.478078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.479661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.491770 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4410/8842] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:51:48.496189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de13dc-f754-4ae7-adae-5d0bc6cf4aed None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.506707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de13dc-f754-4ae7-adae-5d0bc6cf4aed None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.512521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.513156 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.513430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.513699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.514329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.516344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.542493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:48.553147 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4433/8843] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:51:48 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:51:48.554346 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.564945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc651eb-0f5e-410c-be52-696d3c4bea11 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.577589 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:48.626650 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.627270 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.627549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.627823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.628456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.630432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.655467 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4411/8844] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:51:48 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:51:48.660686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.675815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f13a7-a228-49d1-a349-f8689cf2a917 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.680797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.681399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.681670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.681934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.682574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.684155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.694960 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4434/8845] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:51:48.701262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.718765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc240f0-8176-4bd1-8af3-8cebe4550b60 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.724930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ea863-b1c3-4bf9-9462-5f68b8f8bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.725596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ea863-b1c3-4bf9-9462-5f68b8f8bda3 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.725957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ea863-b1c3-4bf9-9462-5f68b8f8bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.726332 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ea863-b1c3-4bf9-9462-5f68b8f8bda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.726994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ea863-b1c3-4bf9-9462-5f68b8f8bda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.728593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ea863-b1c3-4bf9-9462-5f68b8f8bda3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.739967 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4412/8846] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:51:48.744419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.754716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15be7b-9811-45cd-bc03-94bcf0a7a0a0 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.759346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.759959 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.760248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.760513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] PATH_INFO: `/v3/projects/b4eda9ec9a0a4216a8b70d63470e6838/users/b9b3c7e824e74cbba45639dfed007e4d/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.776807 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4eda9ec9a0a4216a8b70d63470e6838, user_id=b9b3c7e824e74cbba45639dfed007e4d, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.779503 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.797362 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4435/8847] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:51:48 2022] PUT /identity/v3/projects/b4eda9ec9a0a4216a8b70d63470e6838/users/b9b3c7e824e74cbba45639dfed007e4d/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:51:48.802970 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfc8d9-a382-4fef-ae2b-4e8785dc7a58 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.803248 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfc8d9-a382-4fef-ae2b-4e8785dc7a58 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.803517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfc8d9-a382-4fef-ae2b-4e8785dc7a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.829462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b1bfc8d9-a382-4fef-ae2b-4e8785dc7a58 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:51:48.830627 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b1bfc8d9-a382-4fef-ae2b-4e8785dc7a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:48.895460 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4413/8848] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:51:48.905329 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:48.907294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:48.968520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542df13-2d30-4f7d-ba47-bad15d75effb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:48.973234 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50356a09-042b-4d95-8850-e26e83c92654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:48.973837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50356a09-042b-4d95-8850-e26e83c92654 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:48.974115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50356a09-042b-4d95-8850-e26e83c92654 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:48.974371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50356a09-042b-4d95-8850-e26e83c92654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:48.983464 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50356a09-042b-4d95-8850-e26e83c92654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:48.985411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50356a09-042b-4d95-8850-e26e83c92654 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:48.995118 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4436/8849] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:51:53.453687 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:53.454443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:53.454443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:53.486509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] MFA Rules not processed for user `b9b3c7e824e74cbba45639dfed007e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:51:53.487631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:53.560459 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4414/8850] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:51:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:51:53.570518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af566-081c-47a8-901a-8cf4e7cde215 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:53.571089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af566-081c-47a8-901a-8cf4e7cde215 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:53.571089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af566-081c-47a8-901a-8cf4e7cde215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:53.596765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-285af566-081c-47a8-901a-8cf4e7cde215 None None] MFA Rules not processed for user `b9b3c7e824e74cbba45639dfed007e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:51:53.598149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-285af566-081c-47a8-901a-8cf4e7cde215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:53.607308 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4437/8851] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:51:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:51:53.619663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:53.621399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:53.650584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:53.705756 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a20d9-0092-4319-82e9-8287bea17404 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:53.711625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a04054-c195-4d7b-92d1-69bf4545651c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:53.712258 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a04054-c195-4d7b-92d1-69bf4545651c service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:53.712536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a04054-c195-4d7b-92d1-69bf4545651c service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:53.712843 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a04054-c195-4d7b-92d1-69bf4545651c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:53.721906 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a04054-c195-4d7b-92d1-69bf4545651c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:53.723538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a04054-c195-4d7b-92d1-69bf4545651c service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:53.733482 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4415/8852] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:51:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:51:53.996017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af566-081c-47a8-901a-8cf4e7cde215 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:51:54.006526 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af566-081c-47a8-901a-8cf4e7cde215 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:51:54.010775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547529b-1cd2-4d9a-b81e-dac7ec993ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:51:54.011375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547529b-1cd2-4d9a-b81e-dac7ec993ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:51:54.011649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547529b-1cd2-4d9a-b81e-dac7ec993ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:51:54.011917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547529b-1cd2-4d9a-b81e-dac7ec993ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:51:54.013786 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5547529b-1cd2-4d9a-b81e-dac7ec993ec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:51:54.027975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547529b-1cd2-4d9a-b81e-dac7ec993ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:51:54.029452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547529b-1cd2-4d9a-b81e-dac7ec993ec8 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:51:54.041947 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4438/8853] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:51:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:03.729305 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=23 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:04.034040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=50,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:05.321070 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695ee56-65e4-455e-94cd-0362c7dda811 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:05.321568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695ee56-65e4-455e-94cd-0362c7dda811 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:05.321568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695ee56-65e4-455e-94cd-0362c7dda811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:05.343941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-2695ee56-65e4-455e-94cd-0362c7dda811 None None] MFA Rules not processed for user `b9b3c7e824e74cbba45639dfed007e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:05.345228 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-2695ee56-65e4-455e-94cd-0362c7dda811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:05.352922 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4416/8854] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:06.476419 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fe231-2880-4618-9731-12e5918ae768 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:06.477072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fe231-2880-4618-9731-12e5918ae768 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:06.477072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fe231-2880-4618-9731-12e5918ae768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:06.501936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-cb9fe231-2880-4618-9731-12e5918ae768 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:06.503379 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-cb9fe231-2880-4618-9731-12e5918ae768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:06.576240 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4439/8855] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Sep 27 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:06.590017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6c513-d12a-4ed5-9a40-0d0f47dac024 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:06.590515 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6c513-d12a-4ed5-9a40-0d0f47dac024 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:06.590629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6c513-d12a-4ed5-9a40-0d0f47dac024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:06.613705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a9b6c513-d12a-4ed5-9a40-0d0f47dac024 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:06.615057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a9b6c513-d12a-4ed5-9a40-0d0f47dac024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:06.625565 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4417/8856] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:06.631938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fe231-2880-4618-9731-12e5918ae768 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:06.643055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fe231-2880-4618-9731-12e5918ae768 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:06.650085 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:06.650689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:06.650963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:06.651230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:06.660946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:06.662885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:06.674279 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4440/8857] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:16.608160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:16.668588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=35 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:23.084354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:23.084972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:23.085189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:23.106611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:23.107762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:23.117935 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4418/8858] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Sep 27 10:52:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:23.131375 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:23.142221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9299a1-cda3-4073-96a9-01456a72c636 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:23.147562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:23.148163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:23.148433 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:23.148719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:23.157994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:23.159578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:23.170095 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4441/8859] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:27.474722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:27.476453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:27.494632 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa5983-b256-4385-b504-cc3773dece6a None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:27.500174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:27.500860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:27.501141 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:27.501413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:27.503186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:27.518924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:27.520868 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:27.538336 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4419/8860] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:27.546418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:27.559501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05664810-a9d8-4ca4-a149-4292f59d4727 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:27.566286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:27.566921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:27.567299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:27.567528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] PATH_INFO: `/v3/users/b9b3c7e824e74cbba45639dfed007e4d` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:27.569045 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b3c7e824e74cbba45639dfed007e4d)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:27.571033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:27.624094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b3c7e824e74cbba45639dfed007e4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.625007 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b3c7e824e74cbba45639dfed007e4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.633583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b3c7e824e74cbba45639dfed007e4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.641312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b3c7e824e74cbba45639dfed007e4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.642020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-26716552-e87b-4e92-aca9-2ccf86c3b45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b3c7e824e74cbba45639dfed007e4d'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.661275 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4442/8861] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:52:27 2022] DELETE /identity/v3/users/b9b3c7e824e74cbba45639dfed007e4d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:52:27.841717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:27.853413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61dfa-fb20-49b9-a1f6-498e50f3d958 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:27.914328 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:27.914964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:27.915237 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:27.915511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] PATH_INFO: `/v3/projects/b4eda9ec9a0a4216a8b70d63470e6838` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:27.916698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4eda9ec9a0a4216a8b70d63470e6838)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:27.918593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:27.966593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4eda9ec9a0a4216a8b70d63470e6838', 'request_id': 'req-55c7b123-a324-46cb-bd2e-faab620659a9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.972703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4eda9ec9a0a4216a8b70d63470e6838', 'request_id': 'req-55c7b123-a324-46cb-bd2e-faab620659a9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.983701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4eda9ec9a0a4216a8b70d63470e6838', 'request_id': 'req-55c7b123-a324-46cb-bd2e-faab620659a9'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.995131 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] The token cache is being invalidate because project b4eda9ec9a0a4216a8b70d63470e6838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:52:27.995426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.996090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-55c7b123-a324-46cb-bd2e-faab620659a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:27.997667 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4420/8862] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:52:27 2022] DELETE /identity/v3/projects/b4eda9ec9a0a4216a8b70d63470e6838 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:28.003106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74a48d-cf43-4282-9c78-49d6ffbd6d9d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.003436 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74a48d-cf43-4282-9c78-49d6ffbd6d9d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.003722 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74a48d-cf43-4282-9c78-49d6ffbd6d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.024851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ff74a48d-cf43-4282-9c78-49d6ffbd6d9d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:28.026072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ff74a48d-cf43-4282-9c78-49d6ffbd6d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:28.087645 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4443/8863] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:28.095918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e97ae2-6d52-4606-a5e1-cf5e47733a50 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.096213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e97ae2-6d52-4606-a5e1-cf5e47733a50 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.096496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e97ae2-6d52-4606-a5e1-cf5e47733a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.118606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-32e97ae2-6d52-4606-a5e1-cf5e47733a50 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:28.119853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-32e97ae2-6d52-4606-a5e1-cf5e47733a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:28.128190 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4421/8864] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:28.135797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74a48d-cf43-4282-9c78-49d6ffbd6d9d None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:28.150057 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74a48d-cf43-4282-9c78-49d6ffbd6d9d None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:28.155843 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:28.156438 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.156747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.157014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.157657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:52:28.157952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:28.159546 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:28.173198 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4444/8865] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:52:28.177780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e97ae2-6d52-4606-a5e1-cf5e47733a50 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:28.188053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e97ae2-6d52-4606-a5e1-cf5e47733a50 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:28.193465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:28.194077 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.194362 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.194630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.195330 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:28.197350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:28.228978 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4422/8866] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:52:28 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:28.233204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:28.243584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54a723-63e5-4343-906a-e8e619c74597 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:28.289855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:28.303977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:28.304574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.304882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.305146 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.305771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:28.307706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:28.330571 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4445/8867] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:52:28 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:28.334943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:28.345575 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186518-abac-4209-b7c2-9c4ae69f0d61 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:28.350586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:28.351206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.351485 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.351759 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.352410 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:28.354116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:28.366957 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4423/8868] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:28.371072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:28.381950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f498ce0-0e24-4dce-8af4-afd01b59e8ca admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:28.387189 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3632d-9687-4c0c-a8b3-4cd0f1ba5905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:28.387787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3632d-9687-4c0c-a8b3-4cd0f1ba5905 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.388060 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3632d-9687-4c0c-a8b3-4cd0f1ba5905 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.388321 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3632d-9687-4c0c-a8b3-4cd0f1ba5905 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.389046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3632d-9687-4c0c-a8b3-4cd0f1ba5905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:28.390689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3632d-9687-4c0c-a8b3-4cd0f1ba5905 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:28.402893 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4446/8869] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:28.407435 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:28.418898 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354e3e7-81ff-49bc-b9db-2a90be1d5e93 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:28.427002 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:28.427722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.428171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.428555 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] PATH_INFO: `/v3/projects/b8b8c3946daa46d3b4952e06e07bf098/users/cb3094c04aa84309a84f4a78d429322a/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.445888 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b8c3946daa46d3b4952e06e07bf098, user_id=cb3094c04aa84309a84f4a78d429322a, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:28.448558 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:28.463962 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4424/8870] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:52:28 2022] PUT /identity/v3/projects/b8b8c3946daa46d3b4952e06e07bf098/users/cb3094c04aa84309a84f4a78d429322a/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:28.475292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3faa33-b039-4de6-bb67-63ea8d737d0e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.475588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3faa33-b039-4de6-bb67-63ea8d737d0e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.475855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3faa33-b039-4de6-bb67-63ea8d737d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.505790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-3f3faa33-b039-4de6-bb67-63ea8d737d0e None None] MFA Rules not processed for user `cb3094c04aa84309a84f4a78d429322a`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:28.507028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f3faa33-b039-4de6-bb67-63ea8d737d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:28.578319 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4447/8871] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:28.588990 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:28.591017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:28.602603 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:28.655483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d49417-9e10-45a7-a76f-79d175941ff3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:28.659641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f3080-5964-46b0-9878-baa7b500e64e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:28.660253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f3080-5964-46b0-9878-baa7b500e64e service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:28.660527 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f3080-5964-46b0-9878-baa7b500e64e service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:28.660830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f3080-5964-46b0-9878-baa7b500e64e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:28.669666 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f3080-5964-46b0-9878-baa7b500e64e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:28.671117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f3080-5964-46b0-9878-baa7b500e64e service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:28.680327 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4425/8872] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 10:52:38.574926 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:38.676171 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=23 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:39.479565 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3faa33-b039-4de6-bb67-63ea8d737d0e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:39.545221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3faa33-b039-4de6-bb67-63ea8d737d0e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:39.549962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:39.550586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:39.550861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:39.551127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] PATH_INFO: `/v3/users/cb3094c04aa84309a84f4a78d429322a` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:39.552292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3094c04aa84309a84f4a78d429322a)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:39.554239 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:39.613429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3094c04aa84309a84f4a78d429322a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:39.614229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3094c04aa84309a84f4a78d429322a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:39.626229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3094c04aa84309a84f4a78d429322a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:39.635964 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3094c04aa84309a84f4a78d429322a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:39.636681 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3094c04aa84309a84f4a78d429322a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:39.654860 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4448/8873] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:52:39 2022] DELETE /identity/v3/users/cb3094c04aa84309a84f4a78d429322a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:39.659651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:39.660024 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:39.660394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:39.681878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:39.683160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:39.746206 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4426/8874] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:39.756376 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:39.758002 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:39.819177 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b22e1-ada1-464e-86f5-6cb9c6996368 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:39.823750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e3c0-1e15-4f43-9781-9e1144457195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:39.824358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e3c0-1e15-4f43-9781-9e1144457195 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:39.824636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e3c0-1e15-4f43-9781-9e1144457195 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:39.824940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e3c0-1e15-4f43-9781-9e1144457195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:39.835883 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e3c0-1e15-4f43-9781-9e1144457195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:39.837289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e3c0-1e15-4f43-9781-9e1144457195 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:39.847626 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4449/8875] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:40.031016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.032887 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:40.045875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901d55d-f21a-43ef-84ca-f1fb4e88d112 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.050862 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.051546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.051826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.052169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] PATH_INFO: `/v3/projects/b8b8c3946daa46d3b4952e06e07bf098` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.053549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8b8c3946daa46d3b4952e06e07bf098)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.055520 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.110844 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8b8c3946daa46d3b4952e06e07bf098', 'request_id': 'req-6487edce-aece-438f-ba2b-2825a1f3e97e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:40.120304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8b8c3946daa46d3b4952e06e07bf098', 'request_id': 'req-6487edce-aece-438f-ba2b-2825a1f3e97e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:40.129867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8b8c3946daa46d3b4952e06e07bf098', 'request_id': 'req-6487edce-aece-438f-ba2b-2825a1f3e97e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:40.141582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] The token cache is being invalidate because project b8b8c3946daa46d3b4952e06e07bf098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:52:40.141848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:40.142476 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6487edce-aece-438f-ba2b-2825a1f3e97e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:40.144021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4427/8876] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:52:40 2022] DELETE /identity/v3/projects/b8b8c3946daa46d3b4952e06e07bf098 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:40.149736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82703981-c4cb-4fe1-9efe-abdb8e338649 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.150012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82703981-c4cb-4fe1-9efe-abdb8e338649 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.150294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82703981-c4cb-4fe1-9efe-abdb8e338649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.171187 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-82703981-c4cb-4fe1-9efe-abdb8e338649 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:40.172405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-82703981-c4cb-4fe1-9efe-abdb8e338649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:40.191083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:40.236494 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4450/8877] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:40.245135 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0af7a-07a5-4c95-a180-dd10cb50c7e6 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.245502 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0af7a-07a5-4c95-a180-dd10cb50c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.245780 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0af7a-07a5-4c95-a180-dd10cb50c7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.268054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4ef0af7a-07a5-4c95-a180-dd10cb50c7e6 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:40.269369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4ef0af7a-07a5-4c95-a180-dd10cb50c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:40.278364 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4428/8878] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:40.282938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82703981-c4cb-4fe1-9efe-abdb8e338649 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.294188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82703981-c4cb-4fe1-9efe-abdb8e338649 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.300180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.302275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.302275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.302275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.302275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:52:40.304176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.305852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.318844 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4451/8879] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:52:40.323615 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0af7a-07a5-4c95-a180-dd10cb50c7e6 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.337507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0af7a-07a5-4c95-a180-dd10cb50c7e6 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.346658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.347586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.347891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.348217 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.349962 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.351913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.384420 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4429/8880] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:52:40 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:40.390400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.401477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a450fea-523a-43a8-84ee-77c1cf868cf2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.461528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.462151 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.462426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.462700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.463315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.465299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.492220 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4452/8881] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:52:40 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:40.493567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.504793 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba066-ddd4-4709-93d4-51453c04a26e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.510344 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.511087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.511368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.511649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.512276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.513926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.524617 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4430/8882] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:40.528861 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.542970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44015c5e-47b8-4a5c-9cd5-dc3794cf57e5 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.548113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837d2e1-ac2e-4a35-af0c-d14d8435ade2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.548749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837d2e1-ac2e-4a35-af0c-d14d8435ade2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.549021 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837d2e1-ac2e-4a35-af0c-d14d8435ade2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.549287 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837d2e1-ac2e-4a35-af0c-d14d8435ade2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.549923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837d2e1-ac2e-4a35-af0c-d14d8435ade2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.551551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837d2e1-ac2e-4a35-af0c-d14d8435ade2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.562233 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4453/8883] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:40.566574 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.577174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49aba84-4620-4c50-8e75-1a8fa2d87e45 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.582159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.582823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.583102 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.583537 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] PATH_INFO: `/v3/projects/240852206589456681bb1a27a74ab4f2/users/93743e2667944f5e932419a39d76aa9e/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.598680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=240852206589456681bb1a27a74ab4f2, user_id=93743e2667944f5e932419a39d76aa9e, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.601604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.619500 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4431/8884] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:52:40 2022] PUT /identity/v3/projects/240852206589456681bb1a27a74ab4f2/users/93743e2667944f5e932419a39d76aa9e/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:40.629921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe882420-cd52-4ed9-86c7-b245bb5e0a33 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.630708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe882420-cd52-4ed9-86c7-b245bb5e0a33 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.630708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe882420-cd52-4ed9-86c7-b245bb5e0a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.661476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-fe882420-cd52-4ed9-86c7-b245bb5e0a33 None None] MFA Rules not processed for user `93743e2667944f5e932419a39d76aa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:40.663351 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fe882420-cd52-4ed9-86c7-b245bb5e0a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:40.732325 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4454/8885] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:40.742641 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:40.744568 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:40.767849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:40.805983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690b7d6-d278-474b-9951-e38ac0430a9d admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:40.813216 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc3933-11ca-4aed-a451-b3a7b05a69d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:40.813864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc3933-11ca-4aed-a451-b3a7b05a69d9 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:40.814143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc3933-11ca-4aed-a451-b3a7b05a69d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:40.814417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc3933-11ca-4aed-a451-b3a7b05a69d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:40.824697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc3933-11ca-4aed-a451-b3a7b05a69d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:40.826075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc3933-11ca-4aed-a451-b3a7b05a69d9 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:40.837397 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4432/8886] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:44.822942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe882420-cd52-4ed9-86c7-b245bb5e0a33 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:44.847629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:44.895219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe882420-cd52-4ed9-86c7-b245bb5e0a33 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:44.899658 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:44.900297 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:44.900569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:44.900870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] PATH_INFO: `/v3/users/93743e2667944f5e932419a39d76aa9e` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:44.902129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93743e2667944f5e932419a39d76aa9e)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:44.904149 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:44.954400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93743e2667944f5e932419a39d76aa9e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:44.955054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93743e2667944f5e932419a39d76aa9e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:44.964398 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93743e2667944f5e932419a39d76aa9e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:44.972972 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93743e2667944f5e932419a39d76aa9e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:44.973685 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93743e2667944f5e932419a39d76aa9e'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:44.992888 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4455/8887] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 10:52:44 2022] DELETE /identity/v3/users/93743e2667944f5e932419a39d76aa9e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:44.997640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:44.998150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:44.998150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.018213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:45.019354 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:45.092991 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4433/8888] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:45.103221 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.104921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:45.183787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd30a0-69fc-4035-9bc7-42894d95c18a admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.188043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8bb8f-0243-4ce5-b6ef-a97e4a7f103b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.188677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8bb8f-0243-4ce5-b6ef-a97e4a7f103b service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.189001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8bb8f-0243-4ce5-b6ef-a97e4a7f103b service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.189227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8bb8f-0243-4ce5-b6ef-a97e4a7f103b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.199736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8bb8f-0243-4ce5-b6ef-a97e4a7f103b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.201248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8bb8f-0243-4ce5-b6ef-a97e4a7f103b service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.211522 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4456/8889] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:45.400993 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.402570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:45.416132 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee1ad0-9b86-493c-897c-8ec7a57af5c5 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.421747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.422459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.422812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.423122 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] PATH_INFO: `/v3/projects/240852206589456681bb1a27a74ab4f2` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.424409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240852206589456681bb1a27a74ab4f2)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.426429 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.478738 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240852206589456681bb1a27a74ab4f2', 'request_id': 'req-b16cc730-91d1-4dc1-a5a0-aad892fc9708'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:45.484995 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240852206589456681bb1a27a74ab4f2', 'request_id': 'req-b16cc730-91d1-4dc1-a5a0-aad892fc9708'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:45.495926 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240852206589456681bb1a27a74ab4f2', 'request_id': 'req-b16cc730-91d1-4dc1-a5a0-aad892fc9708'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:45.507597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] The token cache is being invalidate because project 240852206589456681bb1a27a74ab4f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 10:52:45.507883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:45.508495 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-b16cc730-91d1-4dc1-a5a0-aad892fc9708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 10:52:45.510033 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4434/8890] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:52:45 2022] DELETE /identity/v3/projects/240852206589456681bb1a27a74ab4f2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:45.516011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd301-5501-417d-95b8-bc2a7420af5c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.516292 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd301-5501-417d-95b8-bc2a7420af5c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.516563 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd301-5501-417d-95b8-bc2a7420af5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.537472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d3bdd301-5501-417d-95b8-bc2a7420af5c None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:45.538669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d3bdd301-5501-417d-95b8-bc2a7420af5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:45.565743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:45.605188 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4457/8891] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:45.613505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179086d8-d070-42d6-a7d7-072629b38d20 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.613799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179086d8-d070-42d6-a7d7-072629b38d20 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.614075 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179086d8-d070-42d6-a7d7-072629b38d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.636456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-179086d8-d070-42d6-a7d7-072629b38d20 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:45.637704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-179086d8-d070-42d6-a7d7-072629b38d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:45.646095 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4435/8892] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:45.650573 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdd301-5501-417d-95b8-bc2a7420af5c None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.660425 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdd301-5501-417d-95b8-bc2a7420af5c None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.665817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.666453 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.666725 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.666985 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.667647 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 10:52:45.667966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.669612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.681736 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4458/8893] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 10:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 10:52:45.686161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179086d8-d070-42d6-a7d7-072629b38d20 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.696053 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179086d8-d070-42d6-a7d7-072629b38d20 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.701041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.701631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.701908 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.702177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.702799 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.704703 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.734632 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4436/8894] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:52:45 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:45.739207 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.749721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97c6ec-629c-421d-9df8-b612f3c60af7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.808928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.809530 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.809805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.810082 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.810696 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.812632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.837310 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4459/8895] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:52:45 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:45.843700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.856606 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9c5b-725c-4015-855d-dcdab6e89c81 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.862394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.863003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.863276 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.863538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.864295 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.866041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.877389 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4437/8896] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:45.882827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.893484 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c078bf-cfbd-4d43-a1d9-b77df9bfe75d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.898801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295db86-717a-48e7-955b-fc2dc7f63be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.899305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295db86-717a-48e7-955b-fc2dc7f63be9 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.899578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295db86-717a-48e7-955b-fc2dc7f63be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.899847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295db86-717a-48e7-955b-fc2dc7f63be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.900462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295db86-717a-48e7-955b-fc2dc7f63be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.902148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295db86-717a-48e7-955b-fc2dc7f63be9 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.913966 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4460/8897] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:45.918188 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:45.928619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a941f-1b71-4f7f-88ca-5d491ac7454c admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:45.934243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:45.934886 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.935160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.935413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] PATH_INFO: `/v3/projects/714fa95d9be140018a54ece37d204047/users/906ca88c8f544940b06705711ae7f1af/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.937708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:45.952760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714fa95d9be140018a54ece37d204047, user_id=906ca88c8f544940b06705711ae7f1af, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:45.955986 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:45.970774 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4438/8898] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:52:45 2022] PUT /identity/v3/projects/714fa95d9be140018a54ece37d204047/users/906ca88c8f544940b06705711ae7f1af/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 10:52:45.976363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e5ce-a7d5-409c-bcb5-daa3e5253ef5 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:45.976645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e5ce-a7d5-409c-bcb5-daa3e5253ef5 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:45.976941 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e5ce-a7d5-409c-bcb5-daa3e5253ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:45.996143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0be3e5ce-a7d5-409c-bcb5-daa3e5253ef5 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:45.997380 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0be3e5ce-a7d5-409c-bcb5-daa3e5253ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:46.059982 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4461/8899] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:46.074127 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:46.077434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:46.151686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b2d0-0fb4-427c-9ac4-18d4b292e172 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:46.156823 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:46.157449 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:46.157729 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:46.158008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:46.169532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:46.170983 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:46.181342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4439/8900] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:50.191418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6f443-e8ff-4757-be67-6b6cc3b067bd None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:50.191975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6f443-e8ff-4757-be67-6b6cc3b067bd None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:50.191975 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6f443-e8ff-4757-be67-6b6cc3b067bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:50.219832 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f3a6f443-e8ff-4757-be67-6b6cc3b067bd None None] MFA Rules not processed for user `906ca88c8f544940b06705711ae7f1af`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:50.221188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f3a6f443-e8ff-4757-be67-6b6cc3b067bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:50.247730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:50.294140 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4462/8901] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:50.298924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:50.309869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dee3c-054c-4608-88f4-1bdc387d5f79 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:50.315017 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:50.315979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:50.316439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:50.316753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:50.317638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:50.319846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:50.351488 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4440/8902] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 10:52:50 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:50.355808 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6f443-e8ff-4757-be67-6b6cc3b067bd None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:50.367702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6f443-e8ff-4757-be67-6b6cc3b067bd None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:50.436053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:50.436686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:50.436983 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:50.437256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:50.437891 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:50.439851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:50.468464 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4463/8903] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 10:52:50 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 10:52:50.472797 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:50.484837 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277347c-a90f-40cf-82c0-9e90132ee85f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:50.498287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:50.505422 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:50.505761 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:50.506042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:50.506777 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:50.512651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:50.524928 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4441/8904] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:50.529345 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:50.541393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c5d0b-09ac-44a8-9872-edb9a20db475 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:50.547881 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:50.548710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:50.548989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:50.549262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] PATH_INFO: `/v3/projects/24ddff2c71274788890a7616bf867504/users/dcdd453708084a78907f35cacd13d1c0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:50.566075 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ddff2c71274788890a7616bf867504, user_id=dcdd453708084a78907f35cacd13d1c0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:50.568934 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:50.585153 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4464/8905] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:52:50 2022] PUT /identity/v3/projects/24ddff2c71274788890a7616bf867504/users/dcdd453708084a78907f35cacd13d1c0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:52:50.589475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:50.601961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d06a2-3bd7-4484-9c6d-c00eeb5b06ea admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:50.663774 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c48365-d1e2-465e-9388-b61877f878fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:50.664403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48365-d1e2-465e-9388-b61877f878fa admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:50.664698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48365-d1e2-465e-9388-b61877f878fa admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:50.664994 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48365-d1e2-465e-9388-b61877f878fa admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:50.665642 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c48365-d1e2-465e-9388-b61877f878fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:50.667286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c48365-d1e2-465e-9388-b61877f878fa admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:50.678456 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4442/8906] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 10:52:50.682677 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:50.694037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4b05-25d1-479d-9d20-17197b1754b4 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:50.699504 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77775acf-092d-45a9-bdbf-ad798f08c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:50.700180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775acf-092d-45a9-bdbf-ad798f08c695 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:50.700459 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775acf-092d-45a9-bdbf-ad798f08c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:50.700748 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775acf-092d-45a9-bdbf-ad798f08c695 admin admin] PATH_INFO: `/v3/projects/24ddff2c71274788890a7616bf867504/users/dcdd453708084a78907f35cacd13d1c0/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:50.702859 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77775acf-092d-45a9-bdbf-ad798f08c695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ddff2c71274788890a7616bf867504, user_id=dcdd453708084a78907f35cacd13d1c0, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:50.705576 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77775acf-092d-45a9-bdbf-ad798f08c695 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:50.716709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4465/8907] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 10:52:50 2022] PUT /identity/v3/projects/24ddff2c71274788890a7616bf867504/users/dcdd453708084a78907f35cacd13d1c0/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 10:52:55.361629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b26bdb-b66b-4085-83dc-c630f2179dc8 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:55.362225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b26bdb-b66b-4085-83dc-c630f2179dc8 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:55.362225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b26bdb-b66b-4085-83dc-c630f2179dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:55.392706 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-61b26bdb-b66b-4085-83dc-c630f2179dc8 None None] MFA Rules not processed for user `dcdd453708084a78907f35cacd13d1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:55.393924 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-61b26bdb-b66b-4085-83dc-c630f2179dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:55.437927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:55.470651 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4443/8908] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:52:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:55.504851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbfe51-5698-457e-907b-62b0fba5a75d None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:55.505539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbfe51-5698-457e-907b-62b0fba5a75d None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:55.505539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbfe51-5698-457e-907b-62b0fba5a75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:55.567392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4abbfe51-5698-457e-907b-62b0fba5a75d None None] MFA Rules not processed for user `ed91b2c83d57419dac50595f57fc3411`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:55.569032 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4abbfe51-5698-457e-907b-62b0fba5a75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:55.634951 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:55.646209 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4466/8909] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 27 10:52:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:55.652542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b26bdb-b66b-4085-83dc-c630f2179dc8 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:55.663018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b26bdb-b66b-4085-83dc-c630f2179dc8 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:55.668236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9237f-9b10-4a5a-a0a3-499031ec7983 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:55.669083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9237f-9b10-4a5a-a0a3-499031ec7983 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:55.669511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9237f-9b10-4a5a-a0a3-499031ec7983 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:55.669949 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9237f-9b10-4a5a-a0a3-499031ec7983 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:55.735465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9237f-9b10-4a5a-a0a3-499031ec7983 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:55.736948 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9237f-9b10-4a5a-a0a3-499031ec7983 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:55.753153 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4444/8910] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:52:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:52:55.953396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b3c0-dc9a-486e-99d6-718d703aaecf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:55.953683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b3c0-dc9a-486e-99d6-718d703aaecf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:55.954053 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b3c0-dc9a-486e-99d6-718d703aaecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:56.010399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5888b3c0-dc9a-486e-99d6-718d703aaecf None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:56.011684 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5888b3c0-dc9a-486e-99d6-718d703aaecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:56.089217 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4467/8911] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:52:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:52:56.145071 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892afd-4ede-44d5-90bd-5119c9354e84 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:56.145390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892afd-4ede-44d5-90bd-5119c9354e84 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:56.145714 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892afd-4ede-44d5-90bd-5119c9354e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:56.201192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-9e892afd-4ede-44d5-90bd-5119c9354e84 None None] MFA Rules not processed for user `c6863dfc2c0143a88060bbf5f0e146df`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:52:56.202430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9e892afd-4ede-44d5-90bd-5119c9354e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:52:56.281106 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4445/8912] 10.222.0.26 () {58 vars in 1040 bytes} [Tue Sep 27 10:52:56 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:52:56.288308 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888b3c0-dc9a-486e-99d6-718d703aaecf None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:52:56.302307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888b3c0-dc9a-486e-99d6-718d703aaecf None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:52:56.307532 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:52:56.308224 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:52:56.308500 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:52:56.308796 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:52:56.319540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:52:56.321258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:52:56.345434 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4468/8913] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:52:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 10:53:05.513253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:53:05.513830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:53:05.513830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:53:05.535273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] MFA Rules not processed for user `906ca88c8f544940b06705711ae7f1af`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:53:05.545124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:53:05.553631 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4446/8914] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:53:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:53:05.598386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:53:05.608275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da88b9-26c5-477e-933c-3ea555cd90e7 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:53:05.612755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32156fa-32a4-40f4-b302-d8342825fa59 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:53:05.613363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32156fa-32a4-40f4-b302-d8342825fa59 service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:53:05.613636 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32156fa-32a4-40f4-b302-d8342825fa59 service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:53:05.613904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32156fa-32a4-40f4-b302-d8342825fa59 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:53:05.616820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a32156fa-32a4-40f4-b302-d8342825fa59 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:53:05.702702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32156fa-32a4-40f4-b302-d8342825fa59 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:53:05.704147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32156fa-32a4-40f4-b302-d8342825fa59 service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:53:05.721614 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4469/8915] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:53:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:53:06.329319 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:53:06.331164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:53:06.366189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=100 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:06.393001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d3f1a-2841-472c-b148-4fc0c5b6edd1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:53:06.401648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:53:06.402434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:53:06.402753 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:53:06.403027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:53:06.404883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:53:06.422815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:53:06.424407 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:53:06.440212 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4447/8916] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:53:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:53:07.713710 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204002d-d398-46e1-b46b-4d4c16580391 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:53:07.714240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204002d-d398-46e1-b46b-4d4c16580391 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:53:07.714240 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204002d-d398-46e1-b46b-4d4c16580391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:53:07.735358 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1204002d-d398-46e1-b46b-4d4c16580391 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:53:07.736600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1204002d-d398-46e1-b46b-4d4c16580391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:53:07.746085 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4470/8917] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:53:07.758201 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:53:07.771026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a6ec8-47a0-4860-98a1-3fdad9b8a385 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:53:07.777119 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:53:07.777731 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:53:07.778004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:53:07.778274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:53:07.787629 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:53:07.789252 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:53:07.797686 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4448/8918] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:53:17.729426 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=86 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:17.794464 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=19 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:37.569138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204002d-d398-46e1-b46b-4d4c16580391 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:53:37.578954 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204002d-d398-46e1-b46b-4d4c16580391 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:53:37.582827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142b07-f128-47ba-ad29-560a2bf3d726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:53:37.583418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142b07-f128-47ba-ad29-560a2bf3d726 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:53:37.583689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142b07-f128-47ba-ad29-560a2bf3d726 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:53:37.583960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142b07-f128-47ba-ad29-560a2bf3d726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:53:37.593391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142b07-f128-47ba-ad29-560a2bf3d726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:53:37.594815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142b07-f128-47ba-ad29-560a2bf3d726 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:53:37.605107 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4471/8919] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:53:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:53:47.600318 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:54:20.882778 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:54:20.894643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2432ef9-6cb3-4ed7-88a2-af47be28ae55 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:54:20.901301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:54:20.902099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:54:20.902444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:54:20.902826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:54:20.911336 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:54:20.924572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:54:20.926041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:54:20.937444 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4449/8920] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:54:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:54:30.930782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:55:22.881366 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d471c9-a522-4888-a7e9-e11eb88e7489 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:55:22.881990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d471c9-a522-4888-a7e9-e11eb88e7489 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:55:22.881990 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d471c9-a522-4888-a7e9-e11eb88e7489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:55:22.946236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-a8d471c9-a522-4888-a7e9-e11eb88e7489 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:55:22.947457 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a8d471c9-a522-4888-a7e9-e11eb88e7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:55:22.965615 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4472/8921] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Sep 27 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:55:22.979748 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:55:23.001917 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07e5ba-65c3-4f57-8094-31212825f546 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:55:23.006256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4aa72-6277-476b-9e7a-6be172f8e3a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:55:23.006873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4aa72-6277-476b-9e7a-6be172f8e3a6 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:55:23.007125 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4aa72-6277-476b-9e7a-6be172f8e3a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:55:23.007417 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4aa72-6277-476b-9e7a-6be172f8e3a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:55:23.015191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4aa72-6277-476b-9e7a-6be172f8e3a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:55:23.016678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4aa72-6277-476b-9e7a-6be172f8e3a6 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:55:23.024551 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4450/8922] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:55:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:55:32.962363 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=21 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:55:33.021411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=10 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:35.138454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65371c3f-8648-491f-9a0e-8316926cd5d9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:56:35.139123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65371c3f-8648-491f-9a0e-8316926cd5d9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:56:35.139123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65371c3f-8648-491f-9a0e-8316926cd5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:56:35.180775 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-65371c3f-8648-491f-9a0e-8316926cd5d9 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:56:35.181905 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-65371c3f-8648-491f-9a0e-8316926cd5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:56:35.201665 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4473/8923] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:56:35.207848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349423d-4c27-4c53-aa28-7410225c9985 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:56:35.208395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349423d-4c27-4c53-aa28-7410225c9985 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:56:35.208395 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349423d-4c27-4c53-aa28-7410225c9985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:56:35.229087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b349423d-4c27-4c53-aa28-7410225c9985 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:56:35.230285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b349423d-4c27-4c53-aa28-7410225c9985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:56:35.239978 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4451/8924] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:56:35.252480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65371c3f-8648-491f-9a0e-8316926cd5d9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:56:35.263115 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65371c3f-8648-491f-9a0e-8316926cd5d9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:56:35.266956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd96e53-e11d-4261-931a-f83bb018fef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:56:35.267672 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd96e53-e11d-4261-931a-f83bb018fef9 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:56:35.268010 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd96e53-e11d-4261-931a-f83bb018fef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:56:35.268290 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd96e53-e11d-4261-931a-f83bb018fef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:56:35.279141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd96e53-e11d-4261-931a-f83bb018fef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:56:35.280586 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd96e53-e11d-4261-931a-f83bb018fef9 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:56:35.292176 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4474/8925] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:56:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:56:36.583697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e197-79ba-451a-98a9-39ee5d6abebb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:56:36.584298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e197-79ba-451a-98a9-39ee5d6abebb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:56:36.584298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e197-79ba-451a-98a9-39ee5d6abebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:56:36.604649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-8364e197-79ba-451a-98a9-39ee5d6abebb None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:56:36.606004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8364e197-79ba-451a-98a9-39ee5d6abebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:56:36.617408 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4452/8926] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:56:36.623579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c084c2-4525-4493-a36c-ba2f3561e25c None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:56:36.624065 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c084c2-4525-4493-a36c-ba2f3561e25c None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:56:36.624186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c084c2-4525-4493-a36c-ba2f3561e25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:56:36.644991 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e9c084c2-4525-4493-a36c-ba2f3561e25c None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:56:36.646056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e9c084c2-4525-4493-a36c-ba2f3561e25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:56:36.656784 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4475/8927] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:56:36.672004 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e197-79ba-451a-98a9-39ee5d6abebb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:56:36.682582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e197-79ba-451a-98a9-39ee5d6abebb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:56:36.686707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:56:36.687326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:56:36.687604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:56:36.687876 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:56:36.701114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:56:36.702590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:56:36.714038 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4453/8928] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:56:46.640038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=22 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:46.709275 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:57:57.858150 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8ee00-b8a9-4b1c-8ae7-377caeca1bd0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:57:57.858657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8ee00-b8a9-4b1c-8ae7-377caeca1bd0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:57:57.858657 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8ee00-b8a9-4b1c-8ae7-377caeca1bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:57:57.913474 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5fb8ee00-b8a9-4b1c-8ae7-377caeca1bd0 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:57:57.914857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5fb8ee00-b8a9-4b1c-8ae7-377caeca1bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:57:57.993315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4476/8929] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Sep 27 10:57:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:57:57.999722 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:57:58.012262 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3822484-b506-49df-adac-b9f7bc8afff2 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:57:58.019016 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:57:58.019617 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:57:58.019891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:57:58.020157 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:57:58.022313 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:57:58.035090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:57:58.036683 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:57:58.047021 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4454/8930] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:57:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:07.989970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=44 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:08.042821 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:08.232740 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8ee00-b8a9-4b1c-8ae7-377caeca1bd0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:08.243589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8ee00-b8a9-4b1c-8ae7-377caeca1bd0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:08.249994 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926b694-2b94-4911-b1fb-783b0a65de97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:08.250599 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b694-2b94-4911-b1fb-783b0a65de97 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:08.250874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b694-2b94-4911-b1fb-783b0a65de97 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:08.251162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b694-2b94-4911-b1fb-783b0a65de97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:08.253289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5926b694-2b94-4911-b1fb-783b0a65de97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:08.265695 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926b694-2b94-4911-b1fb-783b0a65de97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:08.267279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926b694-2b94-4911-b1fb-783b0a65de97 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:08.277714 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4477/8931] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:13.691655 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:13.717884 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94ccf3-d139-4704-a7d7-8687dcbd8034 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:13.724742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:13.725355 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:13.725636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:13.725912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:13.735427 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:13.737051 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:13.747562 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4455/8932] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:14.224779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f47c3-6475-4d62-b9c6-8e162ac6ec26 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:14.225275 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f47c3-6475-4d62-b9c6-8e162ac6ec26 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:14.225493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f47c3-6475-4d62-b9c6-8e162ac6ec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:14.263821 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-d27f47c3-6475-4d62-b9c6-8e162ac6ec26 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:14.265183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d27f47c3-6475-4d62-b9c6-8e162ac6ec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:14.275142 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4478/8933] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:58:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:58:14.295292 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:14.311855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a7c84-3c84-4b18-86c1-2de05db5d081 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:14.324260 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:14.325121 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:14.325350 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:14.325674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:14.336605 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:14.340384 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:14.352172 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4456/8934] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:15.410294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca03e-7aa4-41ac-84cf-8cac71181802 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:15.410728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca03e-7aa4-41ac-84cf-8cac71181802 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:15.410836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca03e-7aa4-41ac-84cf-8cac71181802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:15.432936 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-669ca03e-7aa4-41ac-84cf-8cac71181802 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:15.434136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-669ca03e-7aa4-41ac-84cf-8cac71181802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:15.444278 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4479/8935] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:58:15.458612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:15.470788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e75f53-eae0-4958-bc35-f08e1a4eea29 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:15.479094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:15.480001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:15.480483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:15.480987 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:15.491168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:15.493227 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:15.506816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4457/8936] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:21.817384 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78564c-e81a-4e78-ba4d-bf6fdd4349f2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:21.817864 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78564c-e81a-4e78-ba4d-bf6fdd4349f2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:21.817980 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78564c-e81a-4e78-ba4d-bf6fdd4349f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:21.837896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6e78564c-e81a-4e78-ba4d-bf6fdd4349f2 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:21.839217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6e78564c-e81a-4e78-ba4d-bf6fdd4349f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:21.846939 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4480/8937] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 10:58:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:58:21.889042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:21.901066 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b75e31-0a4c-4623-9abc-cc0095d50473 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:21.907184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:21.907786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:21.908072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:21.908697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:21.917808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:21.919424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:21.928129 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4458/8938] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:58:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:58:22.513785 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c5555-979b-4c50-ba7e-46d2658099a2 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:22.514071 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c5555-979b-4c50-ba7e-46d2658099a2 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:22.514346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c5555-979b-4c50-ba7e-46d2658099a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:22.534771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2f7c5555-979b-4c50-ba7e-46d2658099a2 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:22.536003 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f7c5555-979b-4c50-ba7e-46d2658099a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:22.543835 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4481/8939] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:58:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:58:22.556134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:22.566984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b509a54-7812-4ac8-b7b5-d4e39f0d1a71 service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:22.573020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844c278-082e-4587-b2fb-0db7593bb6fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:22.574099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844c278-082e-4587-b2fb-0db7593bb6fa service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:22.574099 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844c278-082e-4587-b2fb-0db7593bb6fa service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:22.574345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844c278-082e-4587-b2fb-0db7593bb6fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:22.583482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844c278-082e-4587-b2fb-0db7593bb6fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:22.585096 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844c278-082e-4587-b2fb-0db7593bb6fa service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:22.593503 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4459/8940] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:58:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:58:31.222200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e07b9-1b71-48dc-a7b3-8bf2c72ed42a None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:31.222805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e07b9-1b71-48dc-a7b3-8bf2c72ed42a None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:31.222805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e07b9-1b71-48dc-a7b3-8bf2c72ed42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:31.247369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e49e07b9-1b71-48dc-a7b3-8bf2c72ed42a None None] MFA Rules not processed for user `906ca88c8f544940b06705711ae7f1af`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:31.248591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e49e07b9-1b71-48dc-a7b3-8bf2c72ed42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:31.256790 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4482/8941] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 10:58:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:58:31.517338 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43fd0-2127-40f0-9250-688b9da8305c None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:31.517922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43fd0-2127-40f0-9250-688b9da8305c None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:31.517922 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43fd0-2127-40f0-9250-688b9da8305c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:31.538443 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-05e43fd0-2127-40f0-9250-688b9da8305c None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:31.539661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-05e43fd0-2127-40f0-9250-688b9da8305c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:31.546936 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4460/8942] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:58:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:58:31.558904 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e07b9-1b71-48dc-a7b3-8bf2c72ed42a None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:31.569730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e07b9-1b71-48dc-a7b3-8bf2c72ed42a None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:31.574051 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66a4f-9d33-4650-922c-da58eb396ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:31.574670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66a4f-9d33-4650-922c-da58eb396ccf service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:31.574949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66a4f-9d33-4650-922c-da58eb396ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:31.575219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66a4f-9d33-4650-922c-da58eb396ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:31.582545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66a4f-9d33-4650-922c-da58eb396ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:31.583978 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66a4f-9d33-4650-922c-da58eb396ccf service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:31.592000 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4483/8943] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:31.931640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e43fd0-2127-40f0-9250-688b9da8305c None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:31.945397 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e43fd0-2127-40f0-9250-688b9da8305c None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:31.950833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:31.951448 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:31.951723 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:31.951991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:31.953795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:31.966684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:31.968291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:31.980279 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4461/8944] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:33.248231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf5d7-f49f-4d78-9655-5af8504132cf None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:33.249399 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf5d7-f49f-4d78-9655-5af8504132cf None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:33.249851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf5d7-f49f-4d78-9655-5af8504132cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:33.277855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-30cdf5d7-f49f-4d78-9655-5af8504132cf None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:33.279179 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-30cdf5d7-f49f-4d78-9655-5af8504132cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:33.290821 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4484/8945] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 10:58:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:58:33.305012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:33.317196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e15d9-0d99-4651-a67f-4e633fa01300 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:33.322953 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:33.323573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:33.323854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:33.324133 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:33.333870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:33.335471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:33.344366 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4462/8946] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 10:58:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 10:58:40.735531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19c85-1175-4de1-931a-c1b949d289b1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:40.736159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19c85-1175-4de1-931a-c1b949d289b1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:40.736159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19c85-1175-4de1-931a-c1b949d289b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:40.758514 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=36 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:40.772702 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-0ae19c85-1175-4de1-931a-c1b949d289b1 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:40.773956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0ae19c85-1175-4de1-931a-c1b949d289b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:40.788631 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4485/8947] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Sep 27 10:58:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 10:58:40.803340 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:40.808234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=49 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:40.815612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc6789-93c4-47e4-ba91-fd0ccb9abcdf service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:40.819612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:40.820224 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:40.820534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:40.820849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:40.828347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:40.829808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:40.837905 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4463/8948] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:46.623750 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedf229-3b65-4335-abf7-7c78e5e7c052 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:46.624291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedf229-3b65-4335-abf7-7c78e5e7c052 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:46.624291 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedf229-3b65-4335-abf7-7c78e5e7c052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:46.645933 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-5eedf229-3b65-4335-abf7-7c78e5e7c052 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 10:58:46.647159 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5eedf229-3b65-4335-abf7-7c78e5e7c052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 10:58:46.657340 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4486/8949] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 10:58:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 10:58:46.675161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:46.701020 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedce0c9-a07d-4e8d-9801-7db08020234b service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:46.706546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79c3ba-71e4-4cc9-889b-74f6d6ab2dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:46.707253 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79c3ba-71e4-4cc9-889b-74f6d6ab2dc8 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:46.707534 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79c3ba-71e4-4cc9-889b-74f6d6ab2dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:46.707809 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79c3ba-71e4-4cc9-889b-74f6d6ab2dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:46.717117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79c3ba-71e4-4cc9-889b-74f6d6ab2dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:46.718697 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79c3ba-71e4-4cc9-889b-74f6d6ab2dc8 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:46.729031 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4464/8950] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:54.960676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedf229-3b65-4335-abf7-7c78e5e7c052 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 10:58:54.970659 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedf229-3b65-4335-abf7-7c78e5e7c052 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 10:58:54.974602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 10:58:54.975204 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 10:58:54.975460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 10:58:54.975731 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 10:58:54.984856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 10:58:54.986370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 10:58:54.995971 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4487/8951] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 10:58:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 10:58:56.724913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=11 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:04.992104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=11 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:14.735477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d022-8e5f-49e8-9e27-78dd2a55e7de None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:14.735991 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d022-8e5f-49e8-9e27-78dd2a55e7de None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:14.736105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d022-8e5f-49e8-9e27-78dd2a55e7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:14.757586 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-d980d022-8e5f-49e8-9e27-78dd2a55e7de None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:00:14.759191 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-d980d022-8e5f-49e8-9e27-78dd2a55e7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:14.797803 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4465/8952] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 11:00:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:00:14.840572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:14.853591 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4f1c1-a0b1-4324-aea8-2d2ae43c40ac service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:14.859818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e753-595d-4a94-9791-ff8d047f2d68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:14.860417 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e753-595d-4a94-9791-ff8d047f2d68 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:14.860732 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e753-595d-4a94-9791-ff8d047f2d68 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:14.861005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e753-595d-4a94-9791-ff8d047f2d68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:14.870391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e753-595d-4a94-9791-ff8d047f2d68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:14.871925 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e753-595d-4a94-9791-ff8d047f2d68 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:14.880258 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4488/8953] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:00:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:00:15.265796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980d022-8e5f-49e8-9e27-78dd2a55e7de None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:15.278630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980d022-8e5f-49e8-9e27-78dd2a55e7de None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:15.283087 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:15.283686 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:15.283963 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:15.284232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:15.358408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:15.359866 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:15.380710 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4466/8954] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:00:18.625108 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad3e1-b17e-411d-88c7-1345da0c0734 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:18.625588 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad3e1-b17e-411d-88c7-1345da0c0734 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:18.625694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad3e1-b17e-411d-88c7-1345da0c0734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:18.646579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f55ad3e1-b17e-411d-88c7-1345da0c0734 None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:00:18.647784 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f55ad3e1-b17e-411d-88c7-1345da0c0734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:18.657832 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4489/8955] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 11:00:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:00:18.671084 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:18.673111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:18.692927 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa1e59-8eaa-4a51-bee6-ce02c9f0420c service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:18.699189 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:18.699830 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:18.700108 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:18.700387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:18.710117 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:18.711772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:18.722387 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4467/8956] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:00:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:00:22.022068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad3e1-b17e-411d-88c7-1345da0c0734 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:22.045584 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad3e1-b17e-411d-88c7-1345da0c0734 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:22.050508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:22.051162 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:22.051442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:22.051711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] PATH_INFO: `/v3/users/906ca88c8f544940b06705711ae7f1af` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:22.053040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=906ca88c8f544940b06705711ae7f1af)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:22.054971 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:22.114733 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906ca88c8f544940b06705711ae7f1af'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.120547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906ca88c8f544940b06705711ae7f1af'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.130760 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906ca88c8f544940b06705711ae7f1af'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.138598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906ca88c8f544940b06705711ae7f1af'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.139286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906ca88c8f544940b06705711ae7f1af'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.157248 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4490/8957] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:00:22 2022] DELETE /identity/v3/users/906ca88c8f544940b06705711ae7f1af => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:00:22.349800 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:22.367826 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f33d7-4646-4b74-a5d1-80894fe07513 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:22.431186 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:22.431822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:22.432105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:22.432371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] PATH_INFO: `/v3/projects/714fa95d9be140018a54ece37d204047` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:22.433570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=714fa95d9be140018a54ece37d204047)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:22.435556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:22.489023 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714fa95d9be140018a54ece37d204047', 'request_id': 'req-0b8913af-1690-469d-b1b9-06759483b36e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.495151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714fa95d9be140018a54ece37d204047', 'request_id': 'req-0b8913af-1690-469d-b1b9-06759483b36e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.498966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:22.504569 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714fa95d9be140018a54ece37d204047', 'request_id': 'req-0b8913af-1690-469d-b1b9-06759483b36e'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.529082 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] The token cache is being invalidate because project 714fa95d9be140018a54ece37d204047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:00:22.529353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.530059 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.531639 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4468/8958] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:00:22 2022] DELETE /identity/v3/projects/714fa95d9be140018a54ece37d204047 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:00:22.536333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:22.537817 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:22.551298 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624839-1025-4ab3-ba48-5b82fb23e9eb admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:22.608332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:22.609047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:22.609327 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:22.609601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] PATH_INFO: `/v3/users/dcdd453708084a78907f35cacd13d1c0` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:22.610749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcdd453708084a78907f35cacd13d1c0)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:22.612724 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:22.664083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcdd453708084a78907f35cacd13d1c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.664827 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcdd453708084a78907f35cacd13d1c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.674751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcdd453708084a78907f35cacd13d1c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.675986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:22.682895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcdd453708084a78907f35cacd13d1c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.684012 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a95f7eca-9c69-44f5-b141-7165e4a8ad59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcdd453708084a78907f35cacd13d1c0'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:22.702228 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4491/8959] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:00:22 2022] DELETE /identity/v3/users/dcdd453708084a78907f35cacd13d1c0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 11:00:22.885581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:22.887311 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:22.901318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8913af-1690-469d-b1b9-06759483b36e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:22.962877 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:22.963517 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:22.963796 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:22.964068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] PATH_INFO: `/v3/projects/24ddff2c71274788890a7616bf867504` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:22.965232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24ddff2c71274788890a7616bf867504)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:22.967151 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.018046 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24ddff2c71274788890a7616bf867504', 'request_id': 'req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:23.024490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24ddff2c71274788890a7616bf867504', 'request_id': 'req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:23.034137 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24ddff2c71274788890a7616bf867504', 'request_id': 'req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:23.045653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] The token cache is being invalidate because project 24ddff2c71274788890a7616bf867504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:00:23.046621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:23.047331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-145f0d75-c289-4ca0-b1b9-7dd5827b35f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:00:23.049414 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4469/8960] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:00:22 2022] DELETE /identity/v3/projects/24ddff2c71274788890a7616bf867504 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:00:23.054668 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21aa4-06eb-4b01-af7e-2b6a135afa19 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.055272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21aa4-06eb-4b01-af7e-2b6a135afa19 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.055272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21aa4-06eb-4b01-af7e-2b6a135afa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.109009 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-32b21aa4-06eb-4b01-af7e-2b6a135afa19 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:00:23.110246 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-32b21aa4-06eb-4b01-af7e-2b6a135afa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:23.177649 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4492/8961] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 11:00:23.186025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612176e0-fffb-414c-a761-2f2ecb6e220b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.186591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612176e0-fffb-414c-a761-2f2ecb6e220b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.187150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612176e0-fffb-414c-a761-2f2ecb6e220b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.208524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-612176e0-fffb-414c-a761-2f2ecb6e220b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:00:23.209852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-612176e0-fffb-414c-a761-2f2ecb6e220b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:23.222795 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4470/8962] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:00:23.227616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b21aa4-06eb-4b01-af7e-2b6a135afa19 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.237929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b21aa4-06eb-4b01-af7e-2b6a135afa19 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.243370 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.243992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.244299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.244539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] PATH_INFO: `/v3/domains` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.245215 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 11:00:23.245531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.247109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.258881 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4493/8963] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 11:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 11:00:23.263556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612176e0-fffb-414c-a761-2f2ecb6e220b None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.276475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612176e0-fffb-414c-a761-2f2ecb6e220b None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.285240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.286062 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.286375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.286663 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.287334 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.290038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.334314 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4471/8964] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 11:00:23 2022] POST /identity/v3/projects => generated 393 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:00:23.339161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.350790 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173700f-80c1-4c22-887f-d60221cca641 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.413970 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.414667 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.414957 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.415233 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.415875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.417869 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.441378 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4494/8965] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 11:00:23 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:00:23.445602 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.456801 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1dc2dc-8acc-4eb0-b25e-bf2024001fb6 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.462710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.463316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.463590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.464242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.464961 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.466556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.477030 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4472/8966] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:00:23.481286 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.491707 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615451b-f7e5-45ee-8f15-09849d1deda2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.496787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b2ed-d3e6-43fd-8620-4da4629eef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.497352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b2ed-d3e6-43fd-8620-4da4629eef9d admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.497683 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b2ed-d3e6-43fd-8620-4da4629eef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.497960 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b2ed-d3e6-43fd-8620-4da4629eef9d admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.498607 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b2ed-d3e6-43fd-8620-4da4629eef9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.500208 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b2ed-d3e6-43fd-8620-4da4629eef9d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.511369 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4495/8967] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:00:23.515675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.527238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a04abf-2184-40a1-b3d2-3f5e091c8675 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.533446 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.534105 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.534383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.534653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] PATH_INFO: `/v3/projects/55b42a416dd942e780e533e4c6f2a8b4/users/3ec60ad271544bc99d7f394b9b06dd88/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.550614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b42a416dd942e780e533e4c6f2a8b4, user_id=3ec60ad271544bc99d7f394b9b06dd88, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.553329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.555852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:23.572441 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4473/8968] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:00:23 2022] PUT /identity/v3/projects/55b42a416dd942e780e533e4c6f2a8b4/users/3ec60ad271544bc99d7f394b9b06dd88/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:00:23.587833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c77fbe-387a-4799-a084-1dbad8d8be43 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.588092 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c77fbe-387a-4799-a084-1dbad8d8be43 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.588392 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c77fbe-387a-4799-a084-1dbad8d8be43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.619822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-68c77fbe-387a-4799-a084-1dbad8d8be43 None None] MFA Rules not processed for user `3ec60ad271544bc99d7f394b9b06dd88`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:00:23.621089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-68c77fbe-387a-4799-a084-1dbad8d8be43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:23.625236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:23.691385 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4496/8969] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:00:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:00:23.696608 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.761647 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fac6d-ec04-43cf-9d87-fd468d8b2f4e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.767321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.767937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.768215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.768499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.769176 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.771107 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.802699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4474/8970] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 11:00:23 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:00:23.807031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c77fbe-387a-4799-a084-1dbad8d8be43 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.818100 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c77fbe-387a-4799-a084-1dbad8d8be43 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.877228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.877851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.878132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.878403 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] PATH_INFO: `/v3/users` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.879018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.880976 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.908102 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4497/8971] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 11:00:23 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:00:23.912302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.923624 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb136b2a-6b7c-4ce9-b666-061444837c3f admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.929612 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.930243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.930518 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.930788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.931398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.933040 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:23.944342 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4475/8972] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:00:23.948875 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:23.959230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b493475-0502-40b6-96cb-5d7391cafc22 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:23.964346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:23.965064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:23.965340 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:23.965632 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] PATH_INFO: `/v3/projects/bda794173bca4e6588a47120125a4a68/users/ade37a2bb2c240ba935ba974ecc3f31d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:23.984068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda794173bca4e6588a47120125a4a68, user_id=ade37a2bb2c240ba935ba974ecc3f31d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:23.986783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:24.002584 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4498/8973] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:00:23 2022] PUT /identity/v3/projects/bda794173bca4e6588a47120125a4a68/users/ade37a2bb2c240ba935ba974ecc3f31d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:00:24.006720 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:24.017323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272434-9991-453a-9d29-c729f92d29d0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:24.074519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35350dcd-e75b-488e-959e-681dbf00cb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:24.075047 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350dcd-e75b-488e-959e-681dbf00cb82 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:24.075323 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350dcd-e75b-488e-959e-681dbf00cb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:24.075593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350dcd-e75b-488e-959e-681dbf00cb82 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:24.076267 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35350dcd-e75b-488e-959e-681dbf00cb82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:24.077873 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35350dcd-e75b-488e-959e-681dbf00cb82 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:24.087518 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4476/8974] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:00:24.091915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:24.102589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfb968-e158-43a8-949c-0ad911132c78 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:24.108262 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:24.108953 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:24.109229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:24.109495 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] PATH_INFO: `/v3/projects/bda794173bca4e6588a47120125a4a68/users/ade37a2bb2c240ba935ba974ecc3f31d/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:24.111856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda794173bca4e6588a47120125a4a68, user_id=ade37a2bb2c240ba935ba974ecc3f31d, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:24.114556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:24.126281 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4499/8975] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:00:24 2022] PUT /identity/v3/projects/bda794173bca4e6588a47120125a4a68/users/ade37a2bb2c240ba935ba974ecc3f31d/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:00:24.136690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:24.137025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:24.137302 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:24.166383 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] MFA Rules not processed for user `ade37a2bb2c240ba935ba974ecc3f31d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:00:24.167572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:24.220142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:24.240802 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4477/8976] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:00:24 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:00:24.255687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:24.257519 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:24.316193 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:24.337001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09f5d6-166e-43c6-b860-7cb9a76c508d admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:24.343260 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca149-e51c-4025-8e47-7668cfa91f55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:24.343874 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca149-e51c-4025-8e47-7668cfa91f55 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:24.344152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca149-e51c-4025-8e47-7668cfa91f55 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:24.344422 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca149-e51c-4025-8e47-7668cfa91f55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:24.408248 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdca149-e51c-4025-8e47-7668cfa91f55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:24.409884 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdca149-e51c-4025-8e47-7668cfa91f55 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:24.419261 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4500/8977] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:00:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:00:27.474535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:27.476057 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:27.535651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20ee78-b8ea-42c5-a196-eaf7d6300e68 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:27.540000 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:27.540616 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:27.541470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:27.541470 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:27.551421 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:27.552907 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:27.563882 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4478/8978] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:00:27 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:00:30.591852 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:30.592320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:30.592515 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:30.619231 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] MFA Rules not processed for user `3ec60ad271544bc99d7f394b9b06dd88`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:00:30.621837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:30.630510 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4501/8979] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:00:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:00:31.439980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:31.441483 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:31.537179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f4585-26f1-4aea-84a0-cff2e4941719 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:31.543564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:31.544440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:31.544933 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:31.545208 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:31.547169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:31.560232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:31.561855 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:31.573333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4479/8980] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:00:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:00:33.611369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:00:33.613856 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:33.628981 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35a6e5-ea82-4eee-9ea3-eaa23a2a411e None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:00:33.633006 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:00:33.633628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:00:33.633931 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:00:33.634199 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:00:33.636331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:00:33.646893 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:00:33.648706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:00:33.658367 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4502/8981] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:00:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:00:41.569712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=63 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:43.654111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=39 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:49.758918 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:01:49.781539 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cad657-54f5-4140-8c97-9dda1d99db5a service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:01:49.785831 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d411168-5816-4b66-b855-365bd0bff4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:01:49.786453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d411168-5816-4b66-b855-365bd0bff4db service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:01:49.786743 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d411168-5816-4b66-b855-365bd0bff4db service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:01:49.787003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d411168-5816-4b66-b855-365bd0bff4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:01:49.788938 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1d411168-5816-4b66-b855-365bd0bff4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:01:49.845138 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d411168-5816-4b66-b855-365bd0bff4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:01:49.846551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d411168-5816-4b66-b855-365bd0bff4db service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:01:49.857659 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4480/8982] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:01:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:01:51.055682 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:01:51.076783 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6713b-3a37-4961-b995-277aeec9788d service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:01:51.081018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:01:51.081621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:01:51.081896 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:01:51.082160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:01:51.084195 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:01:51.098035 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:01:51.099463 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:01:51.110377 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4503/8983] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:01:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:01:59.852904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=28 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:01.105842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=11 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:07.082807 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b32ba-7a74-4e6e-930c-aadd1b8f3b6f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:02:07.083365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b32ba-7a74-4e6e-930c-aadd1b8f3b6f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:02:07.083365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b32ba-7a74-4e6e-930c-aadd1b8f3b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:02:07.104484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-319b32ba-7a74-4e6e-930c-aadd1b8f3b6f None None] MFA Rules not processed for user `19f8118e581143ffb079e2b1aea3870d`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:02:07.105733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-319b32ba-7a74-4e6e-930c-aadd1b8f3b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:02:07.115613 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4481/8984] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:02:07.127218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:02:07.139040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d88cfd-292a-4959-99a2-b4fdf3a9b229 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:02:07.144842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d0f0-be9e-439f-85a0-8179dde7a5de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:02:07.145616 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d0f0-be9e-439f-85a0-8179dde7a5de service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:02:07.145745 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d0f0-be9e-439f-85a0-8179dde7a5de service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:02:07.146056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d0f0-be9e-439f-85a0-8179dde7a5de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:02:07.158104 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0d0f0-be9e-439f-85a0-8179dde7a5de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:02:07.159757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0d0f0-be9e-439f-85a0-8179dde7a5de service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:02:07.171487 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4504/8985] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:02:12.977896 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c0d0d-849f-42ed-be7e-0c5e6754e606 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:02:12.978439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c0d0d-849f-42ed-be7e-0c5e6754e606 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:02:12.978439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c0d0d-849f-42ed-be7e-0c5e6754e606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:02:13.000067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ed2c0d0d-849f-42ed-be7e-0c5e6754e606 None None] MFA Rules not processed for user `3ec60ad271544bc99d7f394b9b06dd88`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:02:13.001578 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ed2c0d0d-849f-42ed-be7e-0c5e6754e606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:02:13.009177 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4482/8986] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:02:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:02:13.417828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f42ec-cc0b-4dd6-9f2b-257b5ee945ea None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:02:13.418847 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f42ec-cc0b-4dd6-9f2b-257b5ee945ea None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:02:13.419429 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f42ec-cc0b-4dd6-9f2b-257b5ee945ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:02:13.448527 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-212f42ec-cc0b-4dd6-9f2b-257b5ee945ea None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:02:13.449945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-212f42ec-cc0b-4dd6-9f2b-257b5ee945ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:02:13.459871 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4505/8987] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 11:02:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:02:13.470222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c0d0d-849f-42ed-be7e-0c5e6754e606 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:02:13.481685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c0d0d-849f-42ed-be7e-0c5e6754e606 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:02:13.487150 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:02:13.487822 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:02:13.488093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:02:13.488358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:02:13.500925 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:02:13.502701 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:02:13.513233 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4483/8988] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:02:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:02:23.443119 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:23.508337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:59.849249 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f42ec-cc0b-4dd6-9f2b-257b5ee945ea None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:02:59.862634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f42ec-cc0b-4dd6-9f2b-257b5ee945ea None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:02:59.868574 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:02:59.869305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:02:59.869646 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:02:59.869992 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:02:59.871818 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:02:59.884582 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:02:59.886226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:02:59.896986 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4506/8989] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:02:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:03:09.892408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:16.192507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:03:16.197914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:03:16.217684 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615f05c-f605-4ea9-a293-c793060f5c1c service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:03:16.225651 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:03:16.226372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:03:16.226631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:03:16.226904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:03:16.229550 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:03:16.247509 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:03:16.249178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:03:16.263613 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4484/8990] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:03:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:03:26.258622 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:47.070899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:03:47.072721 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:03:47.092047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b563753-5513-42c8-b33d-a073628b123e service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:03:47.100020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa130ff8-9cd0-4e65-b9e3-d1677a9485c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:03:47.100649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa130ff8-9cd0-4e65-b9e3-d1677a9485c5 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:03:47.100946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa130ff8-9cd0-4e65-b9e3-d1677a9485c5 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:03:47.101212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa130ff8-9cd0-4e65-b9e3-d1677a9485c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:03:47.103186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fa130ff8-9cd0-4e65-b9e3-d1677a9485c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:03:47.116415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa130ff8-9cd0-4e65-b9e3-d1677a9485c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:03:47.118135 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa130ff8-9cd0-4e65-b9e3-d1677a9485c5 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:03:47.131783 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4507/8991] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:03:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:03:47.208003 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:03:47.222034 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25390d-6479-4318-853a-096c2f39104e service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:03:47.226091 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:03:47.226867 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:03:47.227243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:03:47.227496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:03:47.229366 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:03:47.258765 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:03:47.260210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:03:47.269218 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4485/8992] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:03:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:03:57.126792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:57.265728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=11 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:59.799261 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c09b-54db-4ea3-ae13-3a2a74a3eb30 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:03:59.799809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c09b-54db-4ea3-ae13-3a2a74a3eb30 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:03:59.799809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c09b-54db-4ea3-ae13-3a2a74a3eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:03:59.828266 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-e8b0c09b-54db-4ea3-ae13-3a2a74a3eb30 None None] MFA Rules not processed for user `3ec60ad271544bc99d7f394b9b06dd88`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:03:59.829543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-e8b0c09b-54db-4ea3-ae13-3a2a74a3eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:03:59.837169 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4508/8993] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:03:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:04:03.449535 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:04:03.468389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af895930-7f28-4687-92bc-5ffe1c7bb8d5 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:04:03.477459 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:04:03.478111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:04:03.478604 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:04:03.478891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:04:03.481054 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:04:03.506505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:04:03.507812 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:04:03.521296 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4486/8994] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:04:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:04:09.821256 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:10.095940 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0c09b-54db-4ea3-ae13-3a2a74a3eb30 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:04:10.110299 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0c09b-54db-4ea3-ae13-3a2a74a3eb30 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:04:10.117452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:04:10.118089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:04:10.118364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:04:10.118641 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:04:10.120613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:04:10.135143 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:04:10.136813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:04:10.149709 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4509/8995] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:04:13.515243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:20.146346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:26.978390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:05:26.990185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e895e-d851-492b-b168-b5240601e7f6 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:05:26.997399 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7341c8-56f7-4087-a560-78c117a42a7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:05:26.998015 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7341c8-56f7-4087-a560-78c117a42a7f service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:26.998298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7341c8-56f7-4087-a560-78c117a42a7f service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:26.998573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7341c8-56f7-4087-a560-78c117a42a7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.007648 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7341c8-56f7-4087-a560-78c117a42a7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:05:27.009288 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7341c8-56f7-4087-a560-78c117a42a7f service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:05:27.018684 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4487/8996] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:05:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:05:27.074919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:05:27.095364 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332a10-0ac0-459a-8a72-5c18ec40ad79 service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:05:27.099899 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:05:27.100499 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:27.100809 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:27.101079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.103186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:05:27.113037 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:05:27.114454 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:05:27.122727 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4510/8997] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:05:27.268316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cace3-92ab-48b9-b694-261bb4a9db70 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:27.268316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cace3-92ab-48b9-b694-261bb4a9db70 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:27.268560 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cace3-92ab-48b9-b694-261bb4a9db70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.328902 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-c37cace3-92ab-48b9-b694-261bb4a9db70 None None] MFA Rules not processed for user `065ad0a36b32453c98ca3ecda6f952a7`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:05:27.330694 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-c37cace3-92ab-48b9-b694-261bb4a9db70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:05:27.337102 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4488/8998] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 11:05:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:05:27.356152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:05:27.365070 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a94e96-39e2-4fb6-9a57-8a96262251bc service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:05:27.368973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc099f57-ca7b-4473-abe6-0a88ac8643dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:05:27.369601 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099f57-ca7b-4473-abe6-0a88ac8643dc service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:27.369872 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099f57-ca7b-4473-abe6-0a88ac8643dc service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:27.370139 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099f57-ca7b-4473-abe6-0a88ac8643dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.377820 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc099f57-ca7b-4473-abe6-0a88ac8643dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:05:27.379284 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc099f57-ca7b-4473-abe6-0a88ac8643dc service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:05:27.387272 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4511/8999] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:05:27.478828 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cace3-92ab-48b9-b694-261bb4a9db70 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:05:27.491584 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cace3-92ab-48b9-b694-261bb4a9db70 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:05:27.496705 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:05:27.497331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:27.497637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:27.497912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.500007 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:05:27.512148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:05:27.513768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:05:27.524011 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4489/9000] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:05:27.718962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9f62-1868-4037-8f1d-0c414b594590 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:27.719251 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9f62-1868-4037-8f1d-0c414b594590 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:27.719517 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e9f62-1868-4037-8f1d-0c414b594590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.740315 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4f4e9f62-1868-4037-8f1d-0c414b594590 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:05:27.741477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4f4e9f62-1868-4037-8f1d-0c414b594590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:05:27.751427 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4512/9001] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 11:05:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:05:27.761661 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:05:27.773049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969a80a-c222-40a6-a219-3f35c2411d07 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:05:27.777426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:05:27.778027 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:27.778393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:27.778670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.790238 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:05:27.791673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:05:27.802899 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4490/9002] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:05:27.869079 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e9f62-1868-4037-8f1d-0c414b594590 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:05:27.887774 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e9f62-1868-4037-8f1d-0c414b594590 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:05:27.895630 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:05:27.896243 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:27.896518 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:27.896825 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:27.898765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:05:27.911510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:05:27.913165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:05:27.923297 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4513/9003] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:05:33.407195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:05:33.417320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d822b4-077c-4915-8d5c-2722ed534e6e service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:05:33.421668 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:05:33.422287 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:05:33.422571 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:05:33.422841 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:05:33.431041 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:05:33.432484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:05:33.444943 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4491/9004] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:05:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:05:37.919280 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=25 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:43.441073 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=41 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:22.498126 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:06:22.509699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347f336-4668-4dfe-b578-644a68ae7ead service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:06:22.515819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980dcd8b-668c-45e9-9485-b8994052c405 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:06:22.516424 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dcd8b-668c-45e9-9485-b8994052c405 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:06:22.516711 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dcd8b-668c-45e9-9485-b8994052c405 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:06:22.516989 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dcd8b-668c-45e9-9485-b8994052c405 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:06:22.518956 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-980dcd8b-668c-45e9-9485-b8994052c405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:06:22.531411 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980dcd8b-668c-45e9-9485-b8994052c405 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:06:22.533014 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980dcd8b-668c-45e9-9485-b8994052c405 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:06:22.556492 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4514/9005] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:06:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:06:25.853631 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:06:25.862757 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc990be-8a4b-4b92-b880-1d603abc0e1a service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:06:25.866718 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:06:25.867549 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:06:25.868001 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:06:25.868277 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:06:25.877643 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:06:25.879301 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:06:25.889088 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4492/9006] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:06:25 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:06:28.607382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19012f-fbe4-4d45-81eb-4f17677863d1 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:06:28.607878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19012f-fbe4-4d45-81eb-4f17677863d1 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:06:28.607977 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19012f-fbe4-4d45-81eb-4f17677863d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:06:28.629430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-ae19012f-fbe4-4d45-81eb-4f17677863d1 None None] MFA Rules not processed for user `3ec60ad271544bc99d7f394b9b06dd88`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:06:28.631086 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-ae19012f-fbe4-4d45-81eb-4f17677863d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:06:28.638922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4515/9007] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:06:29.375341 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:06:29.390325 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ca43-486b-442a-b8c0-989bffd3a456 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:06:29.397538 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:06:29.398142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:06:29.398430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:06:29.398710 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:06:29.409818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:06:29.411419 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:06:29.424041 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4493/9008] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:06:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:06:38.623882 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:39.419363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=12 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:56.083945 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19012f-fbe4-4d45-81eb-4f17677863d1 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:07:56.096780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19012f-fbe4-4d45-81eb-4f17677863d1 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:07:56.100716 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:07:56.101346 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:07:56.101623 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:07:56.101916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:07:56.113494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:07:56.114929 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:07:56.137317 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4516/9009] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:07:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:07:57.348815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:07:57.359713 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcdf5b-4a02-4749-ae98-9af95d79161e service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:07:57.363789 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:07:57.364396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:07:57.364698 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:07:57.365028 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:07:57.377168 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:07:57.378610 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:07:57.390645 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4494/9010] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:07:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:08:03.784998 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:08:03.808870 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8b93-b69b-4b17-9283-320fc747643a service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:08:03.814878 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:08:03.815492 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:08:03.815771 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:08:03.816048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:08:03.825229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:08:03.826863 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:08:03.837408 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4517/9011] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:08:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:08:07.385749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:08:13.832737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=20 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:08:19.212975 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:08:19.236400 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16187378-75b8-42fa-9d82-9799f9c50514 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:08:19.244431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:08:19.245250 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:08:19.245524 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:08:19.245792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:08:19.255680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:08:19.257299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:08:19.268678 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4495/9012] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:08:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:08:29.263652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:07.164571 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:09:07.187400 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcf703-370a-4313-a869-aa1b9dc17b7f service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:09:07.193228 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860601a-3248-4f48-bc74-52fa57d36a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:09:07.193854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860601a-3248-4f48-bc74-52fa57d36a57 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:09:07.194134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860601a-3248-4f48-bc74-52fa57d36a57 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:09:07.194415 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860601a-3248-4f48-bc74-52fa57d36a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:09:07.204833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860601a-3248-4f48-bc74-52fa57d36a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:09:07.206449 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860601a-3248-4f48-bc74-52fa57d36a57 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:09:07.218061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4518/9013] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:09:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:09:09.568363 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:09:09.585564 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0a736-9ae5-4681-8a51-e9b4f16f619a service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:09:09.592456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:09:09.593136 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:09:09.593367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:09:09.593626 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:09:09.604964 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:09:09.606618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:09:09.620197 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4496/9014] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:09:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:09:17.213154 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=9 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:19.615307 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:21.023923 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eda43b-967a-40d1-a970-011612ae62ca None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:09:21.024451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eda43b-967a-40d1-a970-011612ae62ca None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:09:21.024451 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eda43b-967a-40d1-a970-011612ae62ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:09:21.045336 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-70eda43b-967a-40d1-a970-011612ae62ca None None] MFA Rules not processed for user `3ec60ad271544bc99d7f394b9b06dd88`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:09:21.047919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-70eda43b-967a-40d1-a970-011612ae62ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:09:21.055684 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4519/9015] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:09:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:09:21.418749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:09:21.444787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afce10-2eea-4d58-8dba-c778cd7634f4 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:09:21.453161 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:09:21.454097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:09:21.454530 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:09:21.454988 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:09:21.470244 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:09:21.472011 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:09:21.483687 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4497/9016] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:09:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:09:31.039815 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:31.479526 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:31.578988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eda43b-967a-40d1-a970-011612ae62ca None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:09:31.593547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eda43b-967a-40d1-a970-011612ae62ca None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:09:31.601652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:09:31.602671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:09:31.603196 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:09:31.603625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:09:31.616137 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:09:31.617880 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:09:31.629550 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4520/9017] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:09:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:09:41.624889 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:38.852967 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:38.854750 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:38.919203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c55c7-5f45-4a16-a409-b2234b4a02d4 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:38.923316 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:38.923930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:38.924209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:38.924481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:38.926499 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:38.936772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:38.938166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:38.946979 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4498/9018] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:10:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:10:39.900382 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:39.978230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffee68-fbd9-448d-ac4e-a7e7db9ae6df service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:39.984490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f814889-0965-4efd-910d-80d3ecdf3351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:39.985138 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814889-0965-4efd-910d-80d3ecdf3351 service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:39.985412 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814889-0965-4efd-910d-80d3ecdf3351 service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:39.985652 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814889-0965-4efd-910d-80d3ecdf3351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:39.987728 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-3f814889-0965-4efd-910d-80d3ecdf3351 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:40.083004 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f814889-0965-4efd-910d-80d3ecdf3351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:40.084800 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f814889-0965-4efd-910d-80d3ecdf3351 service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:40.103395 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4521/9019] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:10:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:10:48.942717 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=26 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:50.100687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=69 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:55.363854 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:55.365542 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:55.468700 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23ca2-85d9-43e8-9317-8c63e4d70cb2 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:55.473236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:55.473904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:55.474185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:55.474486 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] PATH_INFO: `/v3/users/3ec60ad271544bc99d7f394b9b06dd88` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:55.475657 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec60ad271544bc99d7f394b9b06dd88)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:55.477640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:55.533715 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec60ad271544bc99d7f394b9b06dd88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:55.534455 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec60ad271544bc99d7f394b9b06dd88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:55.541966 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec60ad271544bc99d7f394b9b06dd88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:55.551114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec60ad271544bc99d7f394b9b06dd88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:55.563692 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec60ad271544bc99d7f394b9b06dd88'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:55.570042 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4499/9020] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:10:55 2022] DELETE /identity/v3/users/3ec60ad271544bc99d7f394b9b06dd88 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:10:55.574894 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:55.575385 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:55.575493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:55.609879 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:10:55.611337 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:55.672268 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4522/9021] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:10:55.688559 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:55.690501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:55.751649 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc565-eaa7-4f2e-b53d-ce85f1b2f616 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:55.755833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:55.756475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:55.756795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:55.757063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:55.763620 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:55.767858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:55.769763 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:55.781333 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4500/9022] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:10:55.962197 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:55.963805 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:55.985203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c509-4e76-49cf-bbab-18698fdd5d54 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:55.990848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:55.991572 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:55.991826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:55.992113 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] PATH_INFO: `/v3/projects/55b42a416dd942e780e533e4c6f2a8b4` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:55.993307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55b42a416dd942e780e533e4c6f2a8b4)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:55.995227 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:56.046175 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b42a416dd942e780e533e4c6f2a8b4', 'request_id': 'req-f53bf831-3288-44d0-87ed-381dd5f11880'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.056212 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b42a416dd942e780e533e4c6f2a8b4', 'request_id': 'req-f53bf831-3288-44d0-87ed-381dd5f11880'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.063076 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b42a416dd942e780e533e4c6f2a8b4', 'request_id': 'req-f53bf831-3288-44d0-87ed-381dd5f11880'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.074627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] The token cache is being invalidate because project 55b42a416dd942e780e533e4c6f2a8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:10:56.074841 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.075483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.077036 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4523/9023] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:10:55 2022] DELETE /identity/v3/projects/55b42a416dd942e780e533e4c6f2a8b4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:10:56.081329 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:56.091482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac706be-0520-4a4a-a40c-1c82ee4d7cf5 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:56.153482 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:56.154123 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:56.154405 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:56.154678 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] PATH_INFO: `/v3/users/ade37a2bb2c240ba935ba974ecc3f31d` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:56.169078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade37a2bb2c240ba935ba974ecc3f31d)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:56.171081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:56.224696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade37a2bb2c240ba935ba974ecc3f31d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.225396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade37a2bb2c240ba935ba974ecc3f31d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.232679 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade37a2bb2c240ba935ba974ecc3f31d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.242111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade37a2bb2c240ba935ba974ecc3f31d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.254638 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-569b6a10-384a-4b0c-a094-48b424eafcc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade37a2bb2c240ba935ba974ecc3f31d'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.260790 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4501/9024] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:10:56 2022] DELETE /identity/v3/users/ade37a2bb2c240ba935ba974ecc3f31d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:10:56.548644 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:56.550348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:56.567264 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bf831-3288-44d0-87ed-381dd5f11880 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:56.630410 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:56.631055 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:56.631316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:56.632148 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] PATH_INFO: `/v3/projects/bda794173bca4e6588a47120125a4a68` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:56.642885 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda794173bca4e6588a47120125a4a68)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:56.644845 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:56.656578 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:56.697551 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda794173bca4e6588a47120125a4a68', 'request_id': 'req-7f4616eb-caad-4c69-b354-a9ca3b533a1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.706855 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda794173bca4e6588a47120125a4a68', 'request_id': 'req-7f4616eb-caad-4c69-b354-a9ca3b533a1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.713357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda794173bca4e6588a47120125a4a68', 'request_id': 'req-7f4616eb-caad-4c69-b354-a9ca3b533a1a'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.725339 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] The token cache is being invalidate because project bda794173bca4e6588a47120125a4a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:10:56.725625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.726288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7f4616eb-caad-4c69-b354-a9ca3b533a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:10:56.728023 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4524/9025] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:10:56 2022] DELETE /identity/v3/projects/bda794173bca4e6588a47120125a4a68 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:10:56.733782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8704fb-1c6a-4d8e-8ecc-1a653bef45e1 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:56.734268 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8704fb-1c6a-4d8e-8ecc-1a653bef45e1 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:56.734387 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8704fb-1c6a-4d8e-8ecc-1a653bef45e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:56.780160 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-ca8704fb-1c6a-4d8e-8ecc-1a653bef45e1 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:10:56.781433 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ca8704fb-1c6a-4d8e-8ecc-1a653bef45e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:56.835785 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:56.855581 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4502/9026] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 11:10:56.864397 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa32e2-ac32-45ea-ab6a-6fc5623305a0 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:56.864577 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa32e2-ac32-45ea-ab6a-6fc5623305a0 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:56.864907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa32e2-ac32-45ea-ab6a-6fc5623305a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:56.885533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-f8aa32e2-ac32-45ea-ab6a-6fc5623305a0 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:10:56.886742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-f8aa32e2-ac32-45ea-ab6a-6fc5623305a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:56.895814 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4525/9027] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:10:56.900406 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8704fb-1c6a-4d8e-8ecc-1a653bef45e1 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:56.916206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8704fb-1c6a-4d8e-8ecc-1a653bef45e1 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:56.924875 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:56.925787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:56.926113 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:56.926408 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:56.927116 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 11:10:56.927481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:56.930061 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:56.947038 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4503/9028] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 11:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 11:10:56.951730 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa32e2-ac32-45ea-ab6a-6fc5623305a0 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:56.962056 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa32e2-ac32-45ea-ab6a-6fc5623305a0 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:56.967973 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:56.968753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:56.968961 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:56.969268 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:56.969993 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:56.972163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:57.013457 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4526/9029] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 11:10:56 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:10:57.017940 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:57.030992 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8309b4-ff87-4dcd-b524-39a21fd26a46 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:57.092379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:57.093019 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:57.093297 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:57.093561 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:57.094185 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:57.096093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:57.119816 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4504/9030] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 11:10:57 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:10:57.124132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:57.135628 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333da51-c48e-4deb-921d-19b3575933b1 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:57.140528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:57.141155 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:57.141434 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:57.141709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:57.142348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:57.143962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:57.155118 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4527/9031] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:10:57.158911 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:57.169475 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeb5aa-176a-4e58-873e-2b4070cfb741 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:57.175359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0608a-9b7f-49c2-b266-54f0704aee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:57.175955 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0608a-9b7f-49c2-b266-54f0704aee67 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:57.176230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0608a-9b7f-49c2-b266-54f0704aee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:57.176496 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0608a-9b7f-49c2-b266-54f0704aee67 admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:57.177177 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0608a-9b7f-49c2-b266-54f0704aee67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:57.178912 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0608a-9b7f-49c2-b266-54f0704aee67 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:57.194545 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4505/9032] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:10:57.200554 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:57.214352 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83915357-b870-4e32-8058-e9115d0eaa82 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:57.220373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:57.221342 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:57.221643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:57.221912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] PATH_INFO: `/v3/projects/37db46b0eb6a4c7d938085b26d1968a7/users/90cea8d7a5494c45ba299df59797633f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:57.238106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37db46b0eb6a4c7d938085b26d1968a7, user_id=90cea8d7a5494c45ba299df59797633f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:57.241011 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:57.256737 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4528/9033] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:10:57 2022] PUT /identity/v3/projects/37db46b0eb6a4c7d938085b26d1968a7/users/90cea8d7a5494c45ba299df59797633f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:10:57.262110 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cef67-b9ed-4696-9c8c-ea0d1e2bc21b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:57.262394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cef67-b9ed-4696-9c8c-ea0d1e2bc21b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:57.262667 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cef67-b9ed-4696-9c8c-ea0d1e2bc21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:57.283271 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-012cef67-b9ed-4696-9c8c-ea0d1e2bc21b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:10:57.284462 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-012cef67-b9ed-4696-9c8c-ea0d1e2bc21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:57.349283 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4506/9034] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:10:57.359520 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:10:57.362743 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:10:57.423483 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09906a-9248-4d16-82e6-c817cb5d5784 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:10:57.427396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:10:57.428001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:10:57.428296 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:10:57.428737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:10:57.438200 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:10:57.439631 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:10:57.449340 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4529/9035] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:11:01.675619 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007e647-1626-42b0-ac23-f904860c89cb None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:01.676164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007e647-1626-42b0-ac23-f904860c89cb None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:01.676164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007e647-1626-42b0-ac23-f904860c89cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:01.711934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0007e647-1626-42b0-ac23-f904860c89cb None None] MFA Rules not processed for user `90cea8d7a5494c45ba299df59797633f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:11:01.713386 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0007e647-1626-42b0-ac23-f904860c89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:11:01.761591 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:01.788694 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4507/9036] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:11:01.793405 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:01.804386 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c4d5d-0ea6-4f0e-b34b-ce233ea54c6a service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:01.809531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:01.810407 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:01.810908 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:01.811501 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:01.812163 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:01.814161 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:01.850115 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4530/9037] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 11:11:01 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 11:11:01.851221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007e647-1626-42b0-ac23-f904860c89cb None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:01.863972 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007e647-1626-42b0-ac23-f904860c89cb None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:01.930490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:01.931457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:01.931736 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:01.932008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:01.932833 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:01.934870 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:01.960569 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4508/9038] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 11:11:01 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:11:01.965209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:01.976930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ec09f-3f07-488c-bc73-cdc65bdecbe7 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:01.981687 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:01.982338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:01.982619 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:01.982895 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:01.983539 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:01.985217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:01.987027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:01.999904 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4531/9039] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:11:02.004036 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:02.015039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97148f1b-0273-426e-adf6-e107be96eae0 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:02.020389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:02.021111 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:02.021401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:02.021675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] PATH_INFO: `/v3/projects/feda837323ad4605b8ab00a562896745/users/6b6d7e6ecc884875a1e94722acce3d71/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:02.040290 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] RBAC: Authorizing `identity:create_grant(project_id=feda837323ad4605b8ab00a562896745, user_id=6b6d7e6ecc884875a1e94722acce3d71, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:02.043018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:02.058665 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4509/9040] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:11:02 2022] PUT /identity/v3/projects/feda837323ad4605b8ab00a562896745/users/6b6d7e6ecc884875a1e94722acce3d71/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:11:02.062753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:02.073490 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea4dc-f58e-4666-a0ef-1e7a5169a012 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:02.133694 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e867b-bedb-4d2e-bad0-580578472f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:02.134347 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e867b-bedb-4d2e-bad0-580578472f95 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:02.134626 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e867b-bedb-4d2e-bad0-580578472f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:02.134901 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e867b-bedb-4d2e-bad0-580578472f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:02.135543 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e867b-bedb-4d2e-bad0-580578472f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:02.137359 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e867b-bedb-4d2e-bad0-580578472f95 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:02.148475 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4532/9041] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:11:02.153331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:02.164049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40f6e1-dd46-4031-b819-aa9aa2da282a admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:02.168860 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:02.169540 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:02.169811 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:02.170079 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] PATH_INFO: `/v3/projects/feda837323ad4605b8ab00a562896745/users/6b6d7e6ecc884875a1e94722acce3d71/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:02.172049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] RBAC: Authorizing `identity:create_grant(project_id=feda837323ad4605b8ab00a562896745, user_id=6b6d7e6ecc884875a1e94722acce3d71, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:02.174724 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:02.185379 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4510/9042] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:11:02 2022] PUT /identity/v3/projects/feda837323ad4605b8ab00a562896745/users/6b6d7e6ecc884875a1e94722acce3d71/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:11:06.896005 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24565674-7848-4fda-9fe5-009fe55b0fec None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:06.896471 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24565674-7848-4fda-9fe5-009fe55b0fec None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:06.896625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24565674-7848-4fda-9fe5-009fe55b0fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:06.929114 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-24565674-7848-4fda-9fe5-009fe55b0fec None None] MFA Rules not processed for user `6b6d7e6ecc884875a1e94722acce3d71`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:11:06.930332 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-24565674-7848-4fda-9fe5-009fe55b0fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:11:07.012807 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4533/9043] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:11:07.031675 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:07.034243 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:11:07.146365 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d84979-1924-471d-8ff8-383446d3871e admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:07.152708 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:07.153337 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:07.153614 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:07.153882 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:07.177072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:07.221322 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:07.222980 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:07.234045 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4511/9044] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 11:11:08.318917 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61791966-4a06-4ce1-b474-0413db7000e7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:08.319442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61791966-4a06-4ce1-b474-0413db7000e7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:08.319442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61791966-4a06-4ce1-b474-0413db7000e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:08.341396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-61791966-4a06-4ce1-b474-0413db7000e7 None None] MFA Rules not processed for user `90cea8d7a5494c45ba299df59797633f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:11:08.342625 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-61791966-4a06-4ce1-b474-0413db7000e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:11:08.356225 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4534/9045] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:11:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:11:08.684359 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:08.686088 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:11:08.748173 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b3697-4265-441a-8a03-e44c290d3eb8 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:08.753025 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:08.753644 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:08.753923 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:08.754198 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:08.756042 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:11:08.766792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:08.768235 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:08.777743 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4512/9046] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:11:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:11:11.096479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61791966-4a06-4ce1-b474-0413db7000e7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:11.107094 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61791966-4a06-4ce1-b474-0413db7000e7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:11.110877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:11.111637 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:11.111918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:11.112192 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:11.117442 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:11:11.128229 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:11.129828 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:11.138861 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4535/9047] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:11:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:11:18.774282 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=44 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:20.228139 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:11:20.238491 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408315ef-e1aa-4545-a6cc-818972885a40 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:11:20.243327 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:11:20.244008 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:11:20.244293 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:11:20.244548 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:11:20.255439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:11:20.256914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:11:20.267361 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4513/9048] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:11:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:11:21.135031 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=74 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:30.263210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:09.788848 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:09.793427 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:09.941018 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d082d-46c3-4bd7-aa98-81ea3782702e service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:09.951999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:09.953089 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:09.953547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:09.953997 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:09.957598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:09.977826 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:09.979854 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:09.994205 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4536/9049] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:13:17.859749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:17.861401 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:17.964699 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86397db9-d5b7-4649-bd2e-e063ce6d1b74 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:17.968533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:17.969166 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:17.969439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:17.969709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:17.971596 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:17.985389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:17.987094 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:17.999945 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4514/9050] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:13:18.204232 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:18.214995 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4463e29-36ca-48c6-b061-70655da25fcf service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:18.219560 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c9193-d7d2-4af3-b273-d9107adfcbe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:18.220164 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9193-d7d2-4af3-b273-d9107adfcbe4 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:18.220440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9193-d7d2-4af3-b273-d9107adfcbe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:18.220751 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9193-d7d2-4af3-b273-d9107adfcbe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:18.222778 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-5a4c9193-d7d2-4af3-b273-d9107adfcbe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:18.238330 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c9193-d7d2-4af3-b273-d9107adfcbe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:18.239767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c9193-d7d2-4af3-b273-d9107adfcbe4 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:18.250924 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4537/9051] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:13:24.102317 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:24.103913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:24.119206 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e2dfc-8d62-4536-a1f9-7d03d9f87e15 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:24.125583 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:24.126225 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:24.126501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:24.126768 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:24.128838 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:24.141609 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:24.143439 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:24.155722 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4515/9052] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:13:28.246335 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=65 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:31.279099 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db249f2-d7ad-4a29-80e1-69051f3765fb None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:31.279720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db249f2-d7ad-4a29-80e1-69051f3765fb None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:31.279720 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db249f2-d7ad-4a29-80e1-69051f3765fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:31.335446 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-4db249f2-d7ad-4a29-80e1-69051f3765fb None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:13:31.336718 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-4db249f2-d7ad-4a29-80e1-69051f3765fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:31.407584 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4538/9053] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 11:13:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:13:31.456345 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:31.457770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:31.551234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f293e-2fc2-4da6-b42c-8bd3c2b1e222 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:31.556565 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:31.557202 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:31.557494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:31.557764 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:31.566469 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:31.568335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:31.576977 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4516/9054] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:13:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:13:41.404432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=40 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:41.573507 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=87 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:43.875871 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aa6d0-a7f8-49cc-b606-c0c8face7c84 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:43.876461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aa6d0-a7f8-49cc-b606-c0c8face7c84 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:43.876461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aa6d0-a7f8-49cc-b606-c0c8face7c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:43.923493 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-503aa6d0-a7f8-49cc-b606-c0c8face7c84 None None] MFA Rules not processed for user `3c8ed3cf7a5742808029d336ebc784fa`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:13:43.924753 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-503aa6d0-a7f8-49cc-b606-c0c8face7c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:43.934850 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4539/9055] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Sep 27 11:13:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:13:43.947519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:43.958203 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bf51a-4cc6-4985-ad72-bb23d82cdfdb service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:43.962413 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:43.963039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:43.963379 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:43.963658 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:43.976009 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:43.977681 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:43.990678 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4517/9056] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:13:44.063128 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aa6d0-a7f8-49cc-b606-c0c8face7c84 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:44.077629 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aa6d0-a7f8-49cc-b606-c0c8face7c84 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:44.081729 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:44.082338 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:44.082615 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:44.082914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:44.084798 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:44.099834 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:44.101281 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:44.109941 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4540/9057] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 11:13:44.205999 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:44.215891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605357bf-c0f2-4b47-8220-851d12e41b25 service glance] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:44.220784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6334e-b030-40ec-ac38-17557028a316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:44.221389 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6334e-b030-40ec-ac38-17557028a316 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:44.221665 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6334e-b030-40ec-ac38-17557028a316 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:44.221934 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6334e-b030-40ec-ac38-17557028a316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:44.224065 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-eca6334e-b030-40ec-ac38-17557028a316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:44.239312 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6334e-b030-40ec-ac38-17557028a316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:44.240806 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6334e-b030-40ec-ac38-17557028a316 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:44.252412 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4518/9058] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:13:44.316448 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:44.329369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42c8be-a806-4d57-b546-c1b730009cb8 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:44.336779 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:44.337362 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:44.337634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:44.337903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:44.339709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:44.351552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:44.353123 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:44.363526 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4541/9059] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:13:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:13:47.239704 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f850839-59c3-4c19-b0f0-838b443f026f None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:47.240199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f850839-59c3-4c19-b0f0-838b443f026f None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:47.240326 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f850839-59c3-4c19-b0f0-838b443f026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:47.260394 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-0f850839-59c3-4c19-b0f0-838b443f026f None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:13:47.261582 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0f850839-59c3-4c19-b0f0-838b443f026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:13:47.269110 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4519/9060] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 11:13:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:13:47.282350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:13:47.297749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faffc37-e2b7-4e3d-811d-9a0cfc0d50ab service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:13:47.303819 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:13:47.304421 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:13:47.304737 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:13:47.305028 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:13:47.313622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:13:47.315198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:13:47.323237 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4542/9061] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:13:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:13:57.255501 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=15 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:57.325355 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=27 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:14:04.649656 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c318ae-76ba-4c0f-89ee-eca3a60d5a47 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:14:04.650083 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c318ae-76ba-4c0f-89ee-eca3a60d5a47 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:14:04.650169 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c318ae-76ba-4c0f-89ee-eca3a60d5a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:14:04.670473 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-79c318ae-76ba-4c0f-89ee-eca3a60d5a47 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:14:04.671754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-79c318ae-76ba-4c0f-89ee-eca3a60d5a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:14:04.679494 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4520/9062] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 11:14:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:14:04.692559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:14:04.707649 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833196da-8bc0-466f-93c6-534b9087b1fc service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:14:04.715331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:14:04.716106 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:14:04.716388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:14:04.716651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:14:04.725714 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:14:04.727305 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:14:04.735988 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4543/9063] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 11:14:14.665494 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:14:14.732806 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:42.627504 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c318ae-76ba-4c0f-89ee-eca3a60d5a47 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:15:42.644115 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c318ae-76ba-4c0f-89ee-eca3a60d5a47 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:15:42.650787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f17255-1736-4f1b-b6ea-47f000ce9188 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:15:42.651481 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f17255-1736-4f1b-b6ea-47f000ce9188 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:15:42.651829 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f17255-1736-4f1b-b6ea-47f000ce9188 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:15:42.652164 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f17255-1736-4f1b-b6ea-47f000ce9188 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:15:42.654039 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-18f17255-1736-4f1b-b6ea-47f000ce9188 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:15:42.667734 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f17255-1736-4f1b-b6ea-47f000ce9188 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:15:42.669497 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f17255-1736-4f1b-b6ea-47f000ce9188 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:15:42.681660 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4521/9064] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:15:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:15:42.776621 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:15:42.804371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328013a-c7b7-41a5-9265-84bbc7f541c3 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:15:42.809373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:15:42.810083 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:15:42.810487 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:15:42.810777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:15:42.813545 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:15:42.843180 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:15:42.844797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:15:42.854673 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4544/9065] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:15:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:15:47.607234 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:15:47.607711 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:15:47.607825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:15:47.628498 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] MFA Rules not processed for user `90cea8d7a5494c45ba299df59797633f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:15:47.629802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:15:47.637683 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4522/9066] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:15:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:15:51.055432 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:15:51.069064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ceae9-f440-49c0-b14e-1a020c449c27 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:15:51.074509 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:15:51.075210 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:15:51.075579 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:15:51.075857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:15:51.077663 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:15:51.090289 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:15:51.091772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:15:51.102767 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4545/9067] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:15:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:15:53.621187 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:15:53.622671 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:15:53.640570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8110473-529d-45b0-9299-f14e0c66f309 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:15:53.647567 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:15:53.648411 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:15:53.648730 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:15:53.649012 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:15:53.650815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:15:53.664432 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:15:53.666097 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:15:53.678378 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4523/9068] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:15:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:15:59.576341 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:15:59.589955 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76624602-6294-44cd-95e5-efd2a1717751 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:15:59.595645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:15:59.596472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:15:59.596773 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:15:59.597047 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:15:59.598916 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:15:59.614091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:15:59.615813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:15:59.627528 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4546/9069] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:15:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:16:03.672472 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=15 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:16:09.622715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=27 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:16:23.536426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:16:23.546945 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45751271-7ff9-4d02-9a11-91edf712fcc9 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:16:23.551230 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:16:23.551832 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:16:23.552106 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:16:23.552374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:16:23.560885 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:16:23.563076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:16:23.571698 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4524/9070] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:16:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:16:33.568152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:08.844597 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:18:08.866038 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57509dde-ebe1-4303-8883-d554cc1cbd9d service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:18:08.872408 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:18:08.873277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:18:08.873566 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:18:08.873836 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:18:08.882258 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:18:08.883846 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:18:08.893174 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4547/9071] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:18:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:18:11.850067 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:18:11.875076 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a179c-1639-4944-b1bf-b86d2f0da5f3 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:18:11.881192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:18:11.881803 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:18:11.882081 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:18:11.882358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:18:11.891415 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:18:11.893018 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:18:11.903601 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4525/9072] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:18:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:18:17.443371 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:18:17.453209 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335774e-0aed-4a5f-b90d-cc17b2aaae1c service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:18:17.457222 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:18:17.457792 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:18:17.458091 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:18:17.458350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:18:17.468699 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:18:17.470241 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:18:17.481080 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4548/9073] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:18:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:18:18.670566 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:18:18.681209 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbe971-291f-4f5f-a3ce-2b23cf9c33dc service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:18:18.686318 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:18:18.686904 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:18:18.687179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:18:18.687444 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:18:18.701038 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:18:18.703298 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:18:18.716028 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4526/9074] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:18:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:18:22.468512 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:18:22.477938 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c7f78-6bec-4377-b5bf-b4a2020f4d05 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:18:22.482460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:18:22.483068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:18:22.483344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:18:22.483614 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:18:22.491237 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:18:22.492602 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:18:22.502479 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4549/9075] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:18:24.532159 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:18:24.543769 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ecead-78eb-4034-8b81-e4e4b1f88224 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:18:24.549175 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:18:24.549786 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:18:24.550124 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:18:24.550562 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:18:24.559453 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:18:24.561045 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:18:24.571122 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4527/9076] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:18:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:18:32.497066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=22 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:34.567154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=22 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:03.313686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:03.322537 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bca0b2-fd58-47c3-b8c8-780143eeb5d0 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:03.326303 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:03.326903 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:03.327181 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:03.327450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:03.408562 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:03.410048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:03.420006 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4550/9077] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 11:20:06.369232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:06.391100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d15cf-5333-42e2-a681-d5c71538c942 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:06.406409 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:06.407434 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:06.407891 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:06.408353 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:06.411852 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:06.446727 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:06.449846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:06.472739 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4528/9078] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:20:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:20:09.694831 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:09.708325 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fc7dd-1090-4b38-b1ac-7375552bde8c service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:09.720369 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:09.721217 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:09.721494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:09.721759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:09.723717 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:09.735569 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:09.737317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:09.747562 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4551/9079] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:20:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:20:09.759178 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:09.771474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057661b-05a4-445f-8870-53beb23339bc service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:09.776232 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:09.776979 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:09.777167 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:09.777613 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] PATH_INFO: `/v3/users/90cea8d7a5494c45ba299df59797633f` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:09.778772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90cea8d7a5494c45ba299df59797633f)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:09.780742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:09.832787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90cea8d7a5494c45ba299df59797633f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:09.833457 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90cea8d7a5494c45ba299df59797633f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:09.840762 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90cea8d7a5494c45ba299df59797633f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:09.850519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90cea8d7a5494c45ba299df59797633f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:09.863824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90cea8d7a5494c45ba299df59797633f'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:09.870321 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4529/9080] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:20:09 2022] DELETE /identity/v3/users/90cea8d7a5494c45ba299df59797633f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:20:10.055073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:10.065888 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a6927-f364-4bdd-b76f-2a63fa0d43bd service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:10.125307 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:10.125946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:10.126218 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:10.126494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] PATH_INFO: `/v3/projects/37db46b0eb6a4c7d938085b26d1968a7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:10.127589 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37db46b0eb6a4c7d938085b26d1968a7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:10.129533 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:10.179350 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37db46b0eb6a4c7d938085b26d1968a7', 'request_id': 'req-a0b37d16-6720-4035-964f-4968f36d6517'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.188833 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37db46b0eb6a4c7d938085b26d1968a7', 'request_id': 'req-a0b37d16-6720-4035-964f-4968f36d6517'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.195216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37db46b0eb6a4c7d938085b26d1968a7', 'request_id': 'req-a0b37d16-6720-4035-964f-4968f36d6517'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.207877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] The token cache is being invalidate because project 37db46b0eb6a4c7d938085b26d1968a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:20:10.208165 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.208982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.210591 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4552/9081] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:20:10 2022] DELETE /identity/v3/projects/37db46b0eb6a4c7d938085b26d1968a7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:20:10.214815 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:10.216300 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:10.230174 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c369f74-480b-44c2-9cc3-1048f91938f3 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:10.299172 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:10.299824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:10.300100 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:10.300371 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] PATH_INFO: `/v3/users/6b6d7e6ecc884875a1e94722acce3d71` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:10.301618 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6d7e6ecc884875a1e94722acce3d71)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:10.303523 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:10.358179 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6d7e6ecc884875a1e94722acce3d71'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.359308 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6d7e6ecc884875a1e94722acce3d71'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.366721 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6d7e6ecc884875a1e94722acce3d71'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.370372 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:10.378245 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6d7e6ecc884875a1e94722acce3d71'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.391242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e15db1e3-8d6b-431c-b3cb-30e4a31917f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6d7e6ecc884875a1e94722acce3d71'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.397462 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4530/9082] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:20:10 2022] DELETE /identity/v3/users/6b6d7e6ecc884875a1e94722acce3d71 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:20:10.587982 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:10.589477 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:10.607943 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b37d16-6720-4035-964f-4968f36d6517 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:10.618374 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:10.671481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:10.672147 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:10.672452 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:10.672765 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] PATH_INFO: `/v3/projects/feda837323ad4605b8ab00a562896745` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:10.673914 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] RBAC: Authorizing `identity:delete_project(project_id=feda837323ad4605b8ab00a562896745)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:10.675867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:10.726029 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feda837323ad4605b8ab00a562896745', 'request_id': 'req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.735777 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feda837323ad4605b8ab00a562896745', 'request_id': 'req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.742294 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feda837323ad4605b8ab00a562896745', 'request_id': 'req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.754690 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] The token cache is being invalidate because project feda837323ad4605b8ab00a562896745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:20:10.754988 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.755706 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-6812cd2a-da9b-4da0-8a12-e22d0ad3f71b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:20:10.757344 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4553/9083] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:20:10 2022] DELETE /identity/v3/projects/feda837323ad4605b8ab00a562896745 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:20:10.762670 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c731ddd-683e-48fe-bc93-19685716586d None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:10.763154 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c731ddd-683e-48fe-bc93-19685716586d None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:10.763240 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c731ddd-683e-48fe-bc93-19685716586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:10.797749 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-4c731ddd-683e-48fe-bc93-19685716586d None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:20:10.799026 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4c731ddd-683e-48fe-bc93-19685716586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:10.863184 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4531/9084] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 11:20:10.871203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715c83-c229-4121-9444-222fb28e5765 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:10.871491 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715c83-c229-4121-9444-222fb28e5765 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:10.871764 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715c83-c229-4121-9444-222fb28e5765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:10.892472 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-69715c83-c229-4121-9444-222fb28e5765 None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:20:10.893715 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-69715c83-c229-4121-9444-222fb28e5765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:10.902261 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4554/9085] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:20:10.906771 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c731ddd-683e-48fe-bc93-19685716586d None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:10.917221 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c731ddd-683e-48fe-bc93-19685716586d None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:10.922358 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:10.923247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:10.923533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:10.923788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] PATH_INFO: `/v3/domains` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:10.924474 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=104131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 11:20:10.924804 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:10.926430 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:10.938699 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4532/9086] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 27 11:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 11:20:10.943027 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69715c83-c229-4121-9444-222fb28e5765 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:10.952553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69715c83-c229-4121-9444-222fb28e5765 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:10.957357 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:10.958001 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:10.958279 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:10.958553 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:10.959183 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:10.961145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:10.991751 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4555/9087] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 11:20:10 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:20:10.996385 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:11.008784 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41120448-75de-4738-bc7e-7b170a21d138 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:11.068214 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:11.068883 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:11.069162 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:11.069431 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:11.070052 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:11.071959 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:11.096983 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4533/9088] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 11:20:10 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:20:11.101708 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:11.118430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbae286-798e-4e34-85aa-9c70640e5d97 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:11.126160 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:11.126767 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:11.127043 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:11.127316 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:11.127962 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:11.129598 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:11.140165 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4556/9089] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:20:11.144579 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:11.155063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c3357-711d-4fb5-b552-63989da61c6b admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:11.160321 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba42e74-f172-4dd6-82c5-630f03d6476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:11.160941 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42e74-f172-4dd6-82c5-630f03d6476c admin admin] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:11.161212 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42e74-f172-4dd6-82c5-630f03d6476c admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:11.161471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42e74-f172-4dd6-82c5-630f03d6476c admin admin] PATH_INFO: `/v3/roles` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:11.162134 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba42e74-f172-4dd6-82c5-630f03d6476c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:11.163707 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba42e74-f172-4dd6-82c5-630f03d6476c admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:11.175227 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4534/9090] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:20:11.179450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:11.190481 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d953fef-59e5-43cc-8c2e-ff67a45ca711 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:11.195111 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:11.195770 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:11.196046 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:11.196317 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] PATH_INFO: `/v3/projects/34d2d2279cd14736ba71123e751c9aeb/users/8bf66c7321304c91840efc2cbeb52a4f/roles/a0597e11955a4725ae4938bb442c75f6` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:11.211930 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d2d2279cd14736ba71123e751c9aeb, user_id=8bf66c7321304c91840efc2cbeb52a4f, role_id=a0597e11955a4725ae4938bb442c75f6)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:11.214638 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:11.230472 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4557/9091] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:20:11 2022] PUT /identity/v3/projects/34d2d2279cd14736ba71123e751c9aeb/users/8bf66c7321304c91840efc2cbeb52a4f/roles/a0597e11955a4725ae4938bb442c75f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:20:11.237152 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565fd01-467c-43b9-8353-fefd17acb92b None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:11.237541 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565fd01-467c-43b9-8353-fefd17acb92b None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:11.237957 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565fd01-467c-43b9-8353-fefd17acb92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:11.262335 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-e565fd01-467c-43b9-8353-fefd17acb92b None None] MFA Rules not processed for user `7b007883f8f945d6b63ec8a2752bb899`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:20:11.263471 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e565fd01-467c-43b9-8353-fefd17acb92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:11.303747 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:11.335050 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4535/9092] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:20:11.345041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:11.346944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:11.391396 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:11.418676 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f467db-4f42-4dd1-ba62-1e6be22381f6 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:11.424020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:11.424633 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:11.424950 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:11.425219 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:11.435344 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:11.436877 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:11.446487 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4558/9093] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:20:15.649931 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb9d0-25e8-4699-ad4b-b58999096132 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:15.650425 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb9d0-25e8-4699-ad4b-b58999096132 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:15.650674 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb9d0-25e8-4699-ad4b-b58999096132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:15.682533 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-189eb9d0-25e8-4699-ad4b-b58999096132 None None] MFA Rules not processed for user `8bf66c7321304c91840efc2cbeb52a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:20:15.683770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-189eb9d0-25e8-4699-ad4b-b58999096132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:15.751758 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4536/9094] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:20:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:20:15.756600 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:15.767320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843dd15-c2dc-412c-a627-e49a6003a012 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:15.772736 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:15.773390 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:15.773669 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:15.773942 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] PATH_INFO: `/v3/projects` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:15.774592 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:15.776536 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:15.809120 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4559/9095] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 27 11:20:15 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:20:15.810621 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189eb9d0-25e8-4699-ad4b-b58999096132 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:15.825937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189eb9d0-25e8-4699-ad4b-b58999096132 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:15.896049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:15.896878 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:15.897226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:15.897572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] PATH_INFO: `/v3/users` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:15.898396 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:15.900490 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:15.926741 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4537/9096] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 27 11:20:15 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 11:20:15.931041 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:15.942054 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4b901-06b4-46fa-b34d-d4c05918bc5e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:15.955650 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:15.956348 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:15.956627 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:15.956918 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:15.957622 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:15.960912 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:15.973502 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4560/9097] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:20:15.977798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:15.988532 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5dd98-7961-419f-a64c-2816376659b7 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:15.993913 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:15.994818 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:15.994969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:15.995347 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] PATH_INFO: `/v3/projects/fb4612e5b3b2472a981ecaa66ee6f6b3/users/9eaa548214aa4a5f92c70938faf443f7/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:16.017273 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4612e5b3b2472a981ecaa66ee6f6b3, user_id=9eaa548214aa4a5f92c70938faf443f7, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:16.020572 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:16.036421 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4538/9098] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:20:15 2022] PUT /identity/v3/projects/fb4612e5b3b2472a981ecaa66ee6f6b3/users/9eaa548214aa4a5f92c70938faf443f7/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:20:16.040686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:16.051842 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958d696-0fcc-44e6-ba39-97f25c98abf2 admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:16.121996 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ec9ac-3ddc-41b2-a5cd-09eef4a69d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:16.122613 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ec9ac-3ddc-41b2-a5cd-09eef4a69d74 admin admin] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:16.123158 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ec9ac-3ddc-41b2-a5cd-09eef4a69d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:16.123478 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ec9ac-3ddc-41b2-a5cd-09eef4a69d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:16.124494 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ec9ac-3ddc-41b2-a5cd-09eef4a69d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:16.126131 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ec9ac-3ddc-41b2-a5cd-09eef4a69d74 admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:16.140915 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4561/9099] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 11:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 11:20:16.145543 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:16.155937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbdcd9-6ede-4795-8769-019eb9887157 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:16.160597 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:16.161304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] REQUEST_METHOD: `PUT` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:16.161581 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:16.161848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] PATH_INFO: `/v3/projects/fb4612e5b3b2472a981ecaa66ee6f6b3/users/9eaa548214aa4a5f92c70938faf443f7/roles/53aff494b6b44d5da69d62199ff0d237` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:16.163742 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4612e5b3b2472a981ecaa66ee6f6b3, user_id=9eaa548214aa4a5f92c70938faf443f7, role_id=53aff494b6b44d5da69d62199ff0d237)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:16.166426 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:16.177526 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4539/9100] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 27 11:20:16 2022] PUT /identity/v3/projects/fb4612e5b3b2472a981ecaa66ee6f6b3/users/9eaa548214aa4a5f92c70938faf443f7/roles/53aff494b6b44d5da69d62199ff0d237 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:20:20.887700 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d276a55-1b9e-4f21-a9cb-31f9edd238c8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:20.888184 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d276a55-1b9e-4f21-a9cb-31f9edd238c8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:20.888272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d276a55-1b9e-4f21-a9cb-31f9edd238c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:20.932801 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-1d276a55-1b9e-4f21-a9cb-31f9edd238c8 None None] MFA Rules not processed for user `9eaa548214aa4a5f92c70938faf443f7`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:20:20.934312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-1d276a55-1b9e-4f21-a9cb-31f9edd238c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:21.070413 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4562/9101] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:20:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:20:21.090943 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:21.094456 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:21.142813 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:21.217696 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44656e-607e-418e-a2b1-f0a5bb385742 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:21.223556 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:21.224195 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:21.224484 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:21.224791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:21.293798 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:21.295547 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:21.306192 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4540/9102] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:20:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:20:22.373198 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:22.379443 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:22.379741 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:22.400686 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] MFA Rules not processed for user `8bf66c7321304c91840efc2cbeb52a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:20:22.401935 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:22.410131 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4563/9103] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:20:22.752183 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:22.818858 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1dd329-3962-428f-a84c-92a5b8562a68 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:22.822479 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:22.823104 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:22.823377 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:22.823685 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:22.825403 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:22.835598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:22.837101 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:22.846720 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4541/9104] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:20:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:20:32.394755 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:32.842279 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=77 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:33.066793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:20:33.068320 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:20:33.081391 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6cd86-9f99-42a2-9928-1b9afbfe0bc7 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:20:33.085813 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224bb7d9-2628-424a-bcce-cb1cc5e3411e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:20:33.086418 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bb7d9-2628-424a-bcce-cb1cc5e3411e service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:20:33.086697 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bb7d9-2628-424a-bcce-cb1cc5e3411e service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:20:33.086966 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bb7d9-2628-424a-bcce-cb1cc5e3411e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:20:33.096689 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224bb7d9-2628-424a-bcce-cb1cc5e3411e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:20:33.098129 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224bb7d9-2628-424a-bcce-cb1cc5e3411e service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:20:33.107788 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4564/9105] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:20:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:20:43.103431 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:21:26.543563 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:21:26.553049 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e40cbe-75b6-44ea-a93a-bcf3c9669d68 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:21:26.558064 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:21:26.558673 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:21:26.558952 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:21:26.559222 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:21:26.561194 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:21:26.571630 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:21:26.573090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:21:26.581226 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4542/9106] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:21:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:21:36.577787 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:34.670642 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f00e2-9c66-4dbd-9f47-a2c81e48d2c9 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:34.671186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f00e2-9c66-4dbd-9f47-a2c81e48d2c9 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:34.671186 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f00e2-9c66-4dbd-9f47-a2c81e48d2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:34.703236 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-127f00e2-9c66-4dbd-9f47-a2c81e48d2c9 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:22:34.704460 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-127f00e2-9c66-4dbd-9f47-a2c81e48d2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:34.770759 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4565/9107] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 11:22:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:22:34.823320 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:22:34.825595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:34.910920 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63bb32-3521-4f96-abb7-aeaf22030cbb service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:22:34.916849 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c9ff6-da6f-45de-8160-91b0fe6d1df2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:22:34.917460 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c9ff6-da6f-45de-8160-91b0fe6d1df2 service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:34.917754 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c9ff6-da6f-45de-8160-91b0fe6d1df2 service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:34.917998 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c9ff6-da6f-45de-8160-91b0fe6d1df2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:34.926733 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c9ff6-da6f-45de-8160-91b0fe6d1df2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:22:34.928304 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c9ff6-da6f-45de-8160-91b0fe6d1df2 service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:22:34.936882 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4543/9108] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:22:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:22:35.525134 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f00e2-9c66-4dbd-9f47-a2c81e48d2c9 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:22:35.549921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f00e2-9c66-4dbd-9f47-a2c81e48d2c9 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:22:35.559857 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:22:35.560749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:35.561127 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:35.561747 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:35.563968 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:35.606907 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:22:35.609333 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:22:35.628024 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4566/9109] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:22:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:22:44.245296 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aaa1f-eea3-4e51-a719-fba9ff0828bc None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:44.245824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aaa1f-eea3-4e51-a719-fba9ff0828bc None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:44.245824 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aaa1f-eea3-4e51-a719-fba9ff0828bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:44.265781 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-7c1aaa1f-eea3-4e51-a719-fba9ff0828bc None None] MFA Rules not processed for user `8bf66c7321304c91840efc2cbeb52a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:22:44.266984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-7c1aaa1f-eea3-4e51-a719-fba9ff0828bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:44.275171 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4544/9110] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 27 11:22:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:22:44.955772 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:22:44.957323 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:45.028066 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191214-a7db-4682-bacf-732ed51b617c service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:22:45.032949 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:22:45.033555 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:45.033830 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:45.034110 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:45.035787 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:45.047550 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:22:45.049176 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:22:45.059967 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4567/9111] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:22:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:22:46.469570 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f086f-07da-48cc-b0e5-da50933f8122 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:46.470143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f086f-07da-48cc-b0e5-da50933f8122 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:46.470143 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f086f-07da-48cc-b0e5-da50933f8122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:46.492977 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b08f086f-07da-48cc-b0e5-da50933f8122 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:22:46.494215 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b08f086f-07da-48cc-b0e5-da50933f8122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:46.502062 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4545/9112] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 11:22:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:22:46.514388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:22:46.526310 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1b3f6-880e-47d8-9f7a-e5294a903957 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:22:46.531583 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:22:46.532216 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:46.532510 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:46.532816 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:46.541634 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:22:46.543226 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:22:46.551997 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4568/9113] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:22:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:22:56.487424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=38 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:56.548456 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=78 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:56.986181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f086f-07da-48cc-b0e5-da50933f8122 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:22:56.997712 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f086f-07da-48cc-b0e5-da50933f8122 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:22:57.001593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:22:57.002236 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:57.002516 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:57.002792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:57.004636 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:57.015598 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:22:57.017381 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:22:57.027410 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4546/9114] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:22:57.696757 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:22:57.698476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:57.717068 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc8661-3900-4338-bba3-b79de82792b7 service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:22:57.723670 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:22:57.724440 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:22:57.725479 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:22:57.725791 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:22:57.727793 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:22:57.742780 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:22:57.744531 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:22:57.757428 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4569/9115] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:22:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:23:07.022808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:07.752190 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:07.771184 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:23:07.785390 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808036e-07f6-41c4-9e29-cd9a1bbdd8d7 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:23:07.791672 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8ed3cf7a5742808029d336ebc784fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:23:07.792299 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:23:07.792791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:23:07.793068 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:23:07.794950 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:23:07.808640 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:23:07.810264 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:23:07.821130 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4547/9116] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:23:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:23:16.805272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:23:16.841651 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e78869-37ec-4fb4-959d-2eaf4e42191a service nova] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:23:16.846999 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928465cc-9075-44ed-b978-00316278cfe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:23:16.847612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928465cc-9075-44ed-b978-00316278cfe6 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:23:16.847886 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928465cc-9075-44ed-b978-00316278cfe6 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:23:16.848145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928465cc-9075-44ed-b978-00316278cfe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:23:16.849928 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-928465cc-9075-44ed-b978-00316278cfe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:23:16.899430 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928465cc-9075-44ed-b978-00316278cfe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:23:16.901048 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928465cc-9075-44ed-b978-00316278cfe6 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:23:16.911315 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4570/9117] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:23:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:23:17.816788 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:26.907132 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=28 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:36.081611 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:23:36.083782 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:23:36.152595 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b51545-1954-4cd7-bb08-e25bd1bb7de6 service nova] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:23:36.158196 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:23:36.158810 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:23:36.159090 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:23:36.159369 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:23:36.161265 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:23:36.172625 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:23:36.174233 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:23:36.184617 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4548/9118] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:23:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:23:46.180280 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=28 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:13.925524 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:13.926033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:13.926033 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:13.985220 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] MFA Rules not processed for user `ed91b2c83d57419dac50595f57fc3411`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:25:13.986540 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:14.053077 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4571/9119] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Sep 27 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:25:14.062770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:14.078014 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a743-0ef1-46fc-83cd-a9197012070c service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:14.083118 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5007b4-922e-4d7f-a10f-cc68e2d7b525 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:14.083728 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5007b4-922e-4d7f-a10f-cc68e2d7b525 service glance] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:14.084006 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5007b4-922e-4d7f-a10f-cc68e2d7b525 service glance] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:14.084278 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5007b4-922e-4d7f-a10f-cc68e2d7b525 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:14.086142 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-ee5007b4-922e-4d7f-a10f-cc68e2d7b525 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:14.101072 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5007b4-922e-4d7f-a10f-cc68e2d7b525 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:14.102511 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5007b4-922e-4d7f-a10f-cc68e2d7b525 service glance] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:14.113923 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4549/9120] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:25:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:25:21.519173 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:21.520867 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:21.531892 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8022a3-1ff3-46d1-bef6-d3287230a021 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:21.536919 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed91b2c83d57419dac50595f57fc3411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:21.537521 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:21.537797 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:21.538064 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:21.547709 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:21.549205 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:21.559196 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4572/9121] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:25:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:25:22.069393 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4629-8e2e-4856-8663-6e8a3f62cb98 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:22.069869 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4629-8e2e-4856-8663-6e8a3f62cb98 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:22.069969 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4629-8e2e-4856-8663-6e8a3f62cb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:22.124210 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-b60e4629-8e2e-4856-8663-6e8a3f62cb98 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:25:22.125477 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-b60e4629-8e2e-4856-8663-6e8a3f62cb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:22.144256 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4550/9122] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 27 11:25:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:25:22.191203 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:22.214719 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38dc3c-4edb-47b1-a7d1-a0686e82034e service glance] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:22.221042 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:22.221946 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:22.222230 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:22.222513 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:22.231851 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:22.233508 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:22.243133 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4573/9123] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:25:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:25:32.120424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d309369-970e-4ae4-ac06-c0c05cf062f7 None None] REQUEST_METHOD: `POST` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:32.121063 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d309369-970e-4ae4-ac06-c0c05cf062f7 None None] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:32.121291 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d309369-970e-4ae4-ac06-c0c05cf062f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:32.143519 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.auth.core [None req-1d309369-970e-4ae4-ac06-c0c05cf062f7 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:25:32.144848 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-1d309369-970e-4ae4-ac06-c0c05cf062f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:32.153077 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4551/9124] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 11:25:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:25:32.166480 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:32.178136 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d6464-370d-48a2-98e1-8d43606bbb5c service swift] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:32.184152 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6978104-b077-4cea-af12-0751fcb0b51c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:32.185020 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978104-b077-4cea-af12-0751fcb0b51c service swift] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:32.185568 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978104-b077-4cea-af12-0751fcb0b51c service swift] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:32.186072 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978104-b077-4cea-af12-0751fcb0b51c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:32.197211 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6978104-b077-4cea-af12-0751fcb0b51c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:32.199073 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6978104-b077-4cea-af12-0751fcb0b51c service swift] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:32.211221 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4574/9125] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:25:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:25:33.956802 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d309369-970e-4ae4-ac06-c0c05cf062f7 None None] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:33.967029 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d309369-970e-4ae4-ac06-c0c05cf062f7 None None] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:33.970965 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:33.971505 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:33.971937 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:33.972343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:33.974285 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:33.988792 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:33.990247 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:34.001849 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4552/9126] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:25:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:25:37.603952 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569c595-32ad-480b-bbe0-ba98fdf239e8 None None] REQUEST_METHOD: `POST` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:37.604455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569c595-32ad-480b-bbe0-ba98fdf239e8 None None] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:37.604455 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569c595-32ad-480b-bbe0-ba98fdf239e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:37.612789 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=61 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:37.628272 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.auth.core [None req-b569c595-32ad-480b-bbe0-ba98fdf239e8 None None] MFA Rules not processed for user `9afc5552272c4928be73cb202945c8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=104132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 11:25:37.629556 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-b569c595-32ad-480b-bbe0-ba98fdf239e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:37.638174 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4575/9127] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 27 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 11:25:37.652865 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:37.658770 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=34 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:37.668035 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6874e-574d-4983-9f17-e7df0b530d65 service neutron] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:37.675791 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6863dfc2c0143a88060bbf5f0e146df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:37.676375 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:37.676652 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:37.676946 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:37.685261 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:37.686853 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:37.695014 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4553/9128] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 27 11:25:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 11:25:41.781040 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569c595-32ad-480b-bbe0-ba98fdf239e8 None None] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:41.790915 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569c595-32ad-480b-bbe0-ba98fdf239e8 None None] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:41.795141 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '065ad0a36b32453c98ca3ecda6f952a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:41.795742 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:41.796017 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:41.796288 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:41.873074 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:41.874511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:41.896061 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4576/9129] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:25:44.913628 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:44.928190 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7b77-bef9-4600-9175-3b9bfd89c04d service swift] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:44.933546 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:44.934181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:44.934465 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:44.934739 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:44.936601 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:44.955398 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:44.957331 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:44.978414 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4554/9130] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:25:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:25:48.314643 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:48.326645 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc1046-8509-464b-aa22-475638ce8235 service neutron] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:48.331559 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:48.332511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:48.332511 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:48.332844 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] PATH_INFO: `/v3/users/8bf66c7321304c91840efc2cbeb52a4f` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:48.334270 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf66c7321304c91840efc2cbeb52a4f)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:48.336277 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:48.385441 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf66c7321304c91840efc2cbeb52a4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.386145 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf66c7321304c91840efc2cbeb52a4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.396118 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf66c7321304c91840efc2cbeb52a4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.403866 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf66c7321304c91840efc2cbeb52a4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.404552 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf66c7321304c91840efc2cbeb52a4f'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.422140 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4577/9131] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:25:48 2022] DELETE /identity/v3/users/8bf66c7321304c91840efc2cbeb52a4f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:25:48.628573 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:48.642242 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb038d2-069a-4b90-867e-fb973ec9de59 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:48.706153 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:48.706864 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:48.707165 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:48.707440 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] PATH_INFO: `/v3/projects/34d2d2279cd14736ba71123e751c9aeb` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:48.708593 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d2d2279cd14736ba71123e751c9aeb)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:48.710536 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:48.759367 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d2d2279cd14736ba71123e751c9aeb', 'request_id': 'req-e53b8b5b-6541-4806-95e6-50121324ccba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.766213 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d2d2279cd14736ba71123e751c9aeb', 'request_id': 'req-e53b8b5b-6541-4806-95e6-50121324ccba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.778580 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d2d2279cd14736ba71123e751c9aeb', 'request_id': 'req-e53b8b5b-6541-4806-95e6-50121324ccba'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.806890 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] The token cache is being invalidate because project 34d2d2279cd14736ba71123e751c9aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:25:48.807181 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.807834 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.809450 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4555/9132] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:25:48 2022] DELETE /identity/v3/projects/34d2d2279cd14736ba71123e751c9aeb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 11:25:48.813735 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:48.815188 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:48.829528 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df910a85-36c7-4087-af4c-865f11b1cd3e admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:48.872331 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:48.889759 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:48.890393 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:48.890671 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:48.890944 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] PATH_INFO: `/v3/users/9eaa548214aa4a5f92c70938faf443f7` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:48.892462 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eaa548214aa4a5f92c70938faf443f7)` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:48.894612 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:48.943963 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eaa548214aa4a5f92c70938faf443f7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.944618 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eaa548214aa4a5f92c70938faf443f7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.953505 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eaa548214aa4a5f92c70938faf443f7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.960927 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eaa548214aa4a5f92c70938faf443f7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.961570 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.notifications [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eaa548214aa4a5f92c70938faf443f7'} {{(pid=104132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:48.979922 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4578/9133] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 11:25:48 2022] DELETE /identity/v3/users/9eaa548214aa4a5f92c70938faf443f7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 11:25:49.169984 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:25:49.171760 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:25:49.187257 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b8b5b-6541-4806-95e6-50121324ccba admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:25:49.241286 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:49.250093 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b007883f8f945d6b63ec8a2752bb899', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d515506957d41b7867232158c5ad868', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:25:49.251226 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:25:49.251590 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:25:49.252274 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] PATH_INFO: `/v3/projects/fb4612e5b3b2472a981ecaa66ee6f6b3` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:25:49.254192 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4612e5b3b2472a981ecaa66ee6f6b3)` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:25:49.256114 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:25:49.306343 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4612e5b3b2472a981ecaa66ee6f6b3', 'request_id': 'req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:49.312392 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4612e5b3b2472a981ecaa66ee6f6b3', 'request_id': 'req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:49.322808 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4612e5b3b2472a981ecaa66ee6f6b3', 'request_id': 'req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78'} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:49.334078 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] The token cache is being invalidate because project fb4612e5b3b2472a981ecaa66ee6f6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=104131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 11:25:49.334368 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:49.335044 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.notifications [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=104131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 11:25:49.336538 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4556/9134] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 11:25:49 2022] DELETE /identity/v3/projects/fb4612e5b3b2472a981ecaa66ee6f6b3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 11:25:58.976439 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:59.330772 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:28:20.750837 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:28:20.752547 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:28:20.839312 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e853603-c032-4015-a7b4-3e22dfa0b08b admin admin] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:28:20.904921 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:28:20.905450 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:28:20.905723 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:28:20.905986 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:28:20.907947 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.fernet_utils [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:28:20.919838 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:28:20.921465 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:28:20.931417 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4579/9135] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:28:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:28:30.927734 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=62 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:28:44.109914 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:28:44.111342 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:28:44.127506 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ef53d-61e6-4baa-939f-cf58fe8b5f78 admin admin] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:28:44.134551 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:28:44.135148 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:28:44.135424 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:28:44.135690 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:28:44.137930 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.fernet_utils [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=104131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 11:28:44.155709 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:28:44.157374 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:28:44.168000 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4557/9136] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:28:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:28:54.163871 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=6 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:33:27.838461 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] Authenticating user token {{(pid=104132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:33:27.850109 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f782da3-dae0-4b47-a293-6273ba65c255 service placement] Validating token access rules against request {{(pid=104132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:33:27.858383 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2064e-69a5-41eb-b29c-ef87c38bd1d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:33:27.859144 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2064e-69a5-41eb-b29c-ef87c38bd1d3 service placement] REQUEST_METHOD: `GET` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:33:27.859476 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2064e-69a5-41eb-b29c-ef87c38bd1d3 service placement] SCRIPT_NAME: `/identity` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:33:27.859822 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2064e-69a5-41eb-b29c-ef87c38bd1d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:33:27.869749 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2064e-69a5-41eb-b29c-ef87c38bd1d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:33:27.871373 ncdv-master-1 devstack@keystone.service[104132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2064e-69a5-41eb-b29c-ef87c38bd1d3 service placement] RBAC: Authorization granted {{(pid=104132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:33:27.881654 ncdv-master-1 devstack@keystone.service[104132]: [pid: 104132|app: 0|req: 4580/9137] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:33:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:33:37.877388 ncdv-master-1 devstack@keystone.service[104132]: DEBUG dbcounter [-] [104132] Writing DB stats keystone:SELECT=6 {{(pid=104132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:33:46.058637 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] Authenticating user token {{(pid=104131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 11:33:46.106680 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ae5da-ea34-4c6b-9910-9d4d4137edd6 service placement] Validating token access rules against request {{(pid=104131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 11:33:46.113653 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9b47f-e6b4-4c1e-80a0-2992225582cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19f8118e581143ffb079e2b1aea3870d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=104131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 11:33:46.114256 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9b47f-e6b4-4c1e-80a0-2992225582cd service placement] REQUEST_METHOD: `GET` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 11:33:46.114531 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9b47f-e6b4-4c1e-80a0-2992225582cd service placement] SCRIPT_NAME: `/identity` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 11:33:46.114795 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9b47f-e6b4-4c1e-80a0-2992225582cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=104131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 11:33:46.124199 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9b47f-e6b4-4c1e-80a0-2992225582cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 11:33:46.125825 ncdv-master-1 devstack@keystone.service[104131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9b47f-e6b4-4c1e-80a0-2992225582cd service placement] RBAC: Authorization granted {{(pid=104131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 11:33:46.136083 ncdv-master-1 devstack@keystone.service[104131]: [pid: 104131|app: 0|req: 4558/9138] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 27 11:33:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 11:33:56.131846 ncdv-master-1 devstack@keystone.service[104131]: DEBUG dbcounter [-] [104131] Writing DB stats keystone:SELECT=20 {{(pid=104131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}